So, on the one hand we have live dynamic services and organizations… well there’s three things. Live dynamic services. Organizations that run those services, so that you are referring to a hierarchy. You are referring to a system of control. An organization, a government, that represents an organized evolving group. And on the other hand you have artefacts. You have human intellectual artefacts that have the ability to be completely independent from any system of human control. They are out there in the Platonic realm somehow. And shouldn’t in fact be referred to by an organization. They should be referred to in a way that is intrinsic to the intellectual content, that arises out of the intellectual content! I think that is an inevitable and very important way forward, and where this… where I saw that this was a problem was dealing with a man by the name of Nahdmi Auchi. A few years ago was listed by one of the big business magazines in the UK as the fifth richest man in the UK. In 1980 left Iraq. He’d grown rich under Saddam Hussein’s oil industry. And is alleged by the Italian press to be involved in a load of arms trading there, he has over two hundred companies run out of his Luxembourg holding unit. And several that we discovered in Panama. He had infiltrated the British Labour political establishment to the degree that the 20th business birthday in London he was given a painting signed by 146 members Commons including Tony Blair. He’s the same guy who was the principal financier of Tony Rezko. Tony Rezko was the financier and fundraiser of Rod Blagoyevich, from Chicago. Convicted of corruption. Tony Rezko has been convicted of corruption. And Barack Obama. He was the intermediary who helped Barack Obama buy one of his houses and then the money not directly for the house but it bouyed up Tony Rezko’s finances came from that… [indistinct]. So during the - this is detail, but it will get to a point. During the 2008 presidential primaries a lot of attention was turned to Barack Obama by the US press, unsurprisingly. And so it started to look into his fundraisers, and discovered Tony Rezko, and then they just started to turn their eyes towards Nadhmi Auchi. Auchi then hired Carter Ruck, a rather notorious firm of London libel solicitors, whose founder, Carter Ruck, has been described as doing for freedom of speech what the Boston strangler did for door to door salesmen.
And he started writing letters to all of the London papers who had records of his 2003 extradition to France and conviction for corruption in France over the Elf-Acquitaine scandal. Where he had been involved in taking kickbacks on selling the invaded Kuwaiti governments’ oil refineries in order to fund their operations while Iraq had occupied it. So the Guardian pulled three articles from 2003. So they were five years old. They had been in the Guardian’s archive for 5 years. Without saying anything. If you go to those URLs you will not see “removed due to legal threats.” You will see “page not found.” And one from the Telegraph. And a bunch from some American publications. And bloggers, and so on. Important bits of history, recent history, that were relevant to an ongoing presidential campaign in the United States were pulled out of the intellectal record. They were also pulled out of the Guardian’s index of articles. So why? The Guardian’s published in print, and you can go to the library and look up those articles. They are still there in the library. How would you know that they were there in the library? To look up, because they are not there in the Guardian’s index. Not only have they ceased to exist, they have ceased to have ever existed. Which is the modern implementation of Orwell’s dictum that he controls the present controls the past and he who controls the past controls the future. Because the past is stored physically in the present. All records of the past. This issue of preserving politically salient intellectual content while it is under attack is central to what WikiLeaks does – because that is what we are after! We are after those bits that people are trying to suppress, because we suspect, usually rightly, that they’re expending economic work on suppressing those bits because they perceive that they are going to induce some change.”
]]>Following are the answers to these questions, as well as some additional, unanswered, questions and issues the segment could have raised but didn’t.
Q: [Bitcoin has] no power to tax
A: The implicit assumption of this statement is that currencies gain value because they are the exclusive means of remitting taxes to the issuing government. If all citizens are required by law to pay taxes in this currency, then come tax time they must convert wealth from other forms into this currency to pay taxes, creating demand for that currency. Since tax time in most developed countries is a ubiquitous, ongoing, daily occurence (sales tax), then there is a ubiquitous, ongoing, daily demand for that currency, giving it constant steady value.
However, asking that question implies a narrow view of that aspect of currencies. While modern, government-issued, government debt-backed currencies typiclly gain value from their status as taxation legal tender, that is not the only way. Anything that creates demand for a currency can give it value.
For example, the US dollar is also the primary currency for purchasing oil in the world. Any country that needs to buy oil (almost all of them), must convert either its own currency or other wealth into dollars to buy oil, increasing demand for the dollar, and hence its value.
Another way in which value is imparted to the US Dollar, it is the only currency by which one may purchase what has for the past half century (until the recent financial crisis at least) been considered the risk-free rate of return - short-term US Treasury Bills.
Financial instability in Europe or other parts of the world tends to result in the movement of wealth from investment in riskier assets to investment in US Treasuries for this reason, and this fallback status grants the dollar another source of demand and value not available to other currencies.
Another example is gold, simply because it induces a singular psychological response in most humans: charitably - awe at its luster and beauty, uncharitably - desire, avarice. In either case, gold induces demand among humans purely psychologically, independent of any functional use it may have, and this is the source from which its value as a currency is hisorically derived.
Bitcoin may not derive value from its status as exclusive medium of tax remitance, but that has no bearing on whether it is a currency or not. Rather, it must meet the three criteria for definition of a currency:
Bitcoin is certainly a medium of exchange with global scope and efficient payment and transfer capabilities, moreso than any traditional currency, the latter part even moreso than the US Dollar. Transactions are, like cash, non-reversible, requiring buyers to truly caveat emptor, but transactions anywhere in the world are completed in a matter of minutes and locked in (confirmed) in just a few hours.
Bitcoin provides units of account, the smallest being the Satoshi, or 0.00000001 BTC (eight decimal places, or in scientific notation, 1.0 x 10-8).
Finally, is Bitcoin a store of value? Truthfully the jury is still out on that. Some pertinent questions, that the Bloomberg presenters could have been asking here:
Will its price stablize? Volatility is generally not associated with store of value.
Will it appreciate over time? Depreciation is generally not associated with store of value. However, this is a test every debt-backed, inflationary currency, including US Federal Reserve Notes, fails, at least as measured per unit of currency.
What are the long-term technical risks of relying completely on the Internet as both system and database? The Internet is remarkably technically resilient, and bitcoin is designed to be so as well, but are there systemic risks that could undermine it? For example, wide-scale power outages due to solar storms, or control by malicious actor of more than 50% of the computing power of the network. What are the safeguards and redundancies against such events?
What are the long-term regulatory risks? Will governments find ways to regulate bitcoin into uselessness to protect their monopoly on currency issuence?
These are all pertinent critques that a professional news organization like Bloomberg should be asking of Bitcoin right now.
Q: Who finances it?
A: No one. It doesn’t work like traditional banks and currencies, so stop thinking of it in those terms. Bitcoin is FOSS - Free Open Source Software. It is created by hobbyists and concerned citizens working together over the internet, the same as GNU/Linux, BSD Unix, and scads of other highly successful open source software projects that run most of the Internet’s infrastructure.
“Free” has an intentional double meaning here:
Free as in Liberty - you can download the software, modify it, borrow from it, redistribute it, whatever you want. Bitcoin is licensed under the clear, simple MIT License. Free as in zero restrictions what you can do with it.
free as in no monetary cost - you pay nothing to download and use it. No upfront fee, no recurring subscription fees, no hidden taxes down the road, not even any embedded advertising. Completely free.
Having said that, bitcoin does require an infrastructure: the Internet, plus participants in the Bitcoin network who have decided to set up “mining” operations by which new Bitcoins are created.
The Internet obviously was originally financed by taxpayers, and ongoing financing comes from the citizens and organizations who pay fees to connect and use it.
The Bitcoin Mining operations are funded mainly by the innovators and early adopters who saw value in Bitcoin early, and purchased the computer hardware necessary for calculating the hash algorithm that creates bitcoins. Their ROI for that is a set amount of bitcoins every time their system finds an answer to the hash algorithm (lots of answers possible, all relatively difficult to find).
So, if Bitcoin is “financed” by anyone, it is the total computing effort of all participants mining bitcoins, from individuals to pools.
Q: sounds like an ameoba
R: An amoeba? Srsly bro? You’re on Bloomberg News, not Fox News. Lets try to maintain a little higher standard of rigor here.
Q: Is there any fundamental justifcation for appeal of bitcoin
R: Yes, that its value is derived in such a way that it can’t be compromised by a central bank bailing out a fraudulent banking system. The mechanism by which Bitcoin is created and operates is transparent, open, and F/free to the entire world, and more difficult for any one entity or cartel to control and manipulate for the benefit of some at the expense of others.
Whereas prevailing national currency regimes are run by obfuscated operations of the world’s central banks and their parent organization, all of which are privately owned institutions (though usually government-chartered, with the exception of the BIS) whose interests may conflict with those of the citizens of the countries in which they operate. The recent financial crisis has highlighted this inherent conflict of interest, in which national currencies, national debt, and national savings are all being risked to “socialize private losses”.
As a result, despite bitcoin’s relative immaturity and current price volatility,
some people perceive it as an option for a long term store of value relative to
the debt-backed government currencies in which they no longer trust the issuer to
act in citizens’ best interest. Bitcoin’s intrinsic math and mechanisms are set, and
are the same now as they will be 20 years from now, making it a viable long term bet
for those who understand the algorithm, the infrastructure, and the general risks.
These people have been early adopters, driving up demand for bitcoin, and as a result
its value and appeal to others.
Q: I look at Bitcoin and I think to myself, [like] Farmville, this is virtual nonsense
R: I never played Farmville, so can’t comment extensively on it. But, Farmville’s virtual currency is issued by a for-profit private company, on a for-profit social network, for the purpose of making those companies a profit.
Bitcoin is issued by an open, transparent network in which anyone in the world can freely participate in the creation of the currency (and share in the profits that go along with it, at least up until the last bitcoin is created at 21 million).
And, can you even buy anything outside of Farmville with Farmville’s currency?
Just because two things are “virtual” does not mean they are comparable.
Q: “And the fear of the algorithm, think of all the problems we’ve seen, if you talk to any critics of HFT, and they talk about you don’t want to be in a situation where you’re dependent on an algorithm, and we’re talking about actual currency here.”
A: We’re dependent on algorithms all the time. What do you think runs the entire Internet, financial exchanges, the power grid, mobile phone systems, etc. HFT is a special case of many disparate, complex, opaque, proprietary algorithms competing against each other, interacting in sometimes emergent, unpredictable ways (Flash Crashes).
Bitcoin is a case of a single (or few), relatively simple algorithm/s, working together in the same system to solve a well-defined mathematical problem that just wasn’t solvable until the invention of the Internet, public key crytpography, and the Bittorrent distributed communication protocol.
Again, just because two things are “algorithms” does not mean they are comparable.
Q: We will hear a lot more about it, but will it be in the context of things like Zynga’s share price that reflects people’s lack of confidence in “virtual” things?
A: Again, see answer above re: Farmville.
]]>First, don’t think of bitcoin as a currency, but rather as a ledger. It is an electronic ledger, of which a copy is kept on every participant’s computer in the network, and all of which are continually updated, reconciled, and synchronized in real-time. Every participant can make entries in this ledger, which records transactions of a certain amount of currency from one participant to another participant, and every one of those entries is then propagated to the network in realtime, so that every copy on every computer is updated near simultaneously and all copies of the ledger are kept synchronized. The official term for this public, distributed ledger is the ‘blockchain’ (which you can see here), and it uses Bittorrent technology to keep all copies synchronized.
You can also think of Bitcoin not as a currency but as a general solution to a difficult algorithmic problem in the field of distributed systems, colloquially known as Byzantine Fault Tolerance, the Byzantine Generals Problem, or the Two Generals Problem.
The fun, informal statement of the problem is this: Imagine during the Byzantine era, there is a fortified city-state, Byzantium, that contains within its walls wealth beyond the wildest dreams and avarice of its neighbors. It is surrounded by ten other city-states, each of which is also wealthy, but nowhere near as wealthy as Byzantium. All ten neighbors covet Byzantium’s wealth, and wish to invade and take it for their own.
However, Byzantium is so well defended that none of the neighboring city-states is strong enough to succeed at an invasion. Any lone invasion attempt would fail and result in the total annihilation of the invader’s army, leaving it vulnerable to invasion and pillaging by any of the other nine, all of which also covet each other’s wealth and constantly scheme against each other. Moreover, Byzantium’s defenses are so strong, that it will take more than half the of the ten neighbors’ armies attacking simultaneously to overcome.
So, if six or more neighboring armies attack together, they will succeed and gain the riches of Byzantium. However, if one or more betrays the rest of the group by agreeing to invade and then holding back while the others attack, so that only five or less armies attack simultaneously, then all the attacking armies will be annihilated, and then pillaged by the remaining neighbors, including the ones that betrayed them. It is a network of untrusting parties that must nevertheless work together to accomplish a shared mission.
Moreover, the only way that the ten neighbors can communicate and coordinate the attack time is by sending messengers on horseback amongst themselves. They can’t all meet in one place for a conference (none of the kings trust the others outside the safety of his own fortress or army). However, they can send as many messengers, as frequently as they want, and to whomever they want, at any time. Each message contains something like “I will attack on Day 4 at 0600, will you join me?”.
If the recipient agrees, they attach a signed/certified/sealed/verified response to the original message, then send out copies of the newly combined message to all 9 neighbors again, asking each of them to do the same. The eventual objective is to have them all agreeing to the same time by adding all ten of their seals onto the same message chain. The result will be a message chain with ten seals all agreeing to the same time, and probably several discarded message chains that contained a few seals but not all.
However, the problem is that if each city-state sends one messenger apiece to each of the other nine, then that’s 10 cities sending 9 messages each, 90 total in transit at any one time, and each city receiving 9 messages each, ostensibly each with different attack times. Further, some of the cities will agree to more than one attack time, intending to betray the originator, so they will rebroadcast more than one message chain. This system quickly degenerates into a completely untrustworthy tangle of conflicting messages and attack times.
Bitcoin solves this problem with a simple (in hindsight) modification to the system - it adds a cost to sending messages, which slows down the rate of message passing, and does so with an element of randomness that ensures that only one city-state will be able to broadcast at a time. The cost that it adds is a ‘proof of work’, and it is based on calculating a random hash algorithm. A hash is an algorithm that does nothing more than takes some input data and performs a calculation on it that results in a 64-digit string of random numbers and letters, like this:
d70298566aa2f1a66d892dc31fedce6147b5bf509e28d29627078d9a01a8f86b
In the case of bitcoin, the input data consists of the entire ledger/blockchain up to the current point in time. And while a single hash value can be calculated nearly instantaneously with today’s computers, only a hash value where the first thirteen characters are zero is accepted by bitcoin as the ‘proof of work’. Such a thirteen-zero hash code is extremely improbable and rare, and currently takes the entire bitcoin network about 10 minutes of searching before one is found. Billions of invalid hash codes are generated before one machine in the network randomly lands on a valid one, and this is the ‘proof of work’ that slows down the rate of message passing and makes the whole system viable. Here’s what they look like:
f51d0199c4a6d9f6da230b579d850698dff6f695b47d868cc1165c0ce74df5e1
d70298566aa2f1a66d892dc31fedce6147b5bf509e28d29627078d9a01a8f86b
119c506ceaa18a973a5dbcfbf23253bc970114edd1063bd1288fbba468dcb7f8
… millions or billions more like this until finally a valid one …
000000000000084b6550604bf21ad8a955b945a0f78c3408c5002af3cdcc14f5
The machine (or city state in our analogy) that discovers the next valid hash code is the one that gets to take all previous messages, append its own, along with its signature/seal/etc, and broadcast out to the rest of the network. As soon as the other machines in the network receive and verify this new thirteen-zero hash code and attached messages, they stop their current calculations, update their copy of the ledger with new version, and start calculating the hash again by feeding the newly updated ledger/blockchain to the hashing algorithm. The hash calculation race is reset from a new starting point and begun anew. Thus the network constantly syncs itself so that all computers on it are using the same version of the ledger for their hashing calculations.
In the meantime, there are about 10 minutes in between successful hash code discoveries and blockchain updates (this is intentional, and the algorithm difficulty is adjusted every two weeks to make sure it takes the network a constant 10 minutes to find a new valid hash code). During that 10m, participants on the network are sending messages and making transactions, and since every machine on the network is using the same ledger, all those transactions and messages are entered into every copy of the ledger across the entire network. When the blockchain is updated and synced across the network, all transactions that were entered into the blockchain during the previous 10 minutes are also updated and synced. Hence the distributed transaction record is reconciled and synchronized among all participants.
Last, when individuals enter a transaction into the network, they use standard public key cryptography tools built into the bitcoin client program to sign it with their public key and the public key of the recipient. This is the counterpart to the ‘seal’ that the Byzantine Generals signed and verified their messages with. Hence the hash calculation rate limiter combined with public key cryptography, turn an untrusting network into a trusting one, allowing all the participants to agree (on an attack time, or a series of transactions, domain name records, political voting system, or anything else that needs distributed agreement).
So the crux of why Bitcoin is special: It represents a solution to a difficult algorithmic problem that was not possible until a confluence of historical events:
The creation of the Internet.
The invention of public key cryptography.
The invention of the P2P Bitorrent Protocol. Bittorrent was originally developed for sharing many files between relatively small subsets of the users on the network, but Bitcoin repurposes it to share a single file amongst all users.
The realization that adding a simple time delay into the system along with public key cryptography to verify individual transactions could solve the problem.
If some of the most brilliant ideas are simple in hindsight, #4 at least certainly qualifies, though overall the project has stood on the shoulders of giants.
Finally, this solution to the Byzantine Generals Problem can be repurposed to any domain where lack of trust on a distributed network is the core problem. As already mentioned, people are building a distributed Domain Name System for the Internet, and are working distributed voting systems for use in political elections (no website yet), for starters. If people thought that mere filesharing disrupted the world, the Bitcoin solution and protocol are just beginning to open the floodgates.
]]>Election 2012 is shaping up to be interesting for a variety of reasons, but not least because of two reasons involving forecasting:
There are roughly three categories of data-driven, systematic forecasting methodologies:
This is the first US presidential election since 2000 that is actually close and contested, and hence the first ever closely contested presidential election that features mature probability models of the electoral college - alongside tuned, tweaked, evolved and relatively sophisticated polls and betting markets - all attempting to divine the outcome.
One characteristic of this close election is that idiosyncracies, and perhaps flaws and biases, in these forecasting methods are more likely to be exposed under the higher general uncertainty than in the past two presidential election cycles.
Whereas uncontested or landslide elections tend to result in all the forecasting methods roughly agreeing with each other, close elections can result in disagreement among them - as is happening now - and hence a moment of truth: who is most accurate (or to put it in more scientific terms, who is less wrong).
As of writing this, the major national polls generally show Romney eeking out a slight lead in the national popular vote, the electoral college models show a stubborn lead for Obama, and the betting markets generally favor Obama while trending Romney (with some unusual activity which could hurt their credibility even if they correctly predict the election).
So who is least wrong? We’ll know in about two weeks, and can look forward to plenty of postmortem analysies of the analysies afterwards.
]]>And some choice followup commentary:
Dan M.Grove, OK Nate, I really think you overstate your case. I’ll give an easy counter-examples to your statement that narrow theories are better than broad theories: the standard model of physics. From it, all weak intereactions and all of quantum electromagnetism can be derived. And classical electromagnitism has been derived from quantum electrodynamics. These theories have been verified millions of time. They are the basis for our understanding a wide range of technology, from electromagnetism to computers to lasers to quantum optics. You rightly point out that medical papers are often not reproduced. That is because they only need a 95% confidence level (or 2 sigma) to be published. And, since null results are rarely published, its easy to have 19 random unpublished result, and 1 random published one. When charm was found, it was published with a 5-sigma statistical signal. It was reproduced immediately. These are broad ranging theories that have been well identifed. If you want a political science result to be verified, it should be something that isn’t just something that can be restated N different ways, that has stable results when you change the question slightly. In particular, it is a big plus for the theory if you offer a skeptical colleauge the right to reset the question and then recomute the results. Then the results should have less than a 1 in 100 chance of being found randomly. 1 in 1000 would be much better.
Nate Silver, Brooklyn, NY Dan, You make some excellent points. In particular, one of the things I found very problematic when I began to examine the elections “fundamentals” models is that they were not very robust to small change in assumptions. Replace an economic variable with one that is normally closely correlated with it, and you will get a substantially different result in certain elections. But I think one needs to be careful about drawing analogies between the physical and the social sciences. One of the things that characterized Tetlock’s hedgehogs was that they saw the political system as more analogous to a noncomplex (perhaps even Newtonian) physical system than the foxes did. This can sometimes cut the other way as well. For instance, there are some criticisms of global warming forecasts that would be reasonably compelling if they were tantamount to social science predictions, but don’t work as well when the causality of the greenhouse effect, etc. is relatively well understood (although, I certainly don’t claim that global warming forecasts are above criticism or without their problematic elements). Then again, it’s interesting that a lot of Bayesian probability theory really originated with Laplace, who thought that even though the mechanisms understanding the universe were extremely regular, our ability to measure and understand them precisely might not be.
Richard, NY Nate, I think you are confusing complexity with uncertainty in formulation. The weather/climate system is immensely complex and involves a massive number of interactions and feedbacks. Most of those interactions however are reasonably well understood and can be derived from the laws of physics. Social science models on the other hand are complex but also subject to fundamental lack of understanding of the basic interactions involved. This manifests itself in the parametric sensitivity you mentioned. Weather models are not subject to anywhere near this degree of parametric uncertainty even though they are proably more complex. Indeed the largest numerical models in the world are weather/climate models.
Dan M.Grove, OK Thanks for your reply Nate. You are absolutely right that facile comparisons between physical sciences and social sciences are extremely dangerous. But, when you included medicine, I wanted to point out that broad statements can have enormous predictive power. Ecconomics and social sciences are causally dense. So, it is hard to make broad quantitative statements. Still, I don’t think that the attempt to make fields like interenational relations more like science by so limiting the scope of one’s study to make it barely useful is the answer either. People like Huntington still provided insight, even thought they weren’t quantitative. It’s interesting that you mention Laplace because physicists talk about the Laplacian ilusion; since QM shows the inherent indetermancy of physics. Indeed, some measurable properties cannot exist apart from measurement (e.g. electron spin at N degrees). Finally, while it is hard to make general, robust, high probabability statements in the field of political science, it is not impossible. It’s just that most folks in the social sciences, and many in medicine, alas, think they’ve done it when they haven’t. Part of it is the way statistics are improperly treated. Being one of the first scientists who learned his craft when Monte Carlos were reasonably priced, I understand something of the pitfalls and the ways around them. So, I agree, most of the supposedly precise general statements in the social sciences aren’t…but a few are.
]]>Kris NedzynskiWarsaw, Poland Nate, I would be extremely eager to hear your opinion on Jude Wanniski’s approach to prediction in politics. I believe he was a true genius, largely still waiting to be appreciated. He applied Hayek’s “dispersed knowledge” concept (actually it can be traced back to Aristotle) to politics. Through that lens he was analyzing American and global political events, often with amazing accuracy. For instance he warned Senator Jesse Helms in 1998 that if the US government would not start to study origins of terrorism instead of merely defending against it, “terrorist mind will succeed in taking two towers completely”. Wanniski’s theory was laid down in kis “The Way The World Works”, but these may serve as a brief summary: http://tnij.org/q72t http://tnij.org/q72o http://tnij.org/q72p
DMC, Lucerne, Switzerland Are you aware of the “Good Judgement Project?” http://goodjudgmentproject.com/ In brief, these academic researchers asked teams of volunteers to make predictions about a range of international events. The teams varied in the training they received. Results of the predictions were scored in a rigorous way that benefits those who are aware of the uncertainties in their own predictions. I participated as a forecaster in the first year of the project, and found it a very interesting exercise. I am very much looking forward to seeing the papers that will document the results and conclusions.
valleyforge, Valley Forge, PA Political science is a soft science precisely because it is immune to reliable modeling and hence prediction. Humans are not billard balls or even charmed quarks and the society we’ve created is not governed by immutable laws. Hegel, Marx, and Asimov’s fictional psychohistorian Hari Seldon may have believed in deterministic history but the “great man” problem will always defy the models. For illustration just look at the decisive influence that one man’s vote, Anthony Kennedy’s, has on national policy for 312 million people. Or the many unexpected inventions and discoveries, and even disasters, that changed the course of history. Such black swan events by definition cannot be predicted but have tremendous consequences. Judging political scientists who seek merely to explain by the accuracy of their theories implied predictions is inappropriate as it expects that their field of inquiry is ultimately knowable when it is not. Conversely political scientists who have the temerity to make inadequately-qualified predictions deserve the ridicule they get.
Gyre, Pennsylvania I’m not so sure that predictions can’t be made without statistics. I wasn’t at all surprised when the Mali coup occurred, it fit the pattern of coups across the world for the past sixty years. A dangerous separatist conflict, a past history of coups in the region, a fairly poor country and the perception that the civil leaders couldn’t handle it. Admittedly I didn’t predict the coup surviving as long as it has, but predicting the coup itself isn’t so bad. In the interest of fairness to numbers, political scientist Jay Ufelder has a post on using certain numbers and criteria to predict the nations most likely to have coups. http://dartthrowingchimp.wordpress.com/2012/01/30/assessing-coup-risk-in-2012/ Also political scientist Daniel Drezner has an article in response to Stevens. He’s not impressed. http://drezner.foreignpolicy.com/posts/2012/06/25/when_a_stupid_op_ed_produces_some_smart_debate
Richard, NY Some context from the physical sciences might be useful: Weather forecasts have been probably the most successful predictions in the context of large unavoidable uncertainty. It is interesting to look histocially at their skill and the reasons for its slow improvement over decades. Initially (50s-70s) this occured due to better model formulation. Such models were very firmly grounded in well established physics. Later (80s-present) it was due to more complete observational data for prediction initialization. So the lessons would appear to be: 1. Work out the basic rules governing the dynamics of the system of interest. 2. Once step 1) has been achieved, hit the system with tons of data. My sense as a physical scientist is that social/political scientists have a lot of trouble with point 1) and resort to point 2) almost in desperation. Personally I would think a lot more about the basic behavioural dynamics that set political opinion. Economic factors are pretty clearly important but not completely explanatory…..
wheelers, denver Nate, you are part of the solution. the network of wicked problem solvers. http://pressthink.org/2012/06/covering-wicked-problems/
Nassim Nicholas Taleb The pathology of Big Data: the more variables, the DISPROPORTIONATELY higher the number of spurious results that appear “statistically significant”. For a real-life application see this busted article in The N E Journal of Medicine.
Additional clarification in the comments:
]]>Nassim Nicholas Taleb This is called the Wigner Effect in physics: a random matrix with orthogonal components will show a series of declining Principal components… In the Lebanese dialect, the more data, the more illusion of patterns.
Nassim Nicholas Taleb Geert, the problem is that nobody corrects for multiple testing in social science and epidemiology.
Nassim Nicholas Taleb Geert Van Damme, furthermore the researcher is implicitly doing multiple testing throught his career even if he submits to Bonferoni adjustments within a single paper.
Mark Weaver … Stan Young rocks! (http://www.significancemagazine.org/details/magazine/1324539/Deming-data-and-observational-studies.html).
Mark Weaver Iva, just a correction… ALL statistical analysis is not futile! However, if the analysis was based on a method developed after 1955, say, and does not have one of these names attached (Fisher, Tukey, Kempthorne, Neyman) then it most likely is futile because it’s likely based on completely insane assumptions… and, you shouldn’t really need a “statistician” to understand it!
Bob Sundahl There are two separate types of misuse of statistics to prove a hypothesis. The data dredging fallacy examines a very large set of data to discover possible “statistically significant” coincidences, ignoring the certainty that a large data set will always have some coincidences, if enough relationships are examined. The second involves the interpretation of these coincidences, real or imaginary. It is common to conclude that coincidence implies causality, ignoring the possibility (likelihood?) that hidden variables affect both of the observed parameters. When seeing these fallacious arguments used so often, one always asks the same question: Stupidity or mendacity?
Kimmo Vehkalahti Recommended reading: “The Cult of Statistical Significance”, see: http://goo.gl/yYQXM.
Bob Sundahl Here is a useful link to the article identified by Mark Weaver (with excellent cartoons) http://goo.gl/vuTIf.
Bob Sundahl The phenomenon we are describing was diagnoses as “apophenia” By William Gibson is one of his books. Humans have an inate predisposition to look for patterns, and often find them where none exist. This tendency is strongly enhanced when rewards are given for discovering trends. And, as Nassim points out, the lack of punishment for finding false trends is also encouragement. Apophenia is a critical attribute for success in many professions - sports reporters, economists, stock market analysts, astrologers (but I repeat myself).
Guru Anaerobic One manifestation of this are books like ‘The Bible Code’ [Ed: eg, many false positives of pattern identification in large data, the data in this case being the text of the Bible]
Nicholas Teague Concept summed up neatly in this simple cartoon: http://nohype.tumblr.com/post/225060683/confusion-information-graph-a-simple-index-card
Marcelo Schafranski Which one is more accurate: Bonferroni´s correction or False Discovery Rate?
Nassim Nicholas Taleb both inaccurate, of course, but same principle
Marcelo Schafranski I completely lost faith on Fisher´s/Pearson´s p, as long as it refers to the data and not to the hypothesis. It makes the Bonferroni/false discovery rate (which encourages “salame-slicing”) discussion pointless. But unfortunately, in medical science, which is my field, peolpe seem hypnotized by the p value. I sincerely wish they knew efect sizes. Totally agree with you: hard findings do not need statistics.
Pedro Peloso MORE ON CORRELATIONS.
Think you will like this:
http://www.pnas.org/content/105/45/17436
It is related to your post on spurrious correlations. These guys reply to a paper “claiming”climate change is the cause of some amphibian extinctions. They found better correlation of the extinctions with beer production and local production of bananas.
]]>“Before taking the mandated Intro class last year, when I heard ‘computer science,’ I pictured nerdy boys, who turned into nerdy bearded men, slouched over huge computers and click-clacking out codes that meant nothing to me. There’s nothing wrong with nerdy boys, comp sci just didn’t seem like something I would ever be interested in.
“This image was quickly shattered in that first intro class. Computer science started to resonate with me when I worked on my first project, creating a simple animation of a string quartet using Netlogo. It was while I was working on this that I realized comp sci isn’t about nerdy boys sitting at computers and coding out nonsense that turns into violent video games and complicated math problem solvers. No, comp sci isn’t this at all. Comp sci, as I have found in my classes at Stuy, is a medium for expression, a place for creation and creativity.”
My mom and dad are wonderful parents who have done a many great things for my sister and I along the way. One of the most important was that when we were wee little tykes they got us started early with the nascent computer revolution by buying us a series of cutting edge home computers - Tandy TRS-80, Apple IIc, Apple IIgs, and a series of PCs. I still remember learning BASIC on the TRS-80 and connecting to the Internet’s precursors - CompuServe, Prodigy, AOL, and random BBS’s via fast 56k baud modem.
I don’t believe mom and dad were sure exactly what little kids could get out these new devices, but it was no leap of faith to perceive that computers were the way of the way of the future, and getting a head start on the information age was incredibly educational and quite a gift.
The dream of the 70s and 80s Silicon Valley visionaries like Steve Jobs, Bill Gates, Jack Tramiel (Commodore 64), and others was “A computer on every desk in every home”. They achieved it and more - now it’s a computer on every desk, in every backpack, in every pocket and purse, in every corner of the world, from developed to developing nations (well, almost). And look at how they changed the world.
However, the revolution isn’t over, it’s just getting started, and the next phase is going to be “a factory on every desk in every home”. Or, more specifically “A 3D Printer on every desk in every home.” Parents and schools will be buying their kids home 3D printers to go along with their computers, tablets, and smart phones.
But the nexus of the virtual world of design (software) + the meat space of manufacturing (3D printing) on every desktop in every home will result in a synthesis worth much more than the sum of its parts. Individuals will be able to conceive of an item that solves a particular problem they have (say a wrench or a bicycle), design it themselves if they know how, download a design1,2,3, or scan an object with a handheld laser into a CAD/CAM file, and make it on the spot.
The Internet democratized access to information, and along with it any domain that consisted of pure information - journalism (blogs, data journalism, twitter, etc.), media and entertainment production (user-generated content, youtube, bittorrent, etc), financial markets (Etrade, day trading, etc.), government (very much in its nascence with the Government 2.0 movement, but happening inexorably).
If you think of all of that as virtual wealth - things of value that exist only in bits and bytes, 1s and 0s, electromagnetic signals, pure information - then the Internet’s greatest triumph so far was to democratize virtual wealth creation.
Concordantly, the most world-changing revolutionary aspect of this Third Industrial Revolution will be the democratization of real wealth creation. Whereas virtual wealth exists in virtual form, real wealth consists of material products that are made from raw materials, innovation (science, engineering, hacking, serendipitous coincidence), capital, and time. Real wealth creation happens when the value of the final product is worth more than the total cost of all the inputs.
We call that ‘profit’ or ‘net revenue’, but what it really represents is wealth created out of thin air that didn’t previously exist anywhere in the world. It’s the greatest magic trick the human race ever invented, the ultimate rabbit-from-the hat, lead-into-gold trick, that makes steady economic growth possible and lets us escape the Malthusian trap. It is the core mechanic of capitalism.
And personal computers + personal 3D printers are about to revolutionize it, again.
A few of the consequences of this decentralization of wealth creation that I can think of off the top of my head are:
Reverse outsourcing of cheap things. No longer will run-of-the-mill everyday cheap things be made overseas and imported. Rather, people will make them at home and buy the raw materials (plastic powder, etc). Things like household tools, utensils, etc. will simply be made at home from digital designs shared on the internet, while only the raw materials are made in factories somewhere and shipped cross-country and/or imported from overseas. Global flow of trade will be altered.
Some things simply won’t be made in factories anymore - any kind of inexpensive thing that is not significantly complex. Fine, exquisite, artisan, handmade things, like Stradivarious violins, will continue being handmade and valued for it. Highly complex things like cars and computers will also continue to be assembled in factories, but increasingly made of 3D Printed parts. However, their designs may be partially or fully produced by enthusiast communities collaborating over the Internet, similar to how open source software like Linux and Firefox are made today.
Education - you thought computers alone were educational, wait till your kids get a computer paired with a 3d printer! Their creativity will know no bounds. But more importantly they will be learning via a rapid feedback loop of conception -> design -> production -> design improvement -> production -> design improvement -> … ad infinitum. That’s the basic definition of kaizen - continuous, incremental, iterative improvement, and it is now in the hands of children.
The ultimate in lean manufacturing and mass customization - no more guessin… er, forecasting how much of a product to produce to meet market demand, rather individuals will simply print either a single object or exactly the quantity they require to solve whatever problem or address whatever they need they have at that moment.
The estimation and forecasting will shift to the production of raw materials for 3D Printing instead, but these can afford to be less precise and accurate, simply because unlike a spoon or wrench, the raw materials are much more flexible and have more shelf life. They never go out of style, become obsolete slower (only as new printer technology overtakes and replaces prior versions in the market), and can be used to make anything, solve any problem, address any need, supply any demand.
Retail consumers can hoard, overstock, resell, or even borrow and lend unused raw materials in ways that don’t make sense with manufactured items, knowing the raw materials will get used eventually, if not when. This lean, custom, just-in-time, distributed, mass manufacturing will change business planning and forecasting fundamentally.
T. Rowe Price has a good article covering other uses - most importantly is the development of handheld lasers that let one scan an object (or even another person) and generate a real-time CAD/CAM file that can then be 3D Printed. That makes the CAD/CAM design process faster - say you’re designing a prostethic leg for a an amputee, you just scan their remaining leg and use the computer to mirror-image it, then chisel the prosthetic parts from that template. This makes the iterative design process significantly faster and more effecient, and more immediately accessible to children and others just getting started with the technology.
In this new revolution, what’s foreseeable is exciting, but what’s unforeseeable is even moreso.
So, to all those parents (and schools) out there looking to do great things for your children to give them the best head start in life you possibly can, I strongly suggest familiarizing yourself with the rapidly emerging ecosystem and economy of 3D Printing, and get your kids involved from an early age. Prices are coming down, and home 3D printers can already be had for just a few hundred dollars. It will be for their generation what the computer on every desk in every home was for ours, but because it combines both virtual and real wealth creation, it will be even more empowering, enriching, and impactful.
]]>Bathwater = increasing fragility, instability, and systemic risk; regulatory, institutional, and govermental capture; crony capitalism; privatized profits and socialized losses; too big to fail; increasing financialization of the economy -> increasing concentration of paper wealth -> outsourcing of real wealth creation -> increasing concentration of political power -> tyranny of the minority, etc.
Baby = the core mechanic of capitalism - real wealth creation.
Paul Graham wrote an excellent essay on how to create wealth as an entrepreneur. That’s the micro take on it, and I’d like to add the macro take:
There are only two ways of creating real wealth:
Harvest raw materials + apply labor + capital + innovation + time = finished product worth more than the total cost of inputs. [value of final product - total cost of inputs] = [real wealth]. We call it ‘profit’ or ‘net revenue’ but what it really is, is wealth that didn’t previously exist anywhere in the world and was created out of thin air.
For example, we used to take hundreds of dollars worth of iron and wood, and turn it into thousands of dollars worth of trains and ships. Now we take millions of dollars worth of sand, copper, and aluminum, and turn it into billions of dollars worth of microchips. Miracle. Magic. Alchemy. Science. And, the core mechanic of capitalism.
This is the greatest magic trick the human race has ever invented, the ultimate rabit-from-the-hat, lead-into-gold alchemy, and is what makes all else in our modern world possible.
Provide a service that reduces the cost of inputs or increases the value of the final product of #1. For example, there is great economic value in a government and legal system that incentivizes wealth creation #1 and disincentivizes wealth destruction (via fraud, crime, etc.). Improving health and education also reduce the costs of inputs and increase the value of outputs. A system of efficiently allocating capital to real wealth creation ventures (when it’s working correctly) provides great value to #1 as well.
These are two the two fundamental ideas of economics, the core mechanic of capitalism, that everyone living in a modern economy must understand. Two more good essays drawing the distinction between the current problems of capitalism and its core mechanics are by Judge Richard Posner and Gary Becker. There is much to criticize of our current incarnation of capitalism, but any criticism that fails to understand and account for real wealth creation is probably flawed.
]]>The US population is ~300 million, China’s is ~1.2 billion, 4x our size. (India’s is over 1 billion and the EU is ~400 million, for comparison). For every engineer, scientist, genius, etc in the US, there are ~4 in China. Over the long term the US has no hope whatsoever of competing against that without significant structural changes.
Those structural changes include fixing our educational system, fighting tooth and nail for skilled immigrants, offering them better incentives to stay rather than return home to make their fortunes in fast growing developing economies, fixing the broken financial and patent/IP systems, etc.
Judge Posner sums up our strengths and failings well in a recent blog post. It is possible to compete against numbers, but against a numerical advantage as overwhelming as China’s, the US system will need to be operating at an order of magnitude greater efficiency than it currently is.
Immigration is one of the few crucial advantages the US can exploit in that competition, but right now we’re shooting ourselves in the foot so hard there we’re blowing our leg off.
]]>However, I do have a major complaint about the fundamental functionality. The problem with G+ is that you can’t fine-tune the signal:noise ratio enough.
In a nutshell, Facebook is a network of people I know first IRL, so getting pictures of their dinner last night or latest cat’s antics and other useless stuff I can sort of live with, it goes with the territory.
But G+ is more like Twitter with longer posts - I follow a a lot of people I don’t know IRL, but only because of a shared interest, and I’m only interested in their posts on that interest, not the other noise.
Whereas pointless posts on Twitter are only 140 characters, don’t take up much screen real estate, and are easy to skim and/or skip, that’s less the case with G+. I really want a way in G+ to filter out posts by those people that don’t have anything to do with the shared interest.
For example, if I create a “Functional Programming” circle and subscribe to a bunch of Haskell, Ocaml, ML, Lisp, and Scheme programmers that I don’t know IRL, I’m really not interested in their vacation photos and whatnot. But currently there’s no way to filter their vacation photo posts from their posts on functional programming.
An effective 90% solution would be to simply add hash tag filtering to circles, so I can instruct my Functional Programming circle to only accept posts with #functional, #functionalprogramming, #haskell, #ocaml, #ml, #lisp, #scheme, and block anything else without at least one of those hash tags in it.
Not quite perfect, and G+’ers would have to develop the habbit of using hashtags more than they currently do, but it’s functional and flexible enough and provides the tools necessary for the community to solve this problem themselves.
This is my biggest G+ pain point, and while I have nothing negative to say about this redesign, as long as it doesn’t solve this one problem, it will do nothing to get me using G+ more.
]]>Well there’s an easy way to do this with a software version control control system like Git, Mercurial, Subversion, etc. I use Linux and Git, but this technique should work on any operating system with any version control system.
There are three pertinent filesystem locations - the Chrome Bookmarks file, the Chromium Bookmarks file, and the Git repo you will create to sync the two:
1 2 3 | |
Howto:
1 2 | |
1 2 | |
1
| |
.gitignore file that ignores everything except ‘Bookmarks’ and ‘.gitignore’.1
| |
1 2 3 | |
Primary browser (Chromium for this example):
1 2 3 4 | |
Secondary browser (Chrome for this example)
1 2 3 4 | |
I’m working on a post-commit hook that will automatically do this, but not done yet.
Done. Now whenever you add bookmarks to one browser, keep the parent repo updated with:
1 2 3 4 5 | |
And finally just remember to also do a git pull origin master from the other browser before adding any new bookmarks to it as well.
update-alternatives tool.
Download the package, extract or build to /usr/lib/jvm/[version],
/opt/java/[version], /opt/scala/[version], /opt/haskell/[version],
or anywhere else in the system file system (except ~/).
Change owner:group to root:root
1
| |
update-alternatives script (Java, Scala, Haskell). Feel
free to fork and modify for other languages.See my installation guide for Haskell for more details. The same process works
for any package. The tricky part is writing the update-alternatives script, but it
should be clear how it works upon closer inspection of those scripts.
The update-alternatives commands take the form:
1 2 3 4 | |
1
| |
1
| |
Read on for why…
It can be used with any package, not just Java. I currently use it to manage multiple versions of Java, Scala, Haskell GHC, Haskell Platform, Ant, and Maven.
Debian repos are known for providing stable software, but sometimes at the cost of
being up to date. Great for production machines, not so great when you want to
experiment with the latest and greatest on your dev box. update-alternatives
solves that, by letting you easily bypass the repos to manually add current versions
of software to the system (linking them into /usr/bin/, /usr/lib, /usr/share, etc.)
without conflicting with the version from the repos. With update-alternatives you
can install both, and toggle among the active one depending on what you’re working on.
Upgrade to a new version without deleting the old. Both can co-exist on the system
at the same time, unlike installing the software via repo. If the new version breaks
something that depends on it, easily rollback to the prior version with just a
sudo update-alternatives --config.
Keep your system cleaner by putting software in a single location like
/opt/java/jdk/1.6.0_34 and soft linking it to system directories like /usr/bin,
/usr/lib, /usr/share, etc. A typical *nix install is messy - files are deposited
throughout the system in /usr/bin, /usr/lib, /usr/share, etc. update-alternatives
solves that problem.
Easily uninstall any package. For example, switch the current active package to
another version with update-alternatives --config java and delete the unwanted one
with rm -rf /opt/java/jdk/1.7.0_04.
Run the software via System PATH instead of user PATH. update-alternatives
automatically installs software to the system path it is designed for, usually
/usr/bin.
This system reminds me a bit of Gobo Linux, which seeks to [redesign the Linux
filesystem][7] by putting full package distributions in a single location, like
/Programs/Bash/3.0. A great idea that would make Linux much more user-friendly.
update-alternatives is one small step in that direction.
Start DrRacket
Change the language to Module. In the Language menu at the top (or the bottom left):
Language -> Choose Language -> Module -> OK
DrRacket lacks the primitive atom? used extensively in The Little Schemer, so you have to define it yourself. atom? simply returns true if the input is an atom, false if not. In the top window, the Definitions window, add:
1
| |
That’s all, you’re ready to start.
]]>