Wednesday, November 30, 2005

previous entry | main | next entry | TrackBack (0)


Foxes, hedgehogs, and the study of international relations

When we last left off, we were discussing Louis Menand's New Yorker review of Philip Tetlock's Expert Political Judgment: How Good Is It? How Can We Know?.

In his review, Menand highlights an interesting observation by Tetlock on who did better at predicting world political events.

It was no news to Tetlock... that experts got beaten by formulas. But he does believe that he discovered something about why some people make better forecasters than other people. It has to do not with what the experts believe but with the way they think. Tetlock uses Isaiah Berlin’s metaphor from Archilochus, from his essay on Tolstoy, “The Hedgehog and the Fox,” to illustrate the difference. He says:
Low scorers look like hedgehogs: thinkers who “know one big thing,” aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who “do not get it,” and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible “ad hocery” that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess.
A hedgehog is a person who sees international affairs to be ultimately determined by a single bottom-line force: balance-of-power considerations, or the clash of civilizations, or globalization and the spread of free markets. A hedgehog is the kind of person who holds a great-man theory of history, according to which the Cold War does not end if there is no Ronald Reagan. Or he or she might adhere to the “actor-dispensability thesis,” according to which Soviet Communism was doomed no matter what. Whatever it is, the big idea, and that idea alone, dictates the probable outcome of events. For the hedgehog, therefore, predictions that fail are only “off on timing,” or are “almost right,” derailed by an unforeseeable accident. There are always little swerves in the short run, but the long run irons them out.

Foxes, on the other hand, don’t see a single determining explanation in history. They tend, Tetlock says, “to see the world as a shifting mixture of self-fulfilling and self-negating prophecies: self-fulfilling ones in which success breeds success, and failure, failure but only up to a point, and then self-negating prophecies kick in as people recognize that things have gone too far.”

Tetlock did not find, in his sample, any significant correlation between how experts think and what their politics are. His hedgehogs were liberal as well as conservative, and the same with his foxes. (Hedgehogs were, of course, more likely to be extreme politically, whether rightist or leftist.) He also did not find that his foxes scored higher because they were more cautious—that their appreciation of complexity made them less likely to offer firm predictions. Unlike hedgehogs, who actually performed worse in areas in which they specialized, foxes enjoyed a modest benefit from expertise. Hedgehogs routinely over-predicted: twenty per cent of the outcomes that hedgehogs claimed were impossible or nearly impossible came to pass, versus ten per cent for the foxes. More than thirty per cent of the outcomes that hedgehogs thought were sure or near-sure did not, against twenty per cent for foxes.

The upside of being a hedgehog, though, is that when you’re right you can be really and spectacularly right. Great scientists, for example, are often hedgehogs. They value parsimony, the simpler solution over the more complex. In world affairs, parsimony may be a liability—but, even there, there can be traps in the kind of highly integrative thinking that is characteristic of foxes. Elsewhere, Tetlock has published an analysis of the political reasoning of Winston Churchill. Churchill was not a man who let contradictory information interfere with his idées fixes. This led him to make the wrong prediction about Indian independence, which he opposed. But it led him to be right about Hitler. He was never distracted by the contingencies that might combine to make the elimination of Hitler unnecessary. (emphases added)

I'll need to read the book to see the methodology by which Tetlock distinguished hedgehogs from foxes, but let's assume that his finding is correct. What does this imply for the study of international relations?

Potentially a lot -- from my vantage point, the incentives in the IR discipline are heavily skewed towards the hedgehogs. Methodologically, the growing sophistication of formal, statistical, and even qualitative techniques make it increasingly difficult for any one scholar to keep up their abilities in more than one area. Professionally, our field rewards the hedgehogs, the ones who come up with "the big idea" that can explain it all. As a result, my field has a lot of hedgehogs, which means that we may not be of much use when it comes to policy relevance.

Is this a bad thing? I'm sure that many commenters will instinctively say, "yeah!" but it's not so clear cut. First, if the point of the academy is to nourish unpopular but important ideas, then it's a good thing we have a lot of hedgehogs, because every once in a while they will produce the kind of insight that helps to understand Really Big Truths.

Second, asking IR scholars for accurate predictions about the future might be like asking meterologists for an accurate weather forecast three months ahead. That's impossible -- there are just too many variables. It might be that what political scientists do best is not predicting future events but rather explaining the past and present in a way that provides limited but useful insights into the very near future.

Third, there are think tanks for the kind of expert predictions discussed in Tetlock's book. It's true that think tanks have their own perversities, but perhaps the best thing to do is fix them rather than the academy.

Despite those counterarguments, I think a few more IR foxes might be a good idea. [Good idea! Did you know Salma Hayek will be co-hosting the Nobel Peace Prize Concert on December 10th?--ed. That's not who I meant by foxes. Oh.... did you mean Angelina Jolie's work as a United Nations ambassador?--ed. No, and you're not helping right now.]

I'll leave this question to the commenters.

posted by Dan on 11.30.05 at 05:06 PM




Comments:

Well, I think if the hedgehogs recognized their own limitations, and do not claim great expertise in areas where their advice is unlikely to be helpful, there's no harm in keeping lots of them.

The advance of science, after all, relies on people willing to bet years of one's life developing a theory that may be (and probably is) quite wrong. Among a good scientist's virtues, good judgement probably doesn't rank nearly as high as self-confidence and energy.

posted by: Mycroft on 11.30.05 at 05:06 PM [permalink]



Was Churchill really a hedgehog ? In his earlier life, he switched parties quite a lot. He also seems to have changed opinions on Irish home rule at different times.

But if Churchill was a hedgehog, then one has to wonder too about hedgehogs. After all, Churchill was perfectly willng to let millions of Indians starve to death during the Bengal famine despite pleas from the then British Vicerory for assistance.

posted by: erg on 11.30.05 at 05:06 PM [permalink]



In my experience, particularly with academic types, most people believe they are foxes but appear to the rest of the world as hedgehogs. For instance, you will find plenty of historians who scoff at the big man theory but wholly embrace the big idea. Not because it was an idea that intersected with particular timing, but because it was the 'right' idea. The big man proponent would scoff at the idea and embrace the man. Both hedgehogs, both probably viewing themselves as foxes.

I dont think humans are particularly well designed to embrace a truly chaotic, trend driven world, if thats indeed what we live in. Personally I dont buy that. If a particular one of Philip of Macedons sperm takes a left instead of a right in Olympias womb, we would live in a world practically unrecognizable to us. History is built on individuals meeting opportunity.

posted by: Mark Buehner on 11.30.05 at 05:06 PM [permalink]



There should be a complementary relationship between academic hedgehogs and practitioner foxes. A hedgehog may be able to elucidate some particular force or mechanism of great importance, but not figure out the synthesis of that force with others. A smart hedgehog will then go on to see that she her work is important but limited because of all the other forces and mechanisms on the loose in the world.

Foxes then have to try to synthesize the interaction of all those forces in the context of a particular case. If they miss (or misunderstand) an important force, they'll screw up, so they need a robust but diverse community of monomaniacal hedgehogs to investigate all those factors.

Engineering science describes forces and mechanisms, but can never provide an algorithm for designing an optimal airplane or building. Designers need to be foxes exploiting the discoveries of their hedgehog science base. I'm currently trying to express this distinction in a palatable way for the management field.

posted by: steve on 11.30.05 at 05:06 PM [permalink]



Check Collins' "From Good to Great" where he uses the same Foxes vs Hedgehogs segmentation in a business strategy setting, and sides with the Hedhodg firms as overperformers.

posted by: PlanMaestro on 11.30.05 at 05:06 PM [permalink]



Check Collins' "From Good to Great" where he uses the same Foxes vs Hedgehogs analogy in a business strategy setting. He sides with thr Hedhodges as a key ingredient for a overperforming firm.

Even more interesting is his discussión on the leadership style suited to reach "the big idea".

posted by: PlanMaestro on 11.30.05 at 05:06 PM [permalink]



Maybe I'm missing something, but it seems unlikely that the explanation has anything to do with particular views about how politics works. We've known for 50 years that human experts are outpredicted by dumb statistical models, and this finding is true for every predictive problem that's been studied, regardless of intellectual context. In general it appears that experts trust their own judgment more than the stats, and they shouldn't. It's simplest just to assume that this general pattern is also true in politics, rather than searching for a kind of expertise that's reliable. Such searches have a hopeless track record.

posted by: dop on 11.30.05 at 05:06 PM [permalink]



I would tend to agree that hedgehogs are good in academia because the search for parsimony is the one the uncovers the underlying principles of social phenomena. Ideally, when we reach a point when we have a pretty good idea of the ultimate causes and their explanations, we would have a much better explanation of the surrounding social realities.

Until we do so however, we are faced with a perverse outcome and an unfortunate side-effect. The perverse outcome is that if we want to understand most contemporary foreign affairs issues under the prism of one of the major parsimonious theoretical approaches, we would be behind the curve; we would understand less than if we would if we adopted a more ad hoc pragmatic approach.

That in turn makes for the unfortunate side-effect. Because of the incentives in academia, the most successful academics are the ones who made major breakthroughs in parsimony, but can't really provide similar revolutionary thinking in current affairs.

Moreover, I think that academia is hurt theoretically as well. Political science needs the people who will do the dirty work and work on real life issues in all of their complexity even when sacrificing parsimony. After all, it this kind of work that provides the elaborate descriptions of reality upon which more parsimonious-oriented social scientists base their research into ultimate causes. It's also a different layer of public policy work outside of the more political world of think tanks.

On another note, outside of academia, hedgehogs are the worst kind of public intellectual; the used car salesmen of ideas if you will. They usually get one catchy idea, without any of the constrains of academic peer review and they market it incessantly as the one big thing to explain everything under the sun with all the messianic fervor in the world. Ultimately their ideas count for virtually nothing-except to become buzz words for people who are supposedly in the know or are trying to show off in the boardroom or to social gatherings.

posted by: Nick Kaufman on 11.30.05 at 05:06 PM [permalink]



Many years ago I took a polysci grad class on modern Africa. Nearly everything we read was from deep specialists in particular countries or regions, but never anything that compared countries across regions or the world. The prof said comparative polysci didn't get much respect. That's a shame, since comparing African countries with other countries might help shake out which variables really matter to development. The fox can use the hedgehog's work to build broader theories. That's what I do in my field.

posted by: Dude on 11.30.05 at 05:06 PM [permalink]



There seem to be several interrelated issues that make the hedgehog/fox distinction particularly problematic for IR.

The first is measurability, or perhaps signal to noise ratio. Most meaty IR questions have hundreds of possible explanatory variables, but very few data points. Moreover, just about every interesting variable is potentially endogenous because political actors constantly reflect on their situation and change the experiment. In finance, by comparison, event studies can tease out some pretty good conclusions by clever statistical design, because there are often hundreds or thousands of events, and there are prices to serve as a measurement tool.

The second advantage of finance (which has a huge number of explanatory variables) is the "no-arbitrage" effect - i.e. if you are an expressing an opinion by betting and you are wrong, you lose (think spot forex) so you are careful what you say.

A third advantage is that ability to form portfolios - i.e. maybe the "foxes" in finance are diversified portfolios and the "hedgehogs" are individual assets. Unfortunately in IR you can't try 1/1000 of 1000 different Iraq strategies ...

posted by: Robert Bell on 11.30.05 at 05:06 PM [permalink]



As a subfield of political science, IR seems more pathologically hedgehoggish than most others (though less so than normative theory). This is why most IR scholars write big, ponderous books that go largely unread.

Forget prediction for a second, and forget the "real world." It seems to me that, as a model for scientific inquiry (what we're supposed to be doing, yes?), the foxes have it all over the hedgehogs. If the incentives were such that every chemist, engineer, mathematician, and so forth of the past couple centuries was a hedgehog, we'd all be sleeping in thatched huts and communicating by smoke signals.

posted by: The Pooka on 11.30.05 at 05:06 PM [permalink]



See below for a link to a nice piece by Freeman Dyson on the nature of hedgehogs and foxes in science (apparantly the terminology stems from Isaiah Berlin):

"Hedgehogs are interested only in a few problems which they consider fundamental, and stick with the same problems for years or decades. Most of the great discoveries are made by hedgehogs, most of the little discoveries by foxes. Science needs both hedgehogs and foxes for its healthy growth, hedgehogs to dig deep into the nature of things, foxes to explore the complicated details of our marvelous universe. Albert Einstein was a hedgehog; Richard Feynman was a fox."

http://www.nybooks.com/articles/18350

posted by: Erik on 11.30.05 at 05:06 PM [permalink]



Humans are not only observers but also agents who are aware of forecasts, predictions, expectations etc thus this reflexivity will eventually screw up any prediction that is made public.

Plus our concerns in IR, theoretically speaking, such as search for parsimony etc seem misguided...there is no fundamental formula, the equivalent of theory of everything in physics, that describes the dynamics of IR...I used to think that Waltz got it :)) We might find a series of formulas that might interact with certain probabilities to create certain outcomes..such as basic properties of complex adaptive systems stuff...but it is highly unlikely that we will ever accurately determine the probabilities we could attach to those variables...

I am thinking more and more thou..if neuroscience could help us to determine the basic tenants of human decision-making process, cognitive and affective dimensions, we might...and just might...be able to sketch out the fundamentals of foreign policy decision making, first modelling individual processes and later using those to model small-group interactions, which characterize FP decision-making most of the time...the stuff we have so far is very limited and primitive I am afraid...the best is Jervis (1997) in my opinion in terms of making an attempt to introduce complex systems to IR..or perhaps Rosenau..but other so-called decision-making stuff is too simple..including expected utility approaches such as Bueno de Mesquita or Kugler...

Thou such a development requires either serious collobration between psychologists, neuroscientists, and IR theorists...plus help from agent-based modeling, perhaps more help from math types...or provide our grad students high levels of math skills....plus some cognitive and affective psychology training...well...not easy I guess.

posted by: A.N. Onymous on 11.30.05 at 05:06 PM [permalink]



If you're interesed in current work on cognitive/affective connections with IR implications Rose McDermott's work is some of the best out there. She has a book on risk taking, plus a basic one on political psychology and IR. And she had a nice review/projection piece in the December 2004 issue of Perspectives on Politics - "The Feeling of Rationality: The Meaning of Neuroscientific Advances for Political Science."

As to the original piece - Phil Tetlock and his colleagues have been doing this sort of thing for years. There's loads of data that fits with these arguments.

posted by: Armand on 11.30.05 at 05:06 PM [permalink]



I am the Hedgehog!

posted by: Ron Jeremy on 11.30.05 at 05:06 PM [permalink]



No you are not. Thomas Friedman is the Hedgehog.
And if you don't believe me, just listen to a few of these amazing anecdotes...

posted by: MTC on 11.30.05 at 05:06 PM [permalink]






Post a Comment:

Name:


Email Address:


URL:




Comments:


Remember your info?