![]() |
|
Tuesday, November 29, 2005
previous entry | main | next entry | TrackBack (0)
Who needs experts?
Louis Menand has a glowing review in the New Yorker of Philip Tetlock's latest opus, Expert Political Judgment: How Good Is It? How Can We Know?. Some highlights: It is the somewhat gratifying lesson of Philip Tetlock’s new book... that people who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. When they’re wrong, they’re rarely held accountable, and they rarely admit it, either. They insist that they were just off on timing, or blindsided by an improbable event, or almost right, or wrong for the right reasons. They have the same repertoire of self-justifications that everyone has, and are no more inclined than anyone else to revise their beliefs about the way the world works, or ought to work, just because they made a mistake. No one is paying you for your gratuitous opinions about other people, but the experts are being paid, and Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote. Our system of expertise is completely inside out: it rewards bad judgments over good ones....There are intriguing implications for understanding world politics that deserves a post of their own, but suffice it to say that Tetlock's findings will probably warm the cockles of every political blogger out there. posted by Dan on 11.29.05 at 10:41 PM Comments: Who expects political analysists to be 'right' as often as financial analysts are, whose stock predictions are notoriuosly bad? After all, much more relevant information is available about publicly traded companies, and the analysts predictions are about very discrete events like stock price movements. For an exploration of this theme from a interesting prespective - that maybe the very few people who DO get these prediction right over long periods of time are merely lucky and not wise, see Malcom Gladwell's New Yorker article "Blowing Up" here - http://www.gladwell.com/2002/2002_04_29_a_blowingup.htm posted by: Jos Bleau on 11.29.05 at 10:41 PM [permalink]So if Tetlock is right, then he is as full-of-it as any expert, because by writing this book he is saying he is an expert expert, which means he's wrong- so if he's wrong then he's dead-on like all the experts, but then wait a second, I'm confused. Oh wait I get it. It's common knowledge that experts are full of it, so Tetlock is just agreeing with the commonfolk, which means he can be correct. Which means his book won't tell you anything you didn't already suspect. I hope his book has some serious figures to back up his claims or is filled with wit and delightful, well-crafted anecdotes. Otherwise it sounds hypocritical. And I'm no expert, so you can take that opinion to the bank, smartie pants. posted by: Rob C on 11.29.05 at 10:41 PM [permalink]I remember thinking along these lines pre-Iraq: Tetlock also has an unscientific point to make, which is that “we as a society would be better off if participants in policy debates stated their beliefs in testable forms” I also thought I was way naive to think that way. A war on Al Qaeda or Osama was testable ex ante. A war on "terror" is intrinsically not testable. On the other hand, suppose you become attached to just one metric above all others, clinging to it even as circumstances change, going on talk shows and it is the only metric that matters ... posted by: Robert Bell on 11.29.05 at 10:41 PM [permalink]> "will probably warm the cockles of every political blogger out there." Of course it will. Confirmation bias is one of the reasons people get things wrong so often. If a study agrees with my hunch, I swallow it uncritically. If it disagrees, I'll nitpick it to pieces. How do I get a talking head job? I read the NYTimes and I've got lots of free time. Just what I need, warm cockles! posted by: David Pinto on 11.29.05 at 10:41 PM [permalink]I actually believe in experts. I apply this term to anyone who knows a great deal more than I do about a given subject. Now, if you ask any expert about a subject different than (sometimes even slightly different than) his or her area of expertise, the value of the response you get may be no better than you could have gotten by directing your inquiry to someone picked at random from the phone book. It is also true that an expert who may have a great deal to say at some times may not at others. No talk show guest ever answered a question with the words "I have nothing useful to say about this subject right now" -- because guests are flattered to be asked, and want to be asked again. They will say almost anything rather than have to say nothing. Finally, an expert may have excellent judgment as well as extensive knowledge about a given subject. This is not a requirement, however, and it sometimes takes multiple exposures to any particular expert to ascertain whether he or she is really a reliable source for analysis, or merely for data. Where television is concerned, once an expert is established as sufficiently entertaining his judgment and even the extent of his knowledge may not exclude him from being trotted out to give his learned opinion on any old thing. posted by: Zathras on 11.29.05 at 10:41 PM [permalink]what a blowhard. that is the dumbest thing ive ever heard and is just another strike at the continually growing disregard people have - wrongfull so - for political science. econ is suffering this to a degree, but not as much as poli sci. sorry, this guy is just plain stupid. im not going to address this further. posted by: Derek on 11.29.05 at 10:41 PM [permalink]The point of political experts on TV is not to be right. It is to articulate different versions of the conventional wisdom. Yes, if you are a smart reader of the new york times, you can develop your own, better point of view. But if you are not a smart reader of the NYT, only an average cable viewer, you can choose one of the two sides on a cable talk show, and adopt it as your own, without having to develop your own more subtle take. In other words, their function is to show us what opinions to adopt as fashionable or appropriate to our political perspectives. It is not to make us think deeply. posted by: DK on 11.29.05 at 10:41 PM [permalink]Not sure if the same is true with political pundits but I have been called on occasion to make comment about market matters. I can say this about the process by which an expert gets quoted. You will only be quoted if what you say fits the story the journalist wants to tell or thinks they should tell. Usually they will question more sources than they need and cherry pick the bits and pieces they want/need out of a pool of quotes. Hence as an expert you quickly learn what makes the cut. What makes the cut generally might have a bit of novelty but generally not too far from consensus. If you say something to far off you will be cut. posted by: asiequana on 11.29.05 at 10:41 PM [permalink]posted by: enlarger patch penis on 11.29.05 at 10:41 PM [permalink] posted by: prosolution pill on 11.29.05 at 10:41 PM [permalink] When there is no alternative but to look back at the entries made here, I wonder why I keep trying. posted by: penis pills on 11.29.05 at 10:41 PM [permalink]dsnpklc ztfq tedhxmsi edsqtjxk lmwqb zecn ncbvw posted by: kctdjveqz kgcz on 11.29.05 at 10:41 PM [permalink]Post a Comment: |
Politics, economics, globalization, academia, pop culture... all from a
Main home page Reviews of DanielDrezner.com: "Sharp but informal commentary on politics and foreign policy." -- The New Republic "Dan Drezner is terrific.... Excellent blog." -- Andrew Sullivan "Dan's stuff is always worth reading." -- Eugene Volokh "One of the essential weblogs." -- Gawker.com "Old battle horse of the blogosphere." -- Jewcy.com "Soft porn." -- Amitai Etzioni "Spawned grave atrocities and vast destruction." -- Glenn Greenwald "Monday morning quarterback... conservative robot... the very foundation of troubles in this country." -- not-so-random readers Contact me at: ddrezner@gmail.com (But click here to read my e-mail policy) Search the Site TNR's Open University Jacob Levy Glenn Reynolds Andrew Sullivan Mickey Kaus Virginia Postrel The Volokh Conspiracy Josh Marshall Crooked Timber OxBlog Real Clear Politics Kevin Drum Across the Aisle Economist's Free Exchange TNR's The Plank NRO's The Corner TAP's Tapped America Abroad Duck of Minerva Opinio Juris Brad DeLong Jeff Jarvis Mystery Pollster Mark Kleiman Meryl Yourish Megan McArdle Marginal Revolution Michael Munger Chris Lawrence Matthew Yglesias Hit and Run Cold Spring Shops Stephen Green Outside the Beltway Pejman Yousefzadeh Laura McKenna (11D) Elected Swineherd Phil Carter Joe Gandelman Winds of Change Andrew Samwick Greg Mankiw Dani Rodrik Roger L. Simon Tom Maguire Greg Djerejian The American Scene Post Global Democracy Arsenal Recent articles online "Foreign Policy Goes Glam."The National Interest, November/December 2007 "Rise of the Hipster Statesmen." Newsweek International, November 1, 2007 "The New New World Order." Foreign Affairs, March/April 2007 "Mind the Gap." The National Interest, January/February 2007 "The Grandest Strategy Of Them All." Washington Post, December 17, 2006 U.S. Trade Strategy: Free Versus Fair Council on Foreign Relations Press, September 2006. Complete online article archive Blog Archives May 2008April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 November 2003 October 2003 September 2003 August 2003 July 2003 June 2003 May 2003 April 2003 March 2003 February 2003 January 2003 December 2002 November 2002 October 2002 September 2002 Academia Area studies Book club culture economics fence-sitting from Blogger globalization homeland security international relations law Mediasphere My very important posts New Republic outsourcing personal politics Sports The blog paper the blogosphere thesis ideas Trade and Development U.S. foreign policy website maintenance See full archives listing Recent Entries • Hillary Clinton's inexcusable bigotry• So Tuesday was a pretty good day.... • The best commencement address you'll never hear • Some final thoughts on Hillary Clinton • America's awesome influence over the G8 • Hillary Clinton's contribution to the all-purpose excuse genre • What I said at the London conference • Why I'll be (relatively) mute this week • Bitter academics, tenure, torture, and pie • A real policy difference. Yippee!! Site Credits |