![]() |
|
Tuesday, April 11, 2006
previous entry | main | next entry | TrackBack (0)
Horror stories about anonmous peer review
Henry Farrell links to a Chronicle of Higher Education story by Jeffrey Young about how Microsoft Word's tags have eroded anonymity in peer review. Henry adds: Word documents preserve a lot of metadata, including, very often, the author’s name – so that if you submit your review via a Word email attachment (as many journals ask you to these days), and the journal forwards the review unchanged to the article’s author, he or she can figure out who you are without having to play the usual guessing game. I’ve been aware of this for a couple of years (I carefully strip all data before sending reviews out, just in case) – but I suspect that many academics aren’t (some of them may not even realize that Word collates this data automatically).I've been outed once as a reviewer after I rejected a piece, but it was not due to anything as high-tech as MS Word metadata. I faxed the journal -- which shall remain nameless -- my review. The journal then faxed it to the paper-writer -- who shall also remain nameless. The problem was that the journal's fax to the writer contained my department's fax number and identification -- and from there it was pretty damn easy to identify the referee. Here's a link for potential referees about how to stay anonymous if you electronically submit your referee reports. posted by Dan on 04.11.06 at 11:33 PMComments: Any horror stories out their about reviewers who don't use Spellcheck? posted by: Zathras on 04.11.06 at 11:33 PM [permalink]The biggest problem with reviewers --by far --is them not actually doing the review, or getting around to it after being chased four times. A fairly well known scholar in my field always identifies his reviews and insists the identification remain with the review. He believes it leads to a more honest, thoughtful and conscientious review. Frankly I think he is right, and I can't really see a downside to identifying yourself. Sure, academia is full of fragile egos, but it seems to me that anonymity is a often a shield for poor work. On the other hand, submitted papers should be kept anonymous to avoid favoritism (or its oppisite). posted by: Mitchell Young on 04.11.06 at 11:33 PM [permalink]If a reviewer wants, googling an article's title will often reveal the article posted on the author's website, where it was given at a conference, etc. Just another way things aren't as anon as they may appear. posted by: Dan on 04.11.06 at 11:33 PM [permalink]"Collated"?! posted by: P. Froward on 04.11.06 at 11:33 PM [permalink]Even less technical methods of anonymization are not foolproof. Some years ago, I was sent a referee's report from a Dutch journal which was a xerox of what the referee had sent them, with his signature whited out. All I had to do was turn it over and look at the back while holding it up to a bright light: the name was clearly visible. I was also surprised to see the report refer to me by name. Strict double-blind refereeing seems to be a mostly Anglo-Saxon thing. At least in my field (Classics), continental journals often use single-blind refereeing or even let the editor pick and choose whichever submissions he likes. Even before the web, double-blind refereeing was never foolproof. I once asked an eminent classicist who had devoted his entire career to one of the major Greek authors (Homer, Plato, Thucydides, Euripides, one of those guys) how often he knew the name of the supposedly anonymous authors he was reviewing. He said about 50% of the time. After twenty years spent cultivating the same corner of the classical garden, he knew the stylistic tics and ideological approaches of many of his colleagues well enough that he couldn't help knowing who he was reviewing a lot of the time, even when he hadn't seen previous versions of their work at conventions and such. Conclusion: double-blind refereeing is good, but far from foolproof, and the system still depends on the basic fairness of the referees. posted by: Dr. Weevil on 04.11.06 at 11:33 PM [permalink]Post a Comment: |
Politics, economics, globalization, academia, pop culture... all from a
Main home page Reviews of DanielDrezner.com: "Sharp but informal commentary on politics and foreign policy." -- The New Republic "Dan Drezner is terrific.... Excellent blog." -- Andrew Sullivan "Dan's stuff is always worth reading." -- Eugene Volokh "One of the essential weblogs." -- Gawker.com "Old battle horse of the blogosphere." -- Jewcy.com "Soft porn." -- Amitai Etzioni "Spawned grave atrocities and vast destruction." -- Glenn Greenwald "Monday morning quarterback... conservative robot... the very foundation of troubles in this country." -- not-so-random readers Contact me at: ddrezner@gmail.com (But click here to read my e-mail policy) Search the Site TNR's Open University Jacob Levy Glenn Reynolds Andrew Sullivan Mickey Kaus Virginia Postrel The Volokh Conspiracy Josh Marshall Crooked Timber OxBlog Real Clear Politics Kevin Drum Across the Aisle Economist's Free Exchange TNR's The Plank NRO's The Corner TAP's Tapped America Abroad Duck of Minerva Opinio Juris Brad DeLong Jeff Jarvis Mystery Pollster Mark Kleiman Meryl Yourish Megan McArdle Marginal Revolution Michael Munger Chris Lawrence Matthew Yglesias Hit and Run Cold Spring Shops Stephen Green Outside the Beltway Pejman Yousefzadeh Laura McKenna (11D) Elected Swineherd Phil Carter Joe Gandelman Winds of Change Andrew Samwick Greg Mankiw Dani Rodrik Roger L. Simon Tom Maguire Greg Djerejian The American Scene Post Global Democracy Arsenal Recent articles online "Foreign Policy Goes Glam."The National Interest, November/December 2007 "Rise of the Hipster Statesmen." Newsweek International, November 1, 2007 "The New New World Order." Foreign Affairs, March/April 2007 "Mind the Gap." The National Interest, January/February 2007 "The Grandest Strategy Of Them All." Washington Post, December 17, 2006 U.S. Trade Strategy: Free Versus Fair Council on Foreign Relations Press, September 2006. Complete online article archive Blog Archives June 2008May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 November 2003 October 2003 September 2003 August 2003 July 2003 June 2003 May 2003 April 2003 March 2003 February 2003 January 2003 December 2002 November 2002 October 2002 September 2002 Academia Area studies Book club culture economics fence-sitting from Blogger globalization homeland security international relations law Mediasphere My very important posts New Republic outsourcing personal politics Sports The blog paper the blogosphere thesis ideas Trade and Development U.S. foreign policy website maintenance See full archives listing Recent Entries • Someone keep Fleet Street away from Bill Clinton• It rivals Buckley vs. Vidal, I tell you • So.... are the Clintons morons? • The New York Times didn't ask me, but then again, that's why I have this blog • Monica Crowley's jet black pot • Al Qaeda is losing • Speaking of karma.... • The blog post that writes itself • What made me laugh today • Where should Hillary go? Site Credits |