Tuesday, April 11, 2006

previous entry | main | next entry | TrackBack (0)


Horror stories about anonmous peer review

Henry Farrell links to a Chronicle of Higher Education story by Jeffrey Young about how Microsoft Word's tags have eroded anonymity in peer review. Henry adds:

Word documents preserve a lot of metadata, including, very often, the author’s name – so that if you submit your review via a Word email attachment (as many journals ask you to these days), and the journal forwards the review unchanged to the article’s author, he or she can figure out who you are without having to play the usual guessing game. I’ve been aware of this for a couple of years (I carefully strip all data before sending reviews out, just in case) – but I suspect that many academics aren’t (some of them may not even realize that Word collates this data automatically).
I've been outed once as a reviewer after I rejected a piece, but it was not due to anything as high-tech as MS Word metadata.

I faxed the journal -- which shall remain nameless -- my review. The journal then faxed it to the paper-writer -- who shall also remain nameless. The problem was that the journal's fax to the writer contained my department's fax number and identification -- and from there it was pretty damn easy to identify the referee.

Here's a link for potential referees about how to stay anonymous if you electronically submit your referee reports.

posted by Dan on 04.11.06 at 11:33 PM




Comments:

Any horror stories out their about reviewers who don't use Spellcheck?

posted by: Zathras on 04.11.06 at 11:33 PM [permalink]



The biggest problem with reviewers --by far --is them not actually doing the review, or getting around to it after being chased four times.

A fairly well known scholar in my field always identifies his reviews and insists the identification remain with the review. He believes it leads to a more honest, thoughtful and conscientious review. Frankly I think he is right, and I can't really see a downside to identifying yourself. Sure, academia is full of fragile egos, but it seems to me that anonymity is a often a shield for poor work.

On the other hand, submitted papers should be kept anonymous to avoid favoritism (or its oppisite).

posted by: Mitchell Young on 04.11.06 at 11:33 PM [permalink]



If a reviewer wants, googling an article's title will often reveal the article posted on the author's website, where it was given at a conference, etc. Just another way things aren't as anon as they may appear.

posted by: Dan on 04.11.06 at 11:33 PM [permalink]



"Collated"?!

posted by: P. Froward on 04.11.06 at 11:33 PM [permalink]



Even less technical methods of anonymization are not foolproof. Some years ago, I was sent a referee's report from a Dutch journal which was a xerox of what the referee had sent them, with his signature whited out. All I had to do was turn it over and look at the back while holding it up to a bright light: the name was clearly visible.

I was also surprised to see the report refer to me by name. Strict double-blind refereeing seems to be a mostly Anglo-Saxon thing. At least in my field (Classics), continental journals often use single-blind refereeing or even let the editor pick and choose whichever submissions he likes.

Even before the web, double-blind refereeing was never foolproof. I once asked an eminent classicist who had devoted his entire career to one of the major Greek authors (Homer, Plato, Thucydides, Euripides, one of those guys) how often he knew the name of the supposedly anonymous authors he was reviewing. He said about 50% of the time. After twenty years spent cultivating the same corner of the classical garden, he knew the stylistic tics and ideological approaches of many of his colleagues well enough that he couldn't help knowing who he was reviewing a lot of the time, even when he hadn't seen previous versions of their work at conventions and such.

Conclusion: double-blind refereeing is good, but far from foolproof, and the system still depends on the basic fairness of the referees.

posted by: Dr. Weevil on 04.11.06 at 11:33 PM [permalink]






Post a Comment:

Name:


Email Address:


URL:




Comments:


Remember your info?





Politics, economics, globalization, academia, pop culture... all from a untenured tenured perspective

Main home page
Main blog page
About Me
Search My Blog
Favorite Blogs
Book Recommendations
Books of the Month (Summer 2008)






Reviews of DanielDrezner.com:

"Sharp but informal commentary on politics and foreign policy." -- The New Republic

"Dan Drezner is terrific.... Excellent blog." -- Andrew Sullivan

"Dan's stuff is always worth reading." -- Eugene Volokh

"One of the essential weblogs." -- Gawker.com

"Old battle horse of the blogosphere." -- Jewcy.com

"Soft porn." -- Amitai Etzioni

"Spawned grave atrocities and vast destruction." -- Glenn Greenwald

"Monday morning quarterback... conservative robot... the very foundation of troubles in this country." -- not-so-random readers


Contact me at:
ddrezner@gmail.com
(But click here to read my e-mail policy)









Search the Site


Try advanced site search









Favorite Blogs

TNR's Open University
Jacob Levy
Glenn Reynolds
Andrew Sullivan
Mickey Kaus
Virginia Postrel
The Volokh Conspiracy
Josh Marshall
Crooked Timber
OxBlog
Real Clear Politics
Kevin Drum
Across the Aisle
Economist's Free Exchange
TNR's The Plank
NRO's The Corner
TAP's Tapped
America Abroad
Duck of Minerva
Opinio Juris
Brad DeLong

Jeff Jarvis
Mystery Pollster
Mark Kleiman
Meryl Yourish
Megan McArdle
Marginal Revolution
Michael Munger
Chris Lawrence
Matthew Yglesias
Hit and Run
Cold Spring Shops
Stephen Green
Outside the Beltway
Pejman Yousefzadeh
Laura McKenna (11D)
Elected Swineherd
Phil Carter
Joe Gandelman
Winds of Change
Andrew Samwick
Greg Mankiw
Dani Rodrik
Roger L. Simon
Tom Maguire
Greg Djerejian
The American Scene
Post Global
Democracy Arsenal




Recent articles online

"Foreign Policy Goes Glam."
The National Interest, November/December 2007

"Rise of the Hipster Statesmen."
Newsweek International, November 1, 2007

"The New New World Order."
Foreign Affairs, March/April 2007

"Mind the Gap."
The National Interest, January/February 2007

"The Grandest Strategy Of Them All."
Washington Post, December 17, 2006

U.S. Trade Strategy: Free Versus Fair
Council on Foreign Relations Press, September 2006.

Complete online article archive




Blog Archives

June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005
December 2004
November 2004
October 2004
September 2004
August 2004
July 2004
June 2004
May 2004
April 2004
March 2004
February 2004
January 2004
December 2003
November 2003
October 2003
September 2003
August 2003
July 2003
June 2003
May 2003
April 2003
March 2003
February 2003
January 2003
December 2002
November 2002
October 2002
September 2002

Academia
Area studies
Book club
culture
economics
fence-sitting
from Blogger
globalization
homeland security
international relations
law
Mediasphere
My very important posts
New Republic
outsourcing
personal
politics
Sports
The blog paper
the blogosphere
thesis ideas
Trade and Development
U.S. foreign policy
website maintenance

See full archives listing




Recent Entries

Someone keep Fleet Street away from Bill Clinton
It rivals Buckley vs. Vidal, I tell you
So.... are the Clintons morons?
The New York Times didn't ask me, but then again, that's why I have this blog
Monica Crowley's jet black pot
Al Qaeda is losing
Speaking of karma....
The blog post that writes itself
What made me laugh today
Where should Hillary go?




Site Credits