Small sample size.

Callis is full of it. The A's had seven first rounders, but they were almost all at the bottom of the first round. The A's had two picks in the top 25: Swisher at 16 and Blanton at 24. Both made the majors. In contrast 9 out of the other 23 top picks did not make the majors. For the top 25 the A's are clearly better than average, especially if you account for their picks being towards the bottom of the top 25. The first round had 41 total picks in 2002. The A's other five picks were 26, 30, 35, 37, and 39. Only one of those five, Teahen at 39, has made the majors. However out of the other eleven bottom picks made by the other teams, only one has made the majors. Once again the A's did better than average.

Dan,

Thanks very much for link and the endorsement!

Considering the old way of doing things - cranky old men looking at young players and trying to determine quality - and considering how uneven the results were, even from the best of scouts, isn't what Billy Beane tries at least worth looking into?

Or, to put this another way, I'd kill to have Beane in charge of my Cubs. Maybe his method won't ultimately play out, but its gotta be worth at least examining. Brushing it aside because the one draft covered by Michael Lewis might not have been perfect seems hasty.

Looking at the demographic breakdown of the BA top 100 list for 2006 gives some interesting data which may support Nightengale's comments.

Of the 100, 49 were from the real first round and only 6 were from the supplemental first round. 9 were from the second round. So in 2003 the A's had 4 picks in the *real* first round (1-30) and 3 in the supplemental. Even drafting all their prospects in the bottom half of the first round the A's had a 50% hit rate in 2002. They had a 33% hit rate on the supplementals. Judging from the BA demographics on the top 100 it appears that the supplementals have only a 30-40% probability of making the list compared to true first rounders assuming the proportions of supplementals to first rounders holds in most drafts.

Based upon the BA top 100 list (as a proxy for quality in the draft) I think we can assume that the higher the pick the more likely the player will make the majors. So the A's hitting with the 16th and 24th looks pretty good.

I gotta say, this is a pretty weak argument, and seems to betray nothing so much as a desire to cut Michael Lewis and Billy Beane down to size.

The main point, as far as I can tell, is that Baseball America doesn't think that the A's drafted well. Well, gosh, that's utterly shocking, because Baseball America is the premiere *scouting-based* prospect evaluation publication. Do you think there's a chance that they have some slight desire to prove Beane and Lewis wrong? Or even that they simply have different standards?

If you really wanted to prove that the Moneyball draft was weak, you'd need to do what the people in the comments are doing -- compare what they got out of it to what other people got out of it, given their opportunities. And maybe have an interesting notion of value (more so that place on the BA top 100 list). Which might take more than four years (Lewis is engaged in a seven+ year project to track that draft class).

Quoting someone from BA dismissively writing off that draft class as if that represents the consensus of all baseball opinion and thus some sort of received wisdom is just the kind of clubby insiderism which Moneyball so rightfully attacks.

And let me be clear: I don't know if the 2002 A's draft class will prove to be a winner. I don't know if Beane (and Podesta's) methodology for picking players in that draft was a good one. But I do know that if I was going to make a pronouncement on those questions, I would do some research, and look at some context.

And yes, Bonderman may have worked out -- but that doesn't mean that taking a risk on high school pitchers with a top draft pick is thus a good idea. It's a gamble, and even if you win once in a while on an outsider, it's still smart to play the odds.

Hey Dan, don't you think more data than one draft is necessary to tell you much anything about a particular strategy? Even the best strategies probably still have random chance as a large component.

The problem with Callis' analysis is that he doesn't have a control group. Is the group of prospects chosen better than what a non sabermetric front office would have chosen? One possibility is to check Draft previews from 2002. Who was BA recommending the A's take and how did those prospects fair?