Tuesday, February 28, 2006

previous entry | main | next entry | TrackBack (0)

In honor of Baseball Musings

My favorite baseball blogger, David Pinto of Baseball Musings, is celebrating his one year anniversary of being a professional blogger. Click over to make a donation and keep him at Baseball Musings on a full-time basis.

In honor of Pinto's anniversary, I'll raise a contrarian point about the utility of sabermetrics as a management tool that will warm the cockles of Steven Leavitt's heart. In a Baseball America chat about the top 100 prospects, Jim Callis responded to a very interesting question:

Q: Dave from Third Avenue, Manhattan asks: Jim, what is your take on the Moneyball draft, four years out. Swisher and Blanton seem to be doing just fine. Who else bears watching? Jeremy Brown?

A: Jim Callis: ...Given the number of picks the Athletics had in 2002, their Moneyball draft looks pretty average to me. They had seven first-round picks, and they got two solid big leaguers (Swisher, Blanton--both of whom were consensus first-round picks and not Moneyball choices out of the blue, by the way), a fringe regular (Teahen) and four guys who won't do much (McCurdy, Fritz, Brown, Obenchain). After that, there's not much beyond Shane Komine in the ninth round. Don't tell Michael Lewis, but it doesn't look like anyone revolutionized the draft in 2002.
If you re-read Lewis' chapter on the 2002 draft, you could go even further than Callis' assessment. In his chapter on the draft, Lewis recounts how Athletics GM Billy Beane went ballistic because in the previous year, the A's first-round draft pick was.... Jeremy Bonderman. Bonderman was the player to be named later in a deal that sent Ted Lilly from the Yankees to the Athletics. My guess is that Beane would be happy to have the current incarnation of that pitcher given his current price tag.

The 2002 Athletics draft should have been an "easy test" of the Moneyball revolution. The Athletics had a large number of draft picks, and no other team had really embraced the sabermetric philosophy to the extent that the A's had. If that draft failed to yield an above-average number of quality MLB players, what does it say about the utility of sabermetrics as a scouting tool?

The one out I can think of for Lewis is that Beane was able to sign those draft picks for way less than normal market value given when they were picked. There's definitely cost-effectiveness, which is really at the heart of the Moneyball argument. Still, that's pretty weak beer given the way Lewis wrote about the potential of that draft.

I'm certainly not suggesting sabermetrics is useless... but might this approach be overrated as a scouting tool?

posted by Dan on 02.28.06 at 11:45 PM


Small sample size.

posted by: rilkefan on 02.28.06 at 11:45 PM [permalink]

Callis is full of it. The A's had seven first rounders, but they were almost all at the bottom of the first round. The A's had two picks in the top 25: Swisher at 16 and Blanton at 24. Both made the majors. In contrast 9 out of the other 23 top picks did not make the majors. For the top 25 the A's are clearly better than average, especially if you account for their picks being towards the bottom of the top 25. The first round had 41 total picks in 2002. The A's other five picks were 26, 30, 35, 37, and 39. Only one of those five, Teahen at 39, has made the majors. However out of the other eleven bottom picks made by the other teams, only one has made the majors. Once again the A's did better than average.

posted by: Nightengale on 02.28.06 at 11:45 PM [permalink]


Thanks very much for link and the endorsement!

posted by: David Pinto on 02.28.06 at 11:45 PM [permalink]

Considering the old way of doing things - cranky old men looking at young players and trying to determine quality - and considering how uneven the results were, even from the best of scouts, isn't what Billy Beane tries at least worth looking into?

Or, to put this another way, I'd kill to have Beane in charge of my Cubs. Maybe his method won't ultimately play out, but its gotta be worth at least examining. Brushing it aside because the one draft covered by Michael Lewis might not have been perfect seems hasty.

posted by: Sam on 02.28.06 at 11:45 PM [permalink]

Looking at the demographic breakdown of the BA top 100 list for 2006 gives some interesting data which may support Nightengale's comments.

Of the 100, 49 were from the real first round and only 6 were from the supplemental first round. 9 were from the second round. So in 2003 the A's had 4 picks in the *real* first round (1-30) and 3 in the supplemental. Even drafting all their prospects in the bottom half of the first round the A's had a 50% hit rate in 2002. They had a 33% hit rate on the supplementals. Judging from the BA demographics on the top 100 it appears that the supplementals have only a 30-40% probability of making the list compared to true first rounders assuming the proportions of supplementals to first rounders holds in most drafts.

Based upon the BA top 100 list (as a proxy for quality in the draft) I think we can assume that the higher the pick the more likely the player will make the majors. So the A's hitting with the 16th and 24th looks pretty good.

posted by: Don Stadler on 02.28.06 at 11:45 PM [permalink]

I gotta say, this is a pretty weak argument, and seems to betray nothing so much as a desire to cut Michael Lewis and Billy Beane down to size.

The main point, as far as I can tell, is that Baseball America doesn't think that the A's drafted well. Well, gosh, that's utterly shocking, because Baseball America is the premiere *scouting-based* prospect evaluation publication. Do you think there's a chance that they have some slight desire to prove Beane and Lewis wrong? Or even that they simply have different standards?

If you really wanted to prove that the Moneyball draft was weak, you'd need to do what the people in the comments are doing -- compare what they got out of it to what other people got out of it, given their opportunities. And maybe have an interesting notion of value (more so that place on the BA top 100 list). Which might take more than four years (Lewis is engaged in a seven+ year project to track that draft class).

Quoting someone from BA dismissively writing off that draft class as if that represents the consensus of all baseball opinion and thus some sort of received wisdom is just the kind of clubby insiderism which Moneyball so rightfully attacks.

And let me be clear: I don't know if the 2002 A's draft class will prove to be a winner. I don't know if Beane (and Podesta's) methodology for picking players in that draft was a good one. But I do know that if I was going to make a pronouncement on those questions, I would do some research, and look at some context.

And yes, Bonderman may have worked out -- but that doesn't mean that taking a risk on high school pitchers with a top draft pick is thus a good idea. It's a gamble, and even if you win once in a while on an outsider, it's still smart to play the odds.

posted by: Dan Milstein on 02.28.06 at 11:45 PM [permalink]

Hey Dan, don't you think more data than one draft is necessary to tell you much anything about a particular strategy? Even the best strategies probably still have random chance as a large component.

posted by: chris on 02.28.06 at 11:45 PM [permalink]

The problem with Callis' analysis is that he doesn't have a control group. Is the group of prospects chosen better than what a non sabermetric front office would have chosen? One possibility is to check Draft previews from 2002. Who was BA recommending the A's take and how did those prospects fair?

posted by: Mark S. on 02.28.06 at 11:45 PM [permalink]

Post a Comment:


Email Address:



Remember your info?