Monday, November 3, 2003
previous entry | main | next entry | TrackBack (5)
Hey, it is a real story after all
This post is going after three audiences:
A) Loyal blog readers: My critique of the Center for Public Integrity's report has turned into this Slate article. Go check it out!!
B) New Slate readers: Stay for a while -- check out the site. There's a lot about politics and foreign policy, but there are also posts about porn, a list of quality book recommendations, posts that discuss the Hilton sisters, and adorable beagle pictures -- all the colors of the rainbow!! [You're shameless!--ed. Hey, I'm just working the room here!]
C) Those who want more about the CPI report: The following is tailored for those who are still skeptical about my argument. First, click over to my Friday post on the subject. Second, here are some additional rejoinders:
UPDATE: Those who care about the statistical methodologies involved should read these excellent comments by Ethan Ligon here, here, and here(Haynes Goddard has a post that makes a similar point). I respond here and here, to Ethan's satisfaction, I believe.
* For those who believe that the personal relationship between Sullivan and Haave reveal an obvious link, ask yourself the following question -- does this mean that the CIA dispatched Joseph Wilson to Niger merely because he was married to Valerie Plame, a NOC who worked on the nonproliferation division of the Central Intelligence Agency? [You saying there's something to that allegation?--ed. No, I think both of them are absurd.] Why is one allegation different than the other?posted by Dan on 11.03.03 at 02:42 PM
The real fun test would be to run a Heckman selection model among all contributors, with the dv in the selection equation being whether the contributor received a contract and the dv in the second-stage being contract size.
Downside would be that you'd have to collect more data than the CPI's "six-month project" did (which I suspect an enterprising grad student could do in a week, tops). Scarily enough, that'd probably be publishable somewhere (APR? PSQ?), even though I'm 99% sure you'd get null results for the effect of campaign contributions, just as the simple two-variable OLS did. (The one-variable model interpreted as OLS would only explain 3.7% of the variance in the dependent variable [.192^2]. A random number generator would probably do better in that small a sample...)
Of course, this is just the evil methodologist in me thinking out loud.posted by: Chris Lawrence on 11.03.03 at 02:42 PM [permalink]
My first impression is that running a correlation between contract size and campaign contribution size seems like a pretty poor way to assess the question of whether campaign contributions influenced the awarding of contracts. Offhand, it seems like certain companies are more or less qualified to do certain things - so the absolute size of a contract will be determined to a considerable extent by the amount that the US wants or needs to spend in that particular area. So, the fact that there isn't a significant correlation between contribution size and contract size doesn't necessarily tell us a lot about whether campaign contributions influenced the fact that certain companies received contracts.
It's quite possible, for example, that there was no major campaign contributor qualified to do the stuff that, for example, the Research Triangle Institute was brought in to do. In that case, the fact that RTI Institute didn't contribute much, but still got a big contract, would provide zero evidence against the hypothesis that contributions affected the awarding of some contracts.
Similarly, the fact that a substantial percentage of contracts may have been awarded without contributions playing a role doesn't rule out the possibility of corruption in other contracts. You state: "If the corruption argument is true, then the size of campaign contributions should be strongly and positively correlated with the size of government contracts." This seems dubious - the absence of such a strong and significant correlation across ALL contracts doesn't rule out the possibility that contributions played a large role in some contracts.
So, your method seems pretty inappropriate here. Maybe I'm missing something, but how exactly can you justify using a "contribution size vs. contract size" correlation as a method for determining whether or not the awarding of contracts was tied to campaign contributions in many cases?
I scanned over the CPI report, and I'm not sure it proves much - but unless you can further justify your method, I'm not sure your analysis disproves much, either.posted by: N V on 11.03.03 at 02:42 PM [permalink]
I saw your claim that there's no significant correlation between contributions and contract size over at Slate. I was surprised by the claim, since, with a sample size of several dozen and a bivariate normal distribution, a simple correlation coefficient of about 0.2 ought be significant--recall that the distribution of this statistic isn't normal!
I was sufficiently provoked to download the data myself, and compute the rank correlation coefficient. Since we needn't make any distributional assumptions to judge the significance, and we have no good reason to expect a linear relationship, this is certainly the more robust test. The result: Leaving in the people and firms who make no contribution (as I gather you do in computing your statistic), we get a Spearman rank correlation coefficient of 0.55. This is significant at the 1 per cent level, with a p-value of 0.003. Leaving out these people
All this leaves aside whether there's a causal relationship; I very much doubt that we have the data to tackle this question satisfactorily.posted by: Ethan Ligon on 11.03.03 at 02:42 PM [permalink]
I took Statistics but have forgotten almost anything useful for this conversation.
But here's a layman's question:
Considering that companies contribute to politicians for a variety of different reasons, and that the ability of the companies for the given tasks must have been at least a factor in the awarding process, wouldn't a positive correlation be unlikely, even if there was a high degree of cronyism?
The relevant question to me, which is not directly answered by the CPI report, is whether or not the companies rewarded contracts gave more on average to Bush than the ones who were denied contracts.posted by: Alex Parker on 11.03.03 at 02:42 PM [permalink]
It's a good thing that the appearance of impropriety, so long a standard in American life, is no longer an issue, so we can all slide safely into moral relativism.posted by: Lee A. on 11.03.03 at 02:42 PM [permalink]
"’Implied’ is the key word.”
Heck, implying wrongdoing in this particular instance was the beginning and end of the whole charade. There wasn’t enough evidence to even enter a courtroom. A judge would likely never let such a weak case get forwarded to a jury. Nonetheless, the harm done to people reputations will almost certainly linger on.
Does anyone truly believe that the liberal establishment will now bury and forget this shallow study? Not a chance. These folks push the envelope as far as they can. They violate the spirit of the law by pretending to honor its letter. Only a possible slander suit keeps them from going any further. A proper sense of right and wrong is meaningless to these ideologues.
"It's a good thing that the appearance of impropriety, so long a standard in American life, is no longer an issue, so we can all slide safely into moral relativism."
Moral relativism has nothing to do with anything. No, it's exactly the opposite. One should not be charging people with gross misbehavior unless the evidence is substantial. Cynically, this most recent incident is somewhat analogous to accusing someone with being a follower of Adolph Hitler merely because they hate smoking and love dogs!posted by: David Thomson on 11.03.03 at 02:42 PM [permalink]
Right and conservatives never use questionable evidence in their attacks, only liberals do. Oh and never violate the spirit of a law.
On a slight tangent, can someone familiar with the previous anti-profiteering laws explain (hopefully) that this is not as ugly as the headline would make one think?
I don't know enough to argue this very well, but at least on the surface it's pretty repulsive-looking. Can someone explain why it's actually harmless (being a free-market kinda site here and all) I hope?posted by: TG on 11.03.03 at 02:42 PM [permalink]
I'm not a statistician but it seems as if the CPI report could go to show that you just don't have to contribute as much to get a contract if you are good friends of the Admin. or for instance let's say the Vice President still holds stock in your company? That's not a big secret.
I have no doubt that the Center for Public Integrity decided the headline of its press release before looking at any numbers, but aren’t your numbers a little misleading, too? Isn’t it the case that Halliburton and Bechtel are exclusively or almost exclusively construction firms, while that kind of construction is a comparatively small part of GE’s business (that's a real question)?
For the record, my position on almost all of this is that I’d expect a pretty large correlation between donations and government contracts because bigger companies have more employees to donate money and also have greater capacity to do some big things or lots of small things. They way “corporate donations” are defined, I’d bet that there is a strong correlation between Iraq reconstruction contracts and donations to the Harry Browne campaign. That would tell us nothing more than big companies do more business than small companies. --sw
Kudos on the title to your Slate piece. R.E.M. fans everywhere rejoiced.posted by: Meursault on 11.03.03 at 02:42 PM [permalink]
I was disturbed by your article, but not because of its content, rather because of what was not said.
Your article perpetuated the same myth that the "Center For Public Integrity" built its case on: Corporations do not give political contributions, people do.
In most cases a political donation is a personal contribution based on the politics of an individual and it has absolutely nothing to do with that person's employer; and yet, because of this prevailing myth, it will be said that their donation came, not from them, but from the company they work for.
This plays into the hands of people like the "Center" in two ways: it inflates the power of corporations and makes them seem like greater political movers than they really are. At the same time, it deflates the power of individuals and dampens their voice. Instead of a political party or candidate getting the credit for having persuaded a large number of people to donate to the cause, those donations are written off as a sign of corporate-political corruption.posted by: Ann on 11.03.03 at 02:42 PM [permalink]
I hope that Mr Drezner will read very carefully the message above from N V. It says everything that needs to be said and totally destroys his argument. It is always fun to take a bunch of numbers and throw statistical analysis at it, but if the groups you choose to compare are not appropriate and the assumptions you make are unfounded, all you end up with is garbage. I am very sorry that Slate published something so transparently misguided.posted by: Terry on 11.03.03 at 02:42 PM [permalink]
Defense Contractors Announced Wednesday
Citing statistics surrounding Defense Contractors is a dangerous business. You might want to enlarge that grouping surrounding G.E. and Grumman Corp.
Their forte is not resconstruction so much as it is building new weapons systems being used in Iraq and Afghanistan.
Bechtel and Haliburton is known for services and reconstruction. Big difference and the did get the "lion's" share of the contracts.posted by: Loud Mouths on 11.03.03 at 02:42 PM [permalink]
Northrup Grumman, for example, is the primary contractor involved with the "Missle Defense Shield" subsidy. GE builds a lot of the weapons that made most of the mess that Haliburton and Bechtel are "rebuilding." It is disingenious to suggest that, simply because they're not getting the reconstruction contracts in Iraq that NG and GE aren't benefitting from the largesse of their campaign contributions.
Whether that is statistically significant, or simply morally significant, I'll leave to the statisticians.posted by: Sam Hutcheson on 11.03.03 at 02:42 PM [permalink]
Very nicely done, Professor.
"The scientific validity of [a] study is entirely aside the point."
oldman, 2 November 2003, "interesting survey" thread
posted by: Art Wellesley on 11.03.03 at 02:42 PM [permalink]
As an education professional, surely you see that the point of Professor Drezner's piece was to contest the logic/validity of the CPI piece, not to make a case for 'absolutely no influence' as you contend.
In so doing, the 'mathematics' that you can take-away from his piece, relative to the original that he was responding to, is this:
"if the groups you choose to compare are not appropriate and the assumptions you make are unfounded, all you end up with is garbage."
But, you already said that.
I, on the other hand, am NOT very sorry that you published something so transparently misguided as your above "Terry" seconding of the "N V" post - as I rather enjoy seeing all forms reason from academia displayed in public.
yours very respectfully,posted by: Art Wellesley on 11.03.03 at 02:42 PM [permalink]
My very first thought was: and where is "Table 3: Top 10 corporate contributors" ? (Though I would think they all ought to be at least top 100 or more, to actually do a just inquiry. And perhaps even more telling results would come from dividing the corporate contributors via amounts given to specific political parties.)
Since the implication of alleging "impropriety" assumes that the contracts were bought & paid for via the contributions given G.W. Bush &/or his party, if he was in fact trying to pay off on such political favors owed for corporate gifts, you should be able to go down the missing third table & see if any of the top ranked businesses (in a qualifying field) were skipped over &/or those who supported the opposition party specifically snubbed.
I'd like to argue that not all "cronyism" is bad. If my car breaks down & I need it fixed ASAP I
Who in their right mind doesn't do exactly that whenever possible in the context of their lives?! Be it auto mechanic, plumber, electrician, etc. -- if we don't personally know someone, we ask friends if they can refer us to someone.
And in the case of military issues / contracts there are the security issues & liabilities that need be taken into consideration too. If a corporation happens to have leaders with military experience, they'll know how to work with our guys & how to help cover the security problems this first wave of reconstruction folks are facing. UN personnel, aid workers & Iraqi civilians are complaining we aren't doing enough to "protect" them -- but we don't have enough military personnel there to babysit everybody!
Can you imagine what would have happened if we had given a contract to a French or German company -- how loud their outcry would have been at the first sign of their workers being at risk -- how quickly they would have bailed out of the project? Too much rides on us getting things fixed as quickly as possible. If ever there was a time when a degree of "cronyism" was justifiably called for, this is it!
I'm not exactly a Bush fan, but on this matter I'm going to believe he made the best possible choices for all the right reasons -- till someone firmly proves otherwise.
BTW my husband got to "played in the sandbox" & duck SCUDS in 1991, so I'm probably biased about the importance of these contractors not being an Achilles' heel for our military personnel. But if you have any campassion for our service members, you should too. Though I have no clue as to how that can be figured into your stats as a weighed "variable". LOL
Nice to see a "geek" going to bat for us, instead of against us!
You raise an interesting point. My understanding, however, (I'll confess we're skating close to the edge of my statistical competence)is that both Spearman and Kendall-tau are used for ordinal, categorical variables. The data provided was cardinal.posted by: Dan Drezner on 11.03.03 at 02:42 PM [permalink]
My understanding, however, (I'll confess we're skating close to the edge of my statistical competence)is that both Spearman and Kendall-tau are used for ordinal, categorical variables. The data provided was cardinal.
This is correct, but Pearson correlations test only for a linear relationship and are extremely sensitive to "outliers." With bivariate data likes this, nothing beats a graphical display (scatterplot).
The real fun test would be to see if changes in the bidding process have any effect on the relationship between ties to the Administration (measured in a variety of ways)and contract awards. This would get most directly at the charge leveled at the Administration. But we would obviously have to wait a while.posted by: Erik on 11.03.03 at 02:42 PM [permalink]
I still don't understand why Drezner thinks that the correlation between campaign contributions and the size of the contract is the relevant issue here. As far as I can tell, it is not what the CPI report is claiming. Because 1) the abilities of the company would have been an issue, and 2) Companies donate for many other reasons besides this particular project, it seems that a positive correlation would not be likely.
What is the issue is whether or not connected companies had an easier time getting contracts than non-connected companies.posted by: Alex Parker on 11.03.03 at 02:42 PM [permalink]
Two points on the CPI "study". First, all corporations large enough to be players in Iraq make political contributions as a matter of self-protection. Microsoft used to be a company that did not typically hire lobbyists or make significant political contributions, and look what happened to Microsoft. Microsoft now does both, in spades. Second, is CPI only looking at contributions to Republicans? large corporations typically contribute significantly to both political parties, as a matter of self protection, but it seems unlkely a Republican administration motivated by cronyism would be awarding conracts to corporations that contributed heavily to Democrats. Where's the breakdown by party?posted by: jimhanavan on 11.03.03 at 02:42 PM [permalink]
SBW above: "I'm not exactly a Bush fan, but on this matter I'm going to believe he made the best possible choices for all the right reasons -- till someone firmly proves otherwise."
Why on earth would that be your default position, considering the extensive history of tricks, deceptions and obfuscations uttered to date by this WH? Egad.posted by: Dee Mento on 11.03.03 at 02:42 PM [permalink]
Following up on my earlier comment.
I'd say that your claim that there's no
However, this distributional assumption is
In sum, the claim that "the bivariate correlation
-Ethanposted by: Ethan Ligon on 11.03.03 at 02:42 PM [permalink]
As a backup, I ran a mulivariate OLS regression with contract size as the dependent variable and the two independent variables provided in the CPI report...
so, uh, that's nice, but it seems to overlook the "revolving door" hypothesis which provides a perfectly plausible explanatory factor... in fact wouldn't your analysis tend to conceal correlations corresponding to informal (cf. cosa nostra) connections that predate the data points in question?
I have no dog in the "correlation between campaign contributions and contracts" fight, but I do want to make sure that you're not implying that your numbers affect any other hypothesis or support the likelihood of a null hypothesis, cause that's clearly not the case.
also, unless you have some indication that Wilson and/or Plame benefitted financially from Wilson's trip to Niger, comparing the Sullivan/Haave scenario to the Wilson/Plame scenario seems a little disingenuous.
oh wait... perhaps what you're saying that the profit motive has only an insignificant effect on people's behavior? ;-)posted by: radish on 11.03.03 at 02:42 PM [permalink]
.058 > .05 in my book, Ethan. ;-)
In all seriousness, logging the variables might be appropriate; another appropirate approach might be tobit, although the appropriate censoring value is unclear (as contributions
Generally, though, OLS estimates are robust with non-normal data, although I'd probably use HCSEs in this case.posted by: Chris Lawrence on 11.03.03 at 02:42 PM [permalink]
Hmm. I meant to say something about contributions less than $200, but I didn't escape my less-than sign. Sigh.posted by: Chris Lawrence on 11.03.03 at 02:42 PM [permalink]
I take Dan's claim to be that there's no
In my earlier comments I've documented a
-Ethanposted by: Ethan Ligon on 11.03.03 at 02:42 PM [permalink]
Not to get to technical, but I believe that Ethan's .3135 estimate for the correlation coefficient is actually quite conservative: I get .5888 after transforming both variables and entering a zero value for the missing data created by the non-contributors. This accounts for a rather impressive 34% of the variation in the size of the contracts.
While it still doesn't demonstrate a one-to-one relationship between contracts and donations, it certainly strongly suggests a pay-to-play atmosphere dominates these contracts.posted by: Neal Caren on 11.03.03 at 02:42 PM [permalink]
Ethan and Neal,
I crunched the numbers as you suggested, and indeed there does appear to be a stronger and positive correlation between campaign contributions and contracts (although the scatterplot still looks pretty weak). Whether it’s significant depends on whether the zero-values for campaign contributions are included. Nevertheless, point taken.
However, what’s fascinating is what happens when you run a multivariate regression on contract size with both campaign contributions and past contracts awarded with the log taken of all the variables. In this equation, past contracts turns out to be highly significant (p > .00005). Campaign contributions take on a negative but statistically insignificant coefficient (p > .45) if the zero-values are not included. If the zero-values are included, then the coefficient turns positive but remains insignificant (p > .30). The key thing is, once past contracts are factored in, campaign contributions wash out.
So, I think I’m still on strong ground with what I wrote in Slate. However, I agree that the picture is somewhat more complex than I suggested with the correlation coefficient. I appreciate the methodological criticism, and have learned a valuable lesson – crunch the numbers in a lot of different ways before publishing anything statistical in a public forum. I do this with my scholarly work, but it applies to this kind of writing with equal force.
Your regression results are much more
So how about updating/correcting your _Slate_
I did the same ranking based analysis as Ethan Ligon with the same result. I find your revised analysis less convincing than he does. I don't believe that any analysis based on raw revenue data is trustworthy. The range in dollars is just too wide, which means that your results are dominated by the handful of huge contractors and contributors.
As you noted in a Slate footnote, a better analysis would require data on a wider class of potential contractors that includes both contributors and non-contributors to Bush. Then you, or CPI, could try several approaches, including a rankings-based approach or logistic regression, to see if obtaining a reconstruction contract is based on historical contributions to Bush.posted by: Keith Holzmueller on 11.03.03 at 02:42 PM [permalink]
Taking the natural log of the relevant variables solves the extreme outlier problem. When you do that, a scatterplot shows there are no real outliers in the modified data.
You're right about the superior way to do the analysis, which was why I put that footnote in the Slate article. But do bear in mind that I was directly responding to the CPI's allegations and not relying on my own database.posted by: Dan Drezner on 11.03.03 at 02:42 PM [permalink]
Dan: "The key thing is, once past contracts are factored in, campaign contributions wash out."
Well, as I understand it, this doesn't rule out the possibility that past campaign contributions had anything to do with winning past contracts. I don't know if it's feasible to determine that statistically or not, though.posted by: fling93 on 11.03.03 at 02:42 PM [permalink]
I'm glad that you've crunched the numbers in a few different ways--but you're still crunching bad data. The observations are heterogenous. As others have pointed out, GE builds weapons, owns NBC, and has a dozen reasons to buy influence that wouldn't touch its Iraq reconstruction contracts. You can't look at the link between GE's contributions and its Iraq reconstruction contracts and reject (your version of) the CPI's hypothesis. The regression is meaningless. You might as well have tested the correlation between contributions and reconstruction contracts for every Fortune 500 company. Amazing! Pfizer didn't get a single penny in Iraq contracts!
As I'm sure you know, the methodological issues associated with statistically documenting corruption are tremendous. (One solid attempt to get past them can be found in the "Jeffords Effect" paper at http://www.people.fas.harvard.edu/~jayachan/papers.html.) Your Slate article breezes right past them. Please, post a retraction on Slate--and send an e-mail to David Brooks asking him to do the same.posted by: Ben Wikler on 11.03.03 at 02:42 PM [permalink]
Post a Comment: