Eric Cantor's pollster McLaughlin was off by 45 points but he hasn't lost any clients-NY Times

.
9/2/14, "You Lose, We Win: Consultants Profit Even When Candidates Underperform," NY Times, Brendan Nyhan

"Labor Day signals the beginning of the fall campaign for both political candidates and the consultants whom they pay hundreds of millions of dollars to help them win in November. Will these hired guns be held accountable for their performance on Election Day?

The experience of John McLaughlin, the pollster for the former House majority leader, Eric Cantor of Virginia, suggests that the consequences of consultant failure are often minimal. As the Washington Post’s Ben Terris noted, Mr. McLaughlin was “historically wrong” about Mr. Cantor’s defeat in a June primary, missing the final margin by approximately 45 percentage points, but hasn’t lost any clients as a result.

A closer look at the research on political consultants suggests that Mr. McLaughlin’s experience is typical. Firm reputations and client relationships are highly consistent over time and show little responsiveness to results, particularly in terms of the share of the vote that a client receives, a much more informative metric than wins and losses.

Why is the campaign consultant market so inefficient? First, it suffers from what social scientists call an asymmetric information problem. The buyers (candidates) know much less about the service being provided than the sellers (consultants) — the same reason that consumers have a hard time making informed choices on health care. As a result, consultants are often hired based on their prominence or relationships with party insiders rather on than their past performance or other measures of quality.

Second, there are few good sources of data about firm performance. Consultant client lists are often not broadly publicized, preventing firms from being held accountable for their performance in past campaigns. Though a few high-profile failures like Mr. McLaughlin’s receive substantial attention, many other relationships may not be well known, preventing candidates or other clients from learning about a consultant’s performance.

The most widely known analyses of consultant performance are the annual scorecards published in the industry trade magazine Campaigns and Elections. The magazine uses firm surveys as well as interviews and other research techniques to compile a list of consulting firm clients and the outcomes of their campaigns each election cycle.(Here, for example, is the scorecard from the 2008 edition, which was published when the magazine was known as Politics.)

However, when my co-author Jacob Montgomery and I compared the relationships reported in the scorecards with campaign expenditure data later made available by the Federal Election Commission, we found that most consulting firms who received payments from candidates were not listed in the scorecards. As a result, the reported win/loss record of a firm may be a misleading indicator of its performance in a given election cycle. (A much smaller number could not be verified, suggesting that underreporting is a more significant problem than overreporting.)

Another problem is the focus on the winner of the race. While the outcome of the race is what matters most to the parties and candidates and thus receives the most news media coverage, wins and losses are an imprecise measure of consultant performance. For example, some incumbents might only be narrowly re-elected in races that they should win easily, while non-incumbents or other weaker candidates might only narrowly lose. As the political scientists Gregory J. Martin and Zachary Peskowitz argue, a better way to think about consultant performance is whether a candidate received more or less of the vote than we would have expected given the characteristics of the district and the circumstances of the race.

Rather than rewarding vote-share performance, the market appears to respond in only a limited way to wins and losses. At best, it rewards wins outside of safe seats. Dr. Martin and Dr. Peskowitz find that firm revenues increase significantly for wins in potentially competitive races (i.e., excluding those in which an incumbent faces an inexperienced challenger). By contrast, revenue is not significantly affected by whether a firm’s candidates performed better or worse than expected in terms of vote share. As a result, underperforming consultants are not punished and firms’ market shares remain quite steady between election cycles (as with Mr. McLaughlin’s client list).

What the parties need, as as some have suggested on Twitter, is the equivalent of the value over replacement player statistic from baseball — some method to better estimate how much value a consultant adds (or subtracts) relative to a hypothetical alternative who could be hired instead.  

It’s very hard to separate the influence of consultants from other factors affecting campaigns,  

but given the influence that consultants have over the balance of power in Congress and

the dissemination of tactics and issues among candidates, it seems likely that the parties 

will start demanding results."

 
"Brendan Nyhan is an assistant professor of government at Dartmouth College."...

======================

May 15, 2007, "Value Over Replacement Player," Baseball Prospectus, by Derek Jacques