Experiments show this is the best way to win campaigns. But is anyone actually doing it?

There’s a refrain we hear about political campaigns every election cycle: "this year, campaigns waged an unprecedented ground game, having a face-to-face conversation with almost every single voter."

Baloney. As academics who study campaigns, we hear this claim all the time. But we also know it’s important to investigate whether data backs it up. We did. And it doesn’t. In fact, there’s a paradox at the heart of American campaign craft. Mountains of rigorous research show that campaigns should be having personal conversations with voters at their doors. But, campaigns spend almost all their money on TV ads — and, every year, most voters say they’ve never had a conversation about the election at their door. What gives?

Why campaigns’ "ground game" matters

By far the most effective way to turn out voters is with high-quality, face-to-face conversations that urge them to vote. How do we know? Nearly two decades of rigorous randomized experiments have proven it.

Alan Gerber and Don Green ran the first of these "field experiments" in 1998. The professors randomly assigned voters to receive different inducements to vote: some received postcards, some received phone calls, some received a visit from a canvasser, and some received nothing.

The experiment found that voters called on the phone or sent postcards were not noticeably more likely to vote than those sent nothing. But canvassing was different. Just one in-person conversation had a profound effect on a voter’s likelihood to go to the polls, boosting turnout by a whopping 20 percent (or around 9 percentage points).

The nearly two decades since Gerber and Green’s first experiment have consistently borne out their finding that personal conversations have special political potency. Hundreds of academics and campaigns have tested the impacts of various campaign tactics with randomized field trials. High-quality canvassing operations emerge as consistent vote-winners. On the other hand, impersonal methods have consistently failed to produce cost-effective results, no matter how you slice the data or which populations researchers examine.

Quality counts: field operations’ "knock" numbers don’t tell you much

Volunteers from the Yes campaign speak with a voter as they canvass for support for the Yes vote in the Pilton area of Edinburgh on September 16, 2014 in Edinburgh, Scotland. (Matt Cardy/Getty Images)

Given the widely acknowledged importance of a good "ground game," campaigns like to tout statistics that show they’re knocking on huge numbers of doors. These statistics can make their ground games sound quite substantial.

But, in reality, large "knock" numbers often conceal lackluster ground games. Why? Campaign operatives often rush through neighborhoods, hurrying to rack up impressive numbers of "knocks." However, these hurried efforts often fail to reach most voters at all and entail only perfunctory interactions with the voters they do. Campaigns’ ground games can thus sound sizable in terms of "knocks" when they haven’t had any conversations with voters at all.

And, to actually affect voters, research shows that having an actual conversation is crucial. Canvassing seems to work best when voters who don’t care much about politics engage in a genuine conversation about why voting is important. So, when canvassers rush through scripted interactions, just trying to cram their message into voters’ minds, the impacts they leave are minimal — voters might as well have been sitting through a television ad. On the other hand, research has consistently found that authentic interpersonal exchanges usually have sizable impacts.

But facilitating that breed of genuine personal outreach isn’t what many "field" campaigns actually do. Green has seen this in practice. He has found that many canvassing operations have effects "smaller than what we obtained from our initial study or in our follow-up experiments with seasoned groups such as ACORN." But, Green went on to say, "When I'd inquire about the details of these sub-par canvassing efforts, I would often discover that the scripts were awkward or that there was limited attention to training and supervision."

This suggests a picture that should frighten candidates, campaign managers, and donors alike. Even if field operatives have racked up millions of "door knocks," when one looks under the hood of these operations, there often isn’t much reason to believe they’re having many quality conversations with voters at all.

Voters aren’t seeing the ground game

Another reason to doubt campaigns are running good ground games? Voters don’t appear to be seeing them.

A political organization running a field campaign shared data with us that helps quantify just how invisible the ground game is to the very voters supposedly being inundated with it. (The organization requested anonymity when making public their internal research findings.) During October 2014, this organization ran a field campaign in a hotly contested Midwestern gubernatorial race. According to most accounts, this gubernatorial race witnessed the same all-out ground game as other elections this year, and this organization should thus be thought of as only one of many blanketing supportive voters with personal conversations urging them to vote.

What this organization did allows us to critically evaluate how widespread the ground game was — it ran an experiment in which some voters it was targeting were randomly assigned to receive a knock on their door from organization field staffers while others, a "control group," received no contact from the organization (but still received identical efforts from other groups). After the election, this organization conducted an ostensibly unrelated survey in which they asked voters in the two groups what campaign contact they recalled receiving over the last few weeks from any political organization.

The results? The "control" group who received no contact from this organization remembered getting a knock on the door from any campaign only about 21% of the time. But just one conversation at the door from this organization doubled that figure, to over 40%. With just one contact yielding such a large increase, it's hard to believe the ground game in that race reached anything near saturation.

Source: Survey of voters targeted by campaign in midwestern gubernatorial race.

But what about for mail, phone, and television-based appeals? The numbers for these modes show exactly the opposite: voters were saturated. This organization found that around 9 in 10 voters it targeted recalled receiving phone calls, mailers, or seeing TV ads about the same election. The disparity between these numbers and the same figures for field raise questions about the idea that the ground game is already in full swing.

Source: Survey of voters targeted by campaign in Midwestern gubernatorial race in control group who did not receive canvassing.

(Data from the Cooperative Congressional Election Study reveal a similar disparity — a majority of Democratic voters in swing states in 2010 and 2012 recall receiving phone and mail contact, but almost no voters recall someone knocking on their door.)

Spending more on unproven TV ads than on canvassing

The same disparity between field and other techniques manifests in patterns of campaign spending. A recent investigation by the New York Times provides a window into how SuperPACs spent their dollars over the last three election cycles. The results are puzzling. Over 80% of these groups’ spending went to TV advertising, followed by mail and online advertising to round out about 90% of spending in total. Finally, in a distant fourth, came field work — capturing less than 5% of campaigns’ budgets.

Somehow, many campaigns aren’t managing to spend much more on the most effective form of voter contact than on radio.

But, even though campaigns spend a very large share of their budget on TV ads, the research on the impacts of TV ads doesn’t bear out the idea that they powerfully influence elections:

First, there’s little evidence supporting the idea that TV ads can mobilize voters to turn out. In 2008, Jon Krasno and Green exploited quirks in media market boundaries to measure the impacts of presidential advertising. Ryan Enos and Anthony Fowler have examined the impact of the presidential campaigns' TV ads in a similar manner. The results? Voters who receive the heavy volume of TV advertising associated with presidential campaigns are no more likely to vote than voters who see barely any. When it comes to turning out new voters, there’s not much evidence TV ads are of much use. (One exception is a study by Vavreck and Green on Rock the Vote’s television ads, which found mild effects among young voters.)

There is similarly limited evidence that TV ads have an enduring impact on voters’ attitudes towards candidates. Yes, ask voters how they feel within hours of seeing a TV ad, and we sometimes see evidence that they’ve been swayed. But these effects usually fade quickly. In one study, Seth Hill, James Lo, Lynn Vavreck, and John Zaller examined the impacts of presidential television advertisements and found that their effects disappeared within days at most. Likewise, in a collaboration with the Rick Perry for Governor campaign, Gerber, Green, and James Gimpel and Daron Show found that Perry’s ads had a noticeable immediate effect but left no lasting trace.

Even if TV ads provide fodder for much punditry or look impressive in a focus group, there’s not much reason to believe they have lasting impacts on voters’ views or behavior.

Why aren’t more campaigns focused on having personal conversations with voters?

Campaign consultants, like Republican Mike Murphy (left) and Democrat Bob Shrum (right) typically make more when campaigns spend on ads than when they spend on field operations. (Alex Wong/Getty Images for Meet the Press)

We academics are still scratching our heads about this one. Here, we’ll mention just a few possibilities.

First, managing a canvass operation is difficult and requires considerable recruitment, training, and supervision. It’s a lot easier to write a check to an ad agency or mail firm.

It’s also gotten harder to raise a field army. Decades ago, a rich network of civic organizations — think churches, Elk Lodges, and labor unions — could supply ample volunteers for field work. But, as these organizations’ memberships have flagged, professionally-managed, centralized, DC-based groups with weaker grassroots ties have tended to take their place. As a result, knocking on millions of doors now requires recruiting tens of thousands of temporary field staffers or new volunteers. When faced with a logistical challenge of that scale, it becomes mighty appealing to write a check to an ad firm instead.

There’s also a more cynical possibility — campaign consultants urge campaigns to spend more on ineffective tactics because it boosts their bottom line. Candidates and campaigns rely on consultants’ expertise when allocating precious campaign resources. But many consultants take a cut of ad fees, making a healthy commission when campaigns squander their resources on TV. On the other hand, waging field campaigns tends to be a low margin business and thus prove less financially appealing for consultants to recommend to clients.

Campaigns can do better

As effective as high quality field campaigns are today, they’re likelier to get even better as the research improves. Successful turnout interventions also seem to have lasting impacts on individuals, leading them to become lifelong voters, as well as on their cohabitants. But to take advantage of these innovations, campaigns need to seriously increase their focus on field.

The good news is, the necessary financial resources for waging real ground games are already available — campaigns just have to spend their money right. Consider what would happen if campaigns diverted just some of the money they currently spend on TV towards field. Nearly $1.2 billion was spent on TV ads during the 2014 election cycle, capturing about a third of campaign spending. Imagine if campaigns diverted just 30% of that amount to field, for a $350 million ground game — many times more than the amount campaigns actually spent on field this year. Field operatives can often be hired for about $20 an hour (including overhead) and could have two high-quality 20 minute long conversations with voters every hour, for about $10 per conversation. That all adds up to a staggering reality: campaigns could have had a 20 minute conversation with every single registered voter in a state with a close Senate race — and still afford to blanket the airwaves with ads.

Waging a high-quality ground game isn’t easy — but no one said winning elections was. Before 2016, candidates and campaign consultants need to take a hard look at the science, lest the ground game take a back seat yet again. We may need to knock on their doors, too.

David Broockman and Joshua Kalla are graduate students in the Department of Political Science at UC Berkeley.

Update: An earlier version of this post cited research by Michael LaCour and Donald Green on the effects of canvassing campaigns on opinions about gay marriage. That research has since been discovered — by this post's authors — to have been faked by LaCour.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via