CA120: The changing nature of public polls

The public opinion polling industry in many ways is at a crossroads.

For years, public polls were mostly conducted by means of live telephone interviews using a “random digit dialing” (RDD) sampling method that attempts to give all respondents an equal chance of being called.

These probability-based procedures, long the gold standard of the polling industry, are being increasingly displaced by online Internet-based polls, most of which are based on large-scale online panels of pre-recruited respondents.

The reasons for the changeover are many. Foremost among them is the steady decline in the public’s willingness to participate in surveys conducted by phone. Response rates for telephone surveys, which once routinely surpassed 50%, are now typically in the single digits.

Technological changes have also played a role. Cell phones are now nearly ubiquitous, with over 90% of U.S. adults now using them, while fewer and fewer use traditional landlines. While pollsters have generally been successful in tailoring polls to integrate interviews conducted by cell and landline telephones, this has come at a cost.

When dialing cell phone listings, interviewers must individually hand-dial each cell phone listing, since federal law prohibits calling them with automated dialing devices.  This significantly increases the costs of cell phone interviews. In addition, the ever-declining proportion of Americans with residential landline phones has made even polling these phones more expensive as it becomes more difficult to find eligible adults using RDD samples.

This has led to a wholesale reordering of the types of public polls being reported in the news media.

The increasing costs of conducting polls by telephone are coming at a time when the nation’s news media, the traditional sponsors of public polls, have been facing significant budgetary constraints. The huge generational decline in the readership of newspapers has forced cutbacks, consolidation, and even closure of many of the nation’s newspapers, long the chief underwriters of state and local public polls.  In addition, the increasingly diffuse way the public gets its news, fostered by the rapid expansion of social media, has negatively affected the bottom line of the major televisions networks, which in turn has led to cutbacks to their once large polling budgets.

Yet, today’s 24/7 news cycle has increased the demand for news information, and with it, the media’s desire for publishing and broadcasting the latest polls. Some established pollsters have been able to meet this demand through funding from foundations and academic institutions.  However, many others are turning to less expensive, online data collection methods.

This has led to a wholesale reordering of the types of public polls being reported in the news media. According to a post-election round up by The Huffington Post, a majority of all public polls reported by the media in the 2016 presidential election were conducted online or using polling methods other than by traditional RDD telephone surveys.

As we now turn our attention to the state’s upcoming 2018 elections, there is greater than usual media interest in the public polls.

These industry-wide changes have coincided with recent changes in my own professional career.  After nearly forty years at the statewide Field Poll, the last twenty as its director, in early 2017 I shifted gears and became director of the Berkeley IGS Poll, housed within the newly established Jack Citrin Center for Public Opinion Research at the Institute of Governmental Studies (IGS) on the University of California, Berkeley campus.

While my polling focus continues to be California, my new position at IGS has afforded me greater opportunities to experiment with different methods of data collection in our polls.

During my first year,  the Berkeley IGS Poll completed five large-scale statewide surveys using a variety of polling methodologies.  One was administered using traditional RDD telephone polling methods.  A second was conducted by telephone but by sampling voters randomly from the state’s registered voter database.  Two others were completed online using pre-recruited opt-in panels of the state’s electorate.  And, a fifth was completed online by sending email invitations to random samples of registered voters whose email addresses are appended to their voter registration records.

As we now turn our attention to the state’s upcoming 2018 elections, there is greater than usual media interest in the public polls.  This is driven by competitive contests for the state’s top-of-the-ticket elections for Governor, U.S. Senate, Lt Governor, and Attorney General, as well as by interest in the state’s congressional elections, as California is ground zero in the Democratic Party’s efforts nationally to win back control of the House of Representatives. To meet the demand, the Berkeley IGS Poll will be leveraging the knowledge and experience gained from our experimentation with these various data collection methods when deciding on which method is employ for our upcoming surveys.

When polling in the year ahead, IGS will partner with Political Data, Inc. (PDI), who will provide us with samples of registered voters for our surveys.  Use of the voter file as a sampling source offers a number of advantages over pre-election polls drawn from other sources.

Based on our recent experience, we believe that conducting polls online, especially those drawn from samples of voters with email addresses, holds considerable promise.

For one, since the voter file is derived from official state and county records, we can be certain that all respondents in our samples are indeed registered voters.  The file also provides accurate information about each voter’s participation in past statewide primary and general elections, which is critical to the identification of likely voters in pre-election surveys. By contrast, pre-election polls based on RDD telephone samples or using pre-recruited opt-in panels of survey participants must rely on each respondent accurately reporting their registration status and their voting history.  Unfortunately, this is not always the case.

The voter file also provides greater opportunities for polling voters within California’s local political jurisdictions, such as within its congressional, state senate or assembly districts, or within a city or county, since each jurisdiction is accurately labeled on the voter file. By contrast, the increasing need to poll voters on their cell phones makes it almost impossible to complete an RDD telephone survey within local jurisdictions, since the telephone listings assigned to cell phones do not readily identify the geographic location of the voter.

Earlier this year the Berkeley IGS Poll took advantage of voter file-driven methodologies when it launched a series of polls in several of the state’s highly competitive congressional districts. Because of the huge media interest and attention being given to these elections, we are already making plans to expand the number of these congressional polls we’ll be conducting in the months leading up to the November general election.

In addition, when preparing its voter samples, PDI also provides access to the extensive demographic and political characteristics of the electorates under study. This information is invaluable when developing statistical weights to align our registered voter samples to characteristics of the larger registered voter population.

Based on our recent experience, we believe that conducting polls online, especially those drawn from samples of voters with email addresses, holds considerable promise.

The ability to present voters with a complete list of the candidates running for office has particular relevance in the upcoming June primary.

Use of the email invitation method to carry out polls at UC Berkeley actually pre-dates my arrival on campus, and has been in use there by academic researchers, both students and faculty, for a number of years.  Over time, this sampling method should become increasingly attractive, as larger numbers of Californians include their email address when registering or registering to vote.  According to PDI estimates, nearly six million of the state’s voters now have an email address appended to their voting record, and that number is increasing at a rapid rate.

When developing samples of registered voters with email addresses, we first stratify the samples of the registered voters invited in an attempt to retrieve a proper balance of those likely to respond across different segments of voters. Email invitations are then sent to voters in English and Spanish, and each voter is sent reminder emails over an approximately seven to ten day period, seeking their participation in the poll. After data collection is completed, detailed weights are applied to further refine and model the sample of poll participants to parameters of the larger registered voter population.

The self-administered nature of these polls also offers advantages over the telephone survey method when the objective is to measure candidate preferences in an upcoming election.  This is because when completing a poll online, voters can be presented with the entire list of candidates running for each office, along with the job titles and party designations that will appear next to their names on the election ballot.  By contrast, pre-election telephone polls typically measure voter preferences using an abbreviated list of only the most prominent candidates running in each race.

The ability to present voters with a complete list of the candidates running for office has particular relevance in the upcoming June primary. This is because the number of candidates running for office under the state’s new top two primary election system can potentially be quite large, since all candidates from all parties are listed together on one ballot.  The statewide races for U.S. Senate and governor are examples of this, with 32 candidates running for U.S. Senate, and 27 in the governor’s race.

A review of the five Berkeley IGS Polls conducted in the governor’s race in 2017 shows that voter support for Villaraigosa was six percentage points lower on average in the three Berkeley IGS Polls conducted online last year than in the two surveys administered by telephone.

On the other hand, online polls also present pollsters with their own set of challenges.

For example, when administering a December 2017 survey of the state’s electorate by telephone, we also administered a second companion poll online to a separate set of voters using the email invitation method. Both polls were drawn from samples of registered voters drawn from the PDI data file and voters in both surveys were asked a similar set of questions. The data files from both surveys were also similarly weighted to align each to the demographic, geographic and political characteristics of the overall registered voter population.

While most of the statewide findings from the two companion polls were similar, some significant differences were observed within key voter subgroups, even after the alignment weights had been applied.  Some of the largest differences were among the Latino voters in the two samples.

Compared to the telephone survey, the online poll appeared to retrieve as somewhat less diverse set of Latino voters, underrepresenting its Spanish-language dominant and less educated segments. Interestingly, this same deficiency was observed earlier in the year when the Berkeley IGS Poll administered two of its statewide polls online using the pre-recruited opt-in panel method.

We believe these differences are largely due to the fact that Spanish-dominant and less educated Latino voters are less likely to have access to the Internet, making them less likely to be captured in polls conducted online as compared to telephone surveys.

Differences in the composition of Latino voters in California can have a significant impact when assessing voter preferences in an upcoming election, especially in races whose candidates have particular appeal to lower socioeconomic Latino voters.  One such candidate is former Los Angeles Mayor Antonio Villaraigosa in the governor’s race. A review of the five Berkeley IGS Polls conducted in the governor’s race in 2017 shows that voter support for Villaraigosa was six percentage points lower on average in the three Berkeley IGS Polls conducted online last year than in the two surveys administered by telephone.

Potentially offsetting this bias is the fact that Spanish-dominant and less educated Latinos are less likely to vote than other Latinos, especially in primary elections. Nevertheless, when conducting future Berkeley IGS Polls online, we will make special efforts and devote extra polling resources to improve the representation of Latino voters in our samples.

Working at U.C. Berkeley is giving me greater opportunities to explore and evaluate the pluses and minuses of different data collection method when conducting our public polls. With the university’s long ingrained culture of innovation and experimentation, and the encouragement and support of the Institute of Governmental Studies’ new director, Lisa Garcia Bedolla, I look forward to furthering the development of these polling methods for the benefit of the university’s students, faculty and researchers, as well as the media and public at large.
—
Ed’s Note: Mark DiCamillo is currently director of the Berkeley IGS Poll at the Institute of Governmental Studies, U.C. Berkeley, after serving for more than 20 years as director of The Field Poll. He is also executive director of Cal Research Associates, LLC, an opinion research and polling consultancy.  He can be reached at [email protected] or at 415-602-5594.