Minnesota’s Governor’s Race Tightening According to Star Tribune/MPR Poll? A Lesson in How Not to Read Polls

The Star Tribune/MPR declare in a new October 21, 2018 poll that the race for governor has tightened in Minnesota, with Walz holding a narrow lead over Johnson  Is that the reality?  The simple answer is that we do not know based on the polling data the paper provides, but the poll also provides a lesson in how not to read and interpret polls.

I teach polling and survey research.  I learned how to        do this both from professors at Rutgers University who have gone on to run the Pew Research Center and their polls, and from Charlie Backstrom at the University of Minnesota who wrote one of the best books ever on polling.  I say this because while I may not be able to do a good poll myself, I do understand what constitutes a good versus bad poll, or at least how to interpret their results.

In an October 21, 2018 poll of 800 likely voters the Star Tribune/MPR poll shows Walz with a 45%-39% lead over Johnson, with 12% undecided.  The poll has a respectable 95% confidential level and a margin of error of plus/minus 3.5%.  The results in the poll compare to a similar one done by the Star Tribune/MPR on September 16, 2018, also among 800 likely voters and a margin of error of plus/minus 3.5% showing that Walz had a 45%-36% lead with 16% undecided.  The conclusion of the paper was that Walz had a narrow lead that was tightening between the two polls.  Is this a correct conclusion?

There are many reasons to correct how accurate such a conclusion is.  First, consider the margins of error in both polls +/- 3.5%.  In the September 16, poll Walz could have been as high as 48.5% or as low as 41.5%, and Johnson could have ranged from 42.5% to 35.5%.  Compare this to the October 21, poll where the range for Walz could be 48.5% or as low as 41.5%, while for Johnson it could be 42.5% to 35.5%.  What we get are so polls so close in terms of their results that given the margins or errors, the differences in the poll results could be simple polling or sampling errors.  It is difficult on the basis of these two polls alone to conclude very much in terms on anything.

Perhaps the only thing that intuitively makes sense is that fewer voters are undecided in October than  September.  But the difference of four percentage points is so close to the margin of error that it is too possible to conclude that any shift in the number of undecideds is statistically almost insignificant.

There are three other issues with the two polls that raise questions about how much one can infer from them.  First, in both polls 40% of those polled came from cellphones and 60% from landlines.  Nationally and in Minnesota we have reached a point where more than 50% of the population is without a landline according to the National Center for Health Statistics and industry surveys.  The best surveys research now seeks to have approximately 60% cellphone numbers.  The Star Tribune/MPR poll has an almost exact reverse of what is recommended.

Who still uses landlines?  Generally, the older you are the more likely to only have a landline while the younger the more likely to have only wireless.  This is significant because age is a variable in terms of voting patterns, with presently older people more likely to vote Republican than Democrat.  Thus, even though  both the September and October polls may have an approximately correct balance of Republicans, Democrats, and independents, they might have over sampled those who are more likely to vote Republican, especially among those who call themselves independents.  The reason for this is that many independents really are not independent–their voting patterns actually favor one party over another. Thus, both polls might have been biased in favor of Republicans.

Second, both polls perhaps over-sampled the metro area with 61% of respondents coming from the metro area, compared to a more historical norm of 53-55%.  While demographics in the state are changing, this metro bias perhaps meant that  the polls favored voters more likely to lean Democratic.

Third, it is unclear from the polling methodology who is considered a likely voter.  We know that historically between 10-15% of those who vote in Minnesota register at the time of voting.  Mason-Dixon, which does the polling here for the Star Tribune/MPR, in the past has not produced a good methodology that accounts for this phenomena.  Failure to do this again raises questions about the poll’s accuracy.  Finally, the poll fails to account for the fact that voters in greater Minnesota vote in greater percentages than those in the urban areas.

So what is the point I am making?  Comparing these two polls it is hard to infer as much in terms of trends as suggested by the data and we really cannot say that the Minnesota gubernatorial race is  close or tightening.