Showing posts with label Pollsters. Show all posts
Showing posts with label Pollsters. Show all posts

Sunday, December 4, 2016

Lessons Learned or Not from the 2016 Election

Perhaps the only thing more dreary and misinformed than the commentaries and emoting leading up
to the 2016 election are the commentaries and emoting that have taken place since.  It has been approximately one month since  November 8, I have given more than a dozen talks to various groups about the election and have listened to more than my fair share of partisan analysis and general emoting about what happened.  Nate Silver once did a book discussing the need to separate the signal from the noise if we want to make good predictions. What are some of the lessons learned or not since then?  In effect, what was and is still now the signal and noise regarding the election?

The Pollsters did not blow it.  Everyone wants to argue that the pollsters blew the election.  No, they did not.  The last national aggregate polls said Clinton had about a 2-3 point lead in the national popular vote.  Guess what?  The near final vote totals are showing her with a 2 or so point lead in the national popular vote.  But remember two things about polls.  First, polls are never pin-point accurate and will be expressed in terms of margins of error of +/- three or so points.  Clinton was within that margin or error.  Second, and more importantly, as I have repeatedly told people, national aggregate polls are worthless when it comes to presidential elections because we do not elect people by the popular vote–it is the electoral college that matters.  What was more critical to examine were the tracking polls in critical swing states and there we saw clear evidence in the last 72 hours or so that the undecided voters were tracking toward Trump.

But the pundits did blow it.  The NY Times at one point gave Clinton nearly a 92% chance of victory and Nate Silver had it up to the high 70s or 80s.  Almost every major media pundit and outlet  was convinced Clinton would win and simply could not see how she could lose to Trump.  ( For the record, I generally placed Clinton’s chances most of the election at about 55% but consistently saw many reasons why Trump would win and apparently I have been told by many people who attended many of my talks in the last year said that I predicted Trump would win).  There are many reasons why they blew it, much having to do with the intellectual or cultural bubbles they lived in, laziness,  or again accepting the same received wisdom of 2008 that said that Clinton was inevitable.

This election was typical in many ways.  In many ways this was a normal election that came down to some fundamentals and basics.  It was an election that was decided by a handful of swing voters n a few swing counties in a few swing states. These were the same states I wrote about in my book, plus Pennsylvania and Michigan.  No surprises here.  In addition, elections come down to candidates  and their narratives–Clinton lacked a clear and consistent reason to why she should be president, Trump had a narrative.  Despite all the hand wringing and speculation of a divided Republican Party, Trump did a better job mobilizing the GOP base and holding it than did Clinton.  Trump moved more of the swings in the critical swing states.  And true to form as with the most recent elections, it was a polarized election that was very close.  Finally, Tip O’Neill once said never assume people will vote for you–everyone wants to be asked.  Clinton lost in Michigan and Wisconsin because she did not campaign there ask for votes, Trump did.  The same almost happened in Minnesota.

The election was atypical in many ways.  There were many ways this was an unusual election.  First both candidates were tremendously unpopular and that created volatility among some in the electorate undecided regarding whether to vote and for whom. Perversely, both candidates did better in the polls when they did not talk or were not covered by the media.  Second, this was the first presidential campaign in American history with a major party candidate featuring a woman.  Before  the election I estimated that approximately 30% of the electoral would never vote for a woman regardless of who she was.  My point was that sexism was a central problem in this election that most pundits and people largely ignored.  It played out in terms the preoccupation with Clinton’s pantsuits, speech patterns, demeanor, and Trump’s language, both verbal and body during the debates and campaign season.    Other factors that made the election unique were how the Democrats seem to have largely lost working class voters, the impact of the FBI director Comey letter, and the  near complete void in terms of issues as a factor in the race.  We should also not forget about the transformation of the made for television election and presidency into the made for social media election and presidency.  Finally, we should not ignore the important role fake news played and how bots and trolls potentially drove much of the election in the social media.

Politainment lives!  I have written extensively about the rise of politainment, or the convergence of the politics and entertainment (and pop culture) where candidates who master the art of generating clicks and views in a 24/7 for profit media cycle will do well.  Trump understood this, Clinton did not.  The mainstream media was so heavily dependent on Trump for dollars and viewers that it could never figure out how to cover him.  The same is true post-election.    Trump essentially owns the news establishment, preventing the media from being able to cover him in any way that is objective or reasonable.

But even beyond the aforementioned themes, there are several post-election reactions or statements that need to be addressed.

Trump will be the president.  Barring the unforeseen Trump will be president.  A lot of people are saying that “Trump is not my president.”  Like it or not he is.  Some see him as a sexist pig or worse  and cannot believe Clinton lost to him.  The reality is she did lose the electoral college to him.  Moreover, yes the sexism in our culture stinks and one can complain about it forever but that sexism is a reality and female candidates unfortunately have to learn how to campaign facing it.

Don’t blame the voters.  I have seen way too many articles or people blame Millennials, or third party voters, or women, or others for why Clinton lost.  Stop blaming the voters!  The fault is with Clinton.  Ralph Nader once said that no one owns a voter and as Tip O’Neill said, you have to ask and earn votes.  Clinton was not entitled to anyone’s vote and she had to earn it.  If people did not vote for her the fault rests with her so blame her and her campaign instead.

Demographics is not Destiny.  This was the Democratic Party mantra and Clinton strategy leading into this campaign.    Demographics matter but so does candidate quality and message.  Moreover, in many ways demographics did matter in this election–white working class are still a majority of the electorate and they still matter and they show up to vote.  Oh, and the electoral college still matters.

The election was not rigged.  There is little evidence to support this a currently understood.  We do not have much information regarding voter suppression yet.  There is little evidence of Russian hacking or the general hackability of US elections.  Moreover, there is no evidence that millions of illegal votes, especially in patterns that would have only voted for Clinton. No party has a monopoly on virtue and there is also no reason to think that the paltry cheating that occurred took place to the benefit of one party or candidate alone.   If we are talking about rigging in terms of the role of money in politics, election rules that hurt third party candidates, and the media bias, then yes it may have been rigged.

This may or may not have been a critical election.    There is some evidence that traditional voting patterns shifted in this election but it is not clear how much and how permanent.  The best explanatory variable predicting election returns this year seems to be educational level.  Areas on balance with people with more college degrees voted for Clinton and Democrats.  No this does not mean stupid people voted for Trump.  It instead suggests a new cultural divide that needs to be understood better in terms of how American elections move forward.

There are no tanks in the streets.  Calm down everyone.  Yes the campaign was awful in tone and  we saw an unleashing of a lot of bigotry.  Yet it started way before Trump and he may simply be the face of a new ugly era already emerging in politics.  Stories from the Southern Poverty Law Center report hundreds of hate incidents since the election but we have no idea how this fits into an existing trend line or in comparison to comparable recent periods.  There is constant hysteria about what will or will not happen with Trump as president.  Calm down.  For the same reason that the rigidity and stability of our political institutions (checks and balances, separation of powers, bicameralism, and federalism) make political change difficult, the same will also prevent any excesses over the next four years.  We should also simply ignore or shrugged off (as David Brooks recently said) the most recent tweets of Trump.

Overall, understand there is a lot of noise out there.  Ignore most of it.

Sunday, March 13, 2016

Signals v Noises: How and Why the Pundits, Pollsters, Politicians, and Polticial Scientists are Confused About the 2016 Elections

Trump and Sanders perplex the professional politicians, pundits, pollsters, and even political scientists.  When I say Trump and Sanders I do not just mean them but instead they are proxies for the entire US 2016 presidential process thus far and probably until election day November 8.  No less than Nate Silver, Politics 538, and the pollsters blew it with the Michigan Democratic Primary last week.  The forecast was for a double digit Clinton victory, prompting Nate Silver to give Clinton a 98% chance winning that state’s primary.

So the question is why?  The core reason is laziness–assuming and following conventional wisdom is correct and failing to see the proper signals suggesting that something was unique about the 2016 election cycle.

Take us back to May 2015.  Back then the conventional wisdom among political insiders–and that includes politicians, political operatives, and pundits (journalists and commentators)–was that  Clinton and Bush would march to their party nominations and that the final general election would be a contest between these two predicted candidates.  Furthermore, even though Jeb Bush was going to win, the GOP had other strong candidates in Christie, Walker, and Jindal for example such that they would mount a powerful lineup against the inevitable Hillary Clinton.  Sanders campaign was dismissed as Quixote at best, with polls pointing to 60-70% leads by Clinton over him.  Trump too was dismissed as at best a vanity candidate would repeatedly implode, especially after each one of his insulting statements that all were sure would doom him.  But now of course nine months later and well into the primary season Trump is in a terrific position to win the nomination and Clinton, while leading Sanders in committed delegates, is not guaranteed and there are still reasonable scenarios for the Vermont Senator to win.  Even moving forward, assuming a Trump-Clinton contest the received wisdom is
that Clinton wins.

From my perspective all of this is wrong.  Last May I wrote about the chances of Trump and Sanders potentially winning, and I think that in a Trump-Clinton race Trump may win.  So why did do many get it so wrong?

Laziness is the issue.  Better yet, the answer is “Inside the Parkway” or “institutional disease.”  Specifically, those making the predictions are all politicians or pundits who are part of the establishment.  They are located with the narrow confines of Washington, D.C., viewing the world from that perspective.  They share the same world.  Look at CNN, MSNBC, FOX.  All the journalists know one another, their guests are from there.  They all share the same biases and perspectives and fail to see how the world looks from outside the parkway, outside of the formal institutions of power-Washington government on big  corporate for-profit
media–and they do not write from the perspective of how people view the world from the fly over regions of America.  Watch these dreary  shows, read all the on-line publications, and not just th partisan ones, and they reinforce one’s another’s biases.  They have a tight little club with the regular suspects of commentators or analysts  and none of them really look at the world from a different perspective beyond what they see from  their desks and studios in Washington.  They drool out conventional wisdom about how they view or think the world should operate, failing to recognize that just because they so it does not make it so.

What they missed of course then is the depth of anti-institutionalism pervading American society.  They confuse what has politically worked in the past with what is happening now or what will work in the future.  They simply think that the past is a certain predictor of the future without asking if there are any changed conditions that might suggest a new reality this year.  This is the laziness I speak of; and it is the source of confusing signals and noise.

What are some signals that should have been seen?  First, few appreciate the generational shift occurring in American politics.  Baby Boomers just don’t get this.  They are near clueless that  the power shifted from Boomers to Gen Xers with Obama and now it is shifting to Millennials.  They seem clueless to the different objective conditions driving Millennial politics.  This is a structural shift in politics and Clinton and her supporters largely fail to understand this.  Clinton represents old style politics–the type that brought us the Iraq War, massive student loan debt, a grim economic future, and global warming. The Boomers wanted a revolution to change the world and they not only failed but handed Millennials a crappy future.  The politicians and pundit class are Boomers.

But what is also missed is something else.  In a bipartisan fashion the policies of both the Democrats and Republicans over the last two generations have screwed over most people.  Republicans have explicitly become the party of plutocrats, losing track of the strategy Kevin Phillips endorsed in 1969 The Emerging Republican Majority which said that Nixon and the Republicans could capture the silent middle class majority by developing policies to help them. Reagan walked away from this strategy and the GOP has done little to address the economic problems of their base.  Similarly, the Democrats, especially Bill Clinton, became corporate Democrats, and they too have done little for middle class America. This is even true of Obama who worried more about restructuring Wall Street with tepid laws than in helping homeowners. He never supported reform to labor or union laws, never pushed on the minimum wage. Trump and Sanders  emerge as challengers to this anger.

The Republican and Democratic leadership has simply assumed they are the party and do what they want and often do not think that what others think matters.  Yes we have primaries and caucuses, but the GOP establishment has their silent primaries to pick who they want and the Democrats have their super delegates as the fail safe against  the people.  In both cases the leaders of the party are saying that the real people do not matter.  Create an insulated structure like this and it is no wonder the parties failed to see what is happening.

But the other signal missed this year is not understanding on the one hand that presidential politics is mostly television-driven, assuming what I have said is a politainment status that favors candidates who look and speak well on television.  Thus Trump.  But politics also goes to those who can best master new communication forms, and again Trump and Sanders have an advantage.  But at the same time one of the noises confusing so many is that too many come to believe all that is posted on the Internet or that simple spin is enough or that if one blurts out enough rage that will be  enough to change minds or win votes.  In effect, too many people are inferring too much from the social media in terms of what it tells us about the election.

Another noise has been the polls.  Polls have become rarefied and objectified into the belief they are firm predictors of the future. Remember, polls are snap sots of public opinion in time that  reflect knowledge and awareness at a point T in time.  People’s opinions change and they gather information and pay attention.  Pollsters have assumptions about who will participate or vote they are often flawed and even the best polls may not sample properly (this is especially the case with younger cell phone users).  Finally, statistically even the best polls run at least a 5% chance of being wrong, and this does not count sampling errors.

Overall, the point is that laziness, inside the beltway disease, group think, and a host of biases and failures to see signals versus noises is what is making it so hard for so many to make sense out of the 2016 elections.