What Happens When Pollsters Are No Better Than Psychics?

I'm going to get a little political here, but bear with me, this is a tech post in the end.

I'm in the midst of a trip from my new home in Berkeley, through my old stomping grounds in New York City, to my hometown in Canada. Politically, almost everyone in all three places falls in one of two camps: either they view the US Republican Party as evil incarnate, or (like me) they're fiscal conservatives who might once have been vaguely sympathetic to their avowed goals, but now see Republicans as the party of delusional magical-thinking cargo cultists.

Whether we're right or not isn't the point. (Although we are.) The point is that almost no one in any of my filter bubbles can even imagine a person who is not evil, insane, and/or an idiot voting for Romney/Ryan. And yet–according to the polls–they're practically neck and neck with Obama/Biden. If the Republicans win November's election, the people I know will be disappointed…but we won't be in a state of shocked disbelief. Because of the polls.

Trouble is, the polls ain't what they used to be.

The other day the great Nate Silver linked to a fairly amazing set of statistics:

Response rates have dropped by more than half in just the last six years, due to increased cell phone use, and, probably, the decreased likelihood of answering phone calls from strangers. (Remember, we here at TC declared the phone call dead almost two years ago.) But the Pew article does go on to say:

telephone surveys that include landlines and cell phones and are weighted to match the demographic composition of the population continue to provide accurate data on most political, social and economic measures. This comports with the consistent record of accuracy achieved by major polls when it comes to estimating election outcomes

Whew! All OK, then.

Except. Um. What "consistent record of accuracy" are they talking about, exactly?

The last election I voted in, the polls were so confused they were summed up as "someone leads in Ontario." Then, pollsters claimed that Alberta's Wildrose party had a "strong lead" and was set for a "sweeping majority"; instead, they got their clock cleaned, winning a mere 17 of 87 seats. Last year, when the Canadian federal election also confounded expectations, another FiveThirtyEight blogger compared it to the previous UK election, where "the polls turned out to heavily overstate the Liberal Democrats' vote." Heck, even Wisconsin's exit polls a few months ago were wildly off.

Are we beginning to see a pattern here? It seems that in major industrialized nations, polls are

1. bad
2. getting worse fast.

My unscientific (because untestable) hypothesis is that it is getting increasingly hard, verging on impossible, to muster any genuinely representative sample of our Internet-fragmented and increasingly internalized society. Here's my prediction for November: Nate Silver will find himself writing about how and why the election results varied so much from the poll numbers leading up to it.

Can polls be saved? Maybe. Could Twitter and/or Facebook mine their data to take their place? Maaaaaybe…but I don't know. Political talk is exactly the kind of thing that many people avoid on social media, lest they lose friends and alienate work people. Maybe identifying the proxy variables that replace polls will be the giant money-spinning machine for some Big Data startup. Or maybe those decades during which telephone and street surveys were actually a pretty good snapshot were just a weird aberration, and it's just going to get harder and harder to find out what life is really like outside of our own filter bubbles.

Image credit: txkimmers, Flickr.