And one more thing

In my previous post, I linked to a Daily Kos poll that asked:

QUESTION: Do you believe that Barack Obama was born in the United States of America or not?

That wasn’t the only question on that poll; there was one more which apparently preceded it:

QUESTION: Do you believe that America and Africa were once part of the same continent?

I couldn’t for the life of me figure out the purpose of this question much less why it was paired with the Obama birthplace question. Maybe the pollsters figured people who answered “Yes” would consider Obama to have been born in the United States even if he was actually born in Kenya?

I put my questions aside while I wrote but now I’ve done a little digging around and found this Daily Kos post which explains that this was intentionally designed as a bad poll question and intended to prove how crummy Gallup is at doing polls:

So why would only 42% of Americans say that they believe in this extensively-documented, measurable, well-established theory? Should we worry that America is hopelessly backward and mired in some kind of anti-science dark age?

No. For one very good reason: both the question itself, and the presentation of the results are deeply flawed. Intentionally idiotic.

The questions was written to press emotional hot buttons. It's not "were all the continents once merged into Pangea?" or "Was the North American continent once closer to Europe?" or simply "do you believe in continental drift?" It's were "America and Africa were once part of the same continent." If you think that doesn't matter, I invite you to look at the regional breakdown of results -- that particular association of words "America" and "Africa," probably accounts for the 10% greater "No" vote in the same region where "Birtherism" is at its height.

There’s more - and some of it is quite interesting and even valid - but you see the point they’re making here. The problem is there are, well, problems with this.

First, it’s not clear America and Africa were ever the same “continent”. Wikipedia makes it clear the answer depends on what you mean by “continent” and you can see the debate rage here where people are arguing about tectonic plates versus joined land masses versus bordering land masses. So “No” is arguably the more correct answer than “Yes”. Perhaps the “birther” region has a more subtle understanding of Pangaea than those ignoramuses in non-birther regions.* Who knows what answer the pollsters would have gotten if they’d asked:

Were the United States and Africa ever part of a single large landmass?

Second, there’s no control question. No one was asked whether America and Europe were ever part of the same continent, or America and China. If fact, no one was even asked if North and South America were or are part of the same continent. (Again, it depends on what you mean by continent.)

Third, “America” is kind of a vague term. If I was thinking of North America, I might say “No” even though I accept the truth of the great super-continent and even if I considered a joined landmass the same as a continent whereas if I was thinking of South America I’d certainly say “Yes”. Why? Because when I think of Pangaea I think of that bulge of South America tucked nicely into that curve of Africa. I know North America was part of the same landmass but I don’t think of it as being spooned together into one unit with Africa.

Fourth, the Daily Kos article does not present so much as a simple crosstabs to compare how the answers to the continent question match up with the answers to the Obama birthplace question. They simply assume that because more people in the “birther” region said “No” to both questions the same people said “No” to both questions, presumably in all regions. They may be right but their numbers don’t prove that. Why not do the extra hour’s worth of work to see if their assumption is correct?

Fifth, there is no discussion of the effect the first question has on the second. The idea that polling questions impact later questions in the same poll is hardly radical. It would be helpful to at least see a discussion of why the continent question was considered to have no impact on the birthplace question.

Finally, the Daily Kos articles concludes its review of the methodological problems with:

The poll above does reveal that Republicans are much more likely to not believe in the results predicted by continental drift. Which is interesting. But even that result is of little value when it's attached to a question so deeply flawed.

So the question is too deeply flawed to reveal scientific ignorance among Republicans but not too deeply flawed to reveal racism among birthers. My statistics professors are spinning in their graves - and if they’re not dead, this will kill them.

There is one point in the Daily Kos article I can wholeheartedly agree with - at least if I change just one tiny word. When asking why Gallup does such a bad job asking questions and such a good job presenting sensationalized results, the article concludes:

Because it gets headlines. Because it keeps them in the news between elections.

Or maybe I'm giving them too much credit. Maybe the Gallup Daily Kos organization itself is too ignorant to see the numerous flaws in what they're asking and how they're presenting it. It's hard to be sure.

*****

* Daily Kos itself distinguishes between continent and landmass and makes it clear that Panagaea was the latter and not the former. Bizarrely enough, they do this when they explain why “Yes” is the correct answer (emphasis mine):

The collection of all the continents into a unified land mass around 250 million years ago ... In around 300 million years, the continents will hold a reunion, once again forming a single mass before the moving plates carry them away on separate paths.