How Market Researchers Lie: The Online Poker Edition

7th January 2014 // News, Op-ed

“There are two kinds of statistics, the kind you look up and the kind you make up.” — Rex Stout

If you’re trying to figure out the mindset of the American public when it comes to online poker, then the recent release of a couple of public opinion polls on the topic ought to be good news, shedding some real consumer light on the topic. Instead, the opposite has occurred, with recent research efforts showing a wide and seemingly inexplicable disparity of 30 or more percentage points on the topic.

What’s really happened, in both instances, are examples of market researchers using the art of constructing leading questions to skew results and please their masters.

From October of 2013, on the anti-gambling side, stands the skewed research effort generated by the Tarrance Group on behalf of Sheldon Adelson’s and Las Vegas Sands’ “Stop Internet Gambling” campaign. We debunked several of the claims made by Adelson and his supporters in this blog post… and maske no mistake about, it was shoddy, politically driven research.

According to the Tarrance Group’s numbers, some 62% of respondents were against internet poker, while an even higher percentage — 67% — were against internet gambling in general. By the time Adelson got done twisting the information to his own satisfaction he had the numbers above 70%. But it’s lies, all of it. (We’ll get to the why in a minute.)

Little did we know that the pro-gambling side would stoop to the same level in publishing their numbers, in which the exact opposite is claimed. That latest information came from a December 2013 Reason-Rupe study at Reason.com, a generally pro-gambling, libertarian mainstrean site. The Reason-Rupe study found quite an extreme opposite from the competing Tarrance Group poll, claiming a 65-32% split in favor of online gambling from all respondents, and skews as high as 75% among certain important demographic splits.

The Tarrance Group study included over 2,200 respondents, while the Reason-Rupe sampling was a little over 1,000. The Tarrance Group study claimed a margin of error of plus or minus 3.1%, while the Reason-Rupe study’s error margin was comparative at 3.7%.

The difference between the opinions expressed in the two polls, on the same general topic, is more than 30%. And that means something is really, really wrong here.

It would be easy enough for a pro-poker site like us, the friendly folks at FlushDraw, to be hypocritical and point the “Liar!” finger at only one camp, and indeed you’ll find a lot of that through the poker media in recent reports. But the truth is different: Both studies should bear the “Liar!” tag, because each has intentionally skewed the results in favor of the businesses they serve.

Here’s the truism of market research: The greatest single determinant on a “soft” issue is exactly how the question is phrased. Internet-based gambling and poker is an example of a soft issue, one unlikely be a single-topic voting determinant. The economy and abortion are examples of “hard” voter issues, by comparison.

The report published by Adelson’s group on the Stop Internet Gambing site intentionally obscures its methodology, lest its weakness be readily exposed. But what happens is this: Market researchers skew responses by constructing a leading methodology to steer generally uninformed respondents to the opinion the researchers desire. Here’s how Tarrance got the high negative response rate:

1) Establishing a baseline for responses to gambling and casinos in the states where Tarrance selected its respondents, all of which could be generally defined as pro-casino and neutral or slightly negative to gambling, but with some noted presence against online gambling in poker;

2) Setting up a mental construct within a question that depicts internet-based gambling as more evil or dangerous in some way, and therefore more of a threat. Here’s how the Tarrance Group did it:

In a forced choice question on whether or not internet gambling is a different form of gambling, voters are read these two statements:
Internet gambling is no different than the other types of gambling that already exist, and that
it is simply a natural extension of gambling options in this technological age,
OR
Internet gambling is very different from other types of gambling that already exist and that there are a number of key problems and potential abuses with online gambling that do not exist with traditional casino gambling.

This “forced question” is leading to the point of being abusive, with its introduction of such concepts such as “potential abuses with online gambling.” “Abuse” is a negative prompt word, and it is designed to elicit an “anti” reaction. Nowhere prior to this question was the word “abuse” (in any form) brought up in a question, and even here. no specifications were given; it was left up entirely to the generally uninformed respondents to imagine perilous online evils.

That’s an example of how researchers lie. I’d say shame on the Tarrance Group for such blatant opinion manipulation, but let’s be honest — that’s all a business such as theirs does.

And… as if the pro-gambling side’s effort was one whit better or more honest. The Reason-Rupe report intentional misconstrued in exactly the same way, by building an imaginary bogeyman and allowing respondents to react in the opposite. Instead, of “abuses,” Reason-Rupe chose “government,” another word guaranteed to raise the hackles and fears of most respondents.

Asked Reason-Rupe:

“Do you think the government should prohibit people from gambling in online poker games, or should the government allow people to do this?”

Again, the skew is created by the leading question. As soon as a respondent hears the words “government” and “prohibit,” he is likely to respond in the opposite direction. The rest of this question is rather more neutral than the blatantly steering Tarrance Group garbage but simply by reversing the Reason-Rupe question, and asking first if the government should allow rather than deny, the pro-online-gambling percentage would have checked in ten points lower.

That’s how easy it is to lie with market research. Take it — take all of it — with a grain of salt, until you learn how to carve the true opinions from the bluster and the skew.

The best guess is that the general voting populace is split into roughly equal thirds on the topic of online gambling (including online poker). One third is dead-set against it on immoral grounds, just as another third is dead-set for it, largely as an example of libertarian rights and freedom of choice. In the middle is a third of the population that really doesn’t care, and that can unfortunately be steered any which way by a researcher being paid to make them spew nonsense in one direction or the other.