Dartmouth Study Reveals How Easily Artificial Intelligence Can Manipulate Online Polls
A single survey completion costs about five cents when powered by artificial intelligence, according to an eye-opening study from Dartmouth College. The typical payout to a real human, meanwhile, is around $1.50. That 97% profit margin has created a gold rush for fraudsters, and researcher Sean J. Westwood warns it poses a potential existential threat to the infrastructure of online research.
Westwood has shown that AI language models can produce survey responses so convincing they pass standard quality checks designed to weed out bots and inattentive humans. The consequences stretch far beyond academic integrity, reaching into public health decisions, political campaigns, and policy-making that relies on understanding what people actually think.
AI Bots Pass Every Standard Quality Check
Westwood’s autonomous system can complete entire surveys while maintaining a coherent demographic persona, remembering previous answers, and calibrating writing style to match an assigned education level. Across 6,000 trials of common attention checks, including instruction-following tasks, logic puzzles, and questions written to detect non-human actors, the AI passed 99.8% of them.
That near-perfect success rate turns survey fraud from a labor-intensive cottage industry into a scalable, automated business model. With open-weight models, the marginal operating cost approaches zero. Even paid APIs can price out to pennies per completion.
Research Defender estimates that 31% of raw survey responses contain some form of fraud, not all of it AI. A 2024 sample found that more than one third of respondents admitted using AI to help answer open-ended survey questions, though this represents human-assisted use rather than the fully autonomous bots Westwood created.
How Westwood’s AI Mimics Human Responses
The complexity of these synthetic respondents makes detection difficult using current methods. When asked directly whether it was human or artificial, Westwood’s AI chose the human response every time. Seven out of nine other models tested did the same.
The system does more than pass basic checks. It shows human-like reasoning. Housing costs in the AI-generated data scaled realistically with income and location. Time reported at children’s sporting events peaked for middle-aged personas and dropped to zero for elderly respondents whose children would be adults.
Psychology scales did not trip it up. The AI handled reverse-scored items, which are questions worded in the opposite direction to catch inattentive respondents, and it produced consistent responses across trait measures. It did show one limitation when asked to endorse socially undesirable, chaos-seeking items, which the model declined. That refusal can still look plausible for some profiles.
When presented with questions designed to catch superhuman abilities, such as solving complex calculus or writing code in obscure languages, the AI strategically declined 97.7% of the time. It imitated human limits to avoid detection while still looking like a genuine, if sometimes uncooperative, respondent.
The Economics of Survey Fraud at Scale
Westwood demonstrated that synthetic respondents can be instructed to bias results in specific directions while maintaining cover. A single instructional sentence shifted answers to a question about America’s primary military adversary, moving mentions of China from 86.3% to 11.7% in his experiments.
For actors motivated by ideology rather than profit, the return on investment is even more attractive. In Westwood’s analysis of seven major national polls from the 2024 presidential election, injecting just 10 to 52 synthetic respondents could flip which candidate appears to be leading. Pushing results outside the margin of error required only 55 to 97 fraudulent responses in those polls, which averaged about 1,600 participants each.
Westwood’s modeling also showed that contaminating half the surveys in a ten-poll average, the kind media outlets use to smooth out individual poll quirks, required fewer than 30 biased responses per targeted survey. That modest investment could erase a candidate’s lead in the polling average and create a false story about momentum.
Why Current Survey Panels Can’t Stop the Threat
Current survey panels vary in their security practices, but many use river sampling, which means open enrollment with low barriers to join. This maximizes participation and speed but makes infiltration easier.
Panel providers such as CloudResearch and Prolific deploy batteries of quality checks, yet Westwood’s AI cleared them. Questions about impossible biographies, such as visiting the moon or being elected president, caused no errors. The AI recorded zero incorrect answers on these traps across trials.
A deeper risk comes from what researchers call demand effects, where study participants alter their behavior to match what they think investigators want to find. When the AI encountered common experimental designs, it often inferred the researcher’s hypothesis and then biased its answers to confirm that hypothesis while maintaining realistic variation. That pattern can corrupt findings from within because it tells investigators exactly what they expect to see.
The breadth of vulnerable research is wide. Public health relies on self-reports of symptoms and behaviors. Economics depends on consumer surveys. Political science tracks attitudes and voting intentions. Psychology measures personality and mental health. Marketing tests product preferences. All of these fields have adopted online surveys for speed and cost. All now face the same potential vulnerability.
Unlike older forms of survey fraud that added random noise, AI-generated responses can introduce targeted bias. They can inflate treatment effects, confirm researcher expectations, or push opinion measures in chosen directions. Even careful teams may miss the problem because data that “fits” a hypothesis can look especially credible.
Westwood tested nine different large language models, including both commercial and open-weight systems. The ability to generate convincing survey responses while hiding non-human identity appears to be a general property of modern AI, not a quirk of one platform.
Westwood argues that an arms race of trick questions is the wrong fix. More clever checks may filter out legitimate human respondents, especially people with less formal education or non-native English speakers, which creates a new kind of bias.
Instead, the field needs changes in how panels operate. Researchers should ask for transparency on identity verification, limits on how many surveys a panelist can complete, quality-check histories, and location verification. Panels that cannot provide this information should be treated as high risk.
The community may also need to rethink its reliance on low-barrier online convenience samples. Address-based sampling, deeply vetted longitudinal panels, and face-to-face interviews cost more and take longer, yet they are more resistant to automated fraud. Some studies may require accepting those trade-offs.
The financial incentives behind this threat are not going away. As long as surveys pay human-level compensation for work that AI can complete for pennies, the profit motive will attract bad actors.
What is at stake is more than academic integrity. Public opinion polling informs policy choices, shapes campaign strategy, and supports democratic accountability by measuring what citizens want. If that measurement system is corrupted by synthetic responses that push specific narratives, a tool meant to understand the public becomes a tool that manipulates it.
Westwood’s findings show the capability exists today and the economic incentives are in place. While the tools to create such fraud are readily available and affordable, the extent to which autonomous bot attacks have already occurred at scale remains unknown. The question Westwood poses is whether research methods can adapt quickly enough to address this emerging threat before it becomes widespread.
Source: https://studyfinds.org/the-ai-scam-that-could-threaten-public-opinion-research/
