Survey and self-selection flaws
Who answers the survey decides what the survey says.
Fallacy vs. flaw, on this site. Our fallacy page on anecdotal evidence touches the general idea. This page is the LSAT version: Flaw questions get specific about the mechanism, with right answers like "the survey is biased toward respondents with strong opinions." Name the mechanism, not just "bad sample."
Start with the general version: the fallacy page. Then come back here for the test-day version.
What it is
Beyond plain unrepresentative samples, surveys fail in specific ways the LSAT tests: self-selection (only people with strong feelings respond), leading questions (the wording pushes an answer), and non-response bias (the people who did not answer differ from those who did).
Any of these breaks the link between "the survey says" and "the group thinks." The argument treats survey results as group facts. They are respondent facts.
An LSAT-style example
This example is original. It is not a real LSAT question.
An online poll on a fitness forum asked whether members exercise daily. 92 percent said yes, so nearly all adults exercise daily.
The flaw: Three failures at once: the forum is not adults in general, only motivated members answered, and the question invites the "good" answer. The 92 percent describes the respondents, nobody else.
How to spot it on test day
- Who was asked, who answered, and how was it asked? All three can break a survey.
- Voluntary response is the reddest flag: the angriest and happiest people answer; everyone in the middle stays silent.
- Vague questions ("do you exercise regularly?") let respondents define the terms. Definitions decide results.
The right answer vs. the trap
The right answer
Usually "the poll suffers from self-selection bias" or "those surveyed are not representative of the group in the conclusion."
The trap
"The argument confuses correlation with causation." Survey flaws are about who was measured, not about causes. Keep them separate.
Find your weak flaws
The free diagnostic scores flaw questions separately and shows whether this pattern is costing you points.