The sampling flaw
A bad sample proves nothing about the whole group.
Fallacy vs. flaw, on this site. Our fallacy pages on hasty generalization and anecdotal evidence teach the general error. This page is the LSAT version: Flaw questions love surveys, polls, and studies, and the right answer usually says the sample is "unrepresentative" or "unlikely to be representative of" the group in the conclusion.
Start with the general version: the fallacy page. Then come back here for the test-day version.
What it is
The sampling flaw happens when an argument draws a conclusion about a whole group from a sample that is too small or not representative. The classic version: a survey of an unusual subgroup, treated as if it speaks for everyone.
Size matters less than people think. A thousand-person survey can still be fatally flawed if all thousand share the trait being measured. Representativeness is the real test.
An LSAT-style example
This example is original. It is not a real LSAT question.
A poll of visitors to a vegan recipe website found that 85 percent never eat meat. So most Americans have given up meat.
The flaw: Visitors to a vegan recipe site are nothing like Americans in general. The sample is self-selected and wildly unrepresentative, so the conclusion about "most Americans" is unsupported.
How to spot it on test day
- Find the sample in the evidence and the group in the conclusion. Are they the same group? If not, flag it.
- Watch for self-selected samples: website visitors, magazine readers, people who wrote in. They chose themselves.
- Watch for tiny samples treated as decisive: "three customers complained, so the product is failing."
The right answer vs. the trap
The right answer
Usually "draws a general conclusion from an unrepresentative sample" or "the sample is too small to support the conclusion."
The trap
"The argument confuses correlation with causation." Sampling questions sometimes include a correlation in the evidence as decoration. If the conclusion is about a whole group, it is sampling, not causation.
Find your weak flaws
The free diagnostic scores flaw questions separately and shows whether this pattern is costing you points.