The critical understand that, while some social statistics may be
pretty good, they are never perfect. Every statistic is a way of
summarizing complex information into relatively simple numbers.
Inevitably, some information, some of the complexity, is lost whenever
we use statistics. The critical recognize that this is an inevitable
limitation of statistics. Moreover, they realize that every statistic
is the product of choices -- the choice between defining a category
broadly or narrowly, the choice of one measurement over another, the
choice of a sample. People choose definitions, measurements, and
samples for all sorts of reasons: Perhaps they want to emphasize some
aspect of a problem; perhaps it is easier or cheaper to gather data in
a particular way -- many considerations can come into play. Every
statistic is a compromise among choices. This means that every
definition -- and every measurement and every sample -- probably has
limitations and can be criticized.
-- Joel Best, "Telling the Truth About Damned Lies and Statistics"
The Chronicle of Higher Education (May 4, 2001)
http://chronicle.com/weekly/v47/i34/34b00701.htm
from Damned Lies and Statistics: Untangling Numbers From the Media,
Politicians, and Activists, published by the University of California
Press, 2001.