Just found a great example in Richard Rothstein’s book “The way we were” about how we a lot of people misinterpret data:
” [A] Siemens manager noted that American youths (at plants in Florida, North Carolina, and Kentucky) scored higher on his company’s apprenticeship exams than comparable German young people who followed the same curriculum and took the same test….”
“These employment-test anecdotes are suspect, among other reasons, because they are not representative or properly controlled samples. It is not known, for example, from the Siemens manager’s speech whether Siemens’s pay scales in the United States are relatively high while in Germany they are relatively low: if this were the case, it might be that Siemens’s apprenticeship programs in America enticed the cream of our non-college-bound high-school graduates, while similar programs in Germany attracted the less able. ”
Lessons learned:
1. Get your hands with the data dirty before you come up with assumptions that don’t make sense.
2. Always use a sample size calculator
[3. American schools are not as bad as newspapers want to tell us]
