Every trade has its tricks, solutions to its own specific problems, and distinctive ways of doing something with which laypeople have a lot of trouble. The trade of data analysis (defined here as conceptualization, implementation, and presentation of data analyses), no less than cooking or painting, has tricks developed and preserved by generations of professional researchers to address peculiar issues, questions, and concerns. These tricks are of different natures. Some of them are simple rules of thumb extrapolated from experience (e.g., in presenting your results, do not forget to specify degrees of freedom for your analyses). Others are the result of scientific analysis (e.g., Becker, 1998). Some of these tricks are learned from formal sources (e.g., Abelson, 1995). Others are acquired through hanging around “experts” in the field and learning to use them, the way apprentices learn craft skills by watching the master.

Knowing tricks of the trade constitutes an informed way of carrying out data analysis. Correspondingly, a lack of this knowledge leads, on many occasions, to poor data analyses. Some researchers (Sutton & Maynard, 1993) have stated that poor data analysis amounts to faking (defined in the Oxford Dictionary as “contriving with poor material”).

CLARIFICATIONS

The data world is huge. Moreover, the sheer quantity of data and summaries of data derived directly from or about individuals, groups, and cultures has increased rapidly in recent years and will increase more. These data stem from ability and achievement testing, measures of attitudes, surveys of satisfaction, socioeconomic indexes, and many other sources.