Published online by Cambridge University Press: 06 March 2018
Science is a complex task. It involves the creation and dissemination of knowledge. The creation of knowledge requires identifying and abstracting patterns (i.e., identifying phenomena and theorizing about the processes that bring it about), as well as systematically observing to better see and quantify the patterns (e.g., effect size estimating) or assess the validity of the abstractions used to explain the patterns (i.e., theory testing). To help (a) hone in on what observations would be useful and (b) communicate what the patterns mean, we are encouraged to develop and report hypotheses. That is, strategically, hypotheses facilitate the planning of data collection by helping the researcher understand what patterns need to be observed to assess the merit of an explanation. Meanwhile, tactically, hypotheses help focus the audience on the crucial patterns needed to answer a question or test a theory. When the strategic hypotheses are not supported, it raises a question regarding what to do tactically. Depending on the result (i.e., different direction; null), one might construct a hypothesis to facilitate dissemination without reporting this post hoc construction or remove mention of a hypothesis altogether. This practice is called HARKing (i.e., hypothesizing after results are known). HARKing has been so disparaged as to be considered a “detrimental research practice” (Grand et al., 2018, p. 6). As such, the Society for Industrial and Organizational Psychology's (SIOP) Robust and Reliability Science task force appears to be recommending that HARKing not be taught by educators, encouraged by reviewers or editors, or practiced by authors. I do not agree with those recommendations, and I elaborate on my position below.
Target article
A Systems-Based Approach to Fostering Robust Science in Industrial-Organizational Psychology
Related commentaries (7)
Enough Talk, It's Time to Transform: A Call for Editorial Leadership for a Robust Science
If Robust Science Is Relevant Science, Then Make I-O Psychology Research More Relevant: Thoughts From a Practitioner Point of View
In Defense of HARKing
Open Science Is Robust Science
Robust Science: A Review of Journal Practices in Industrial-Organizational Psychology
The Last Line of Defense: Corrigenda and Retractions
The Role of Professional Associations in Promoting Robust Science