The focus of statistical tests on significance can lead researchers to desperately seek significance, particularly when an experiment has 'failed'. However, this chapter tries to make clear, using the framework of severe testing, the problem of seeking significance at any cost and the resulting weakening of results based on over-testing or fishing for significance. The chapter proposes some rules to guide the researcher to both explore data thoroughly but not go too far in pursuit of significance.