Taking a falsificationist perspective, the present paper identifies two major shortcomings of existing approaches to comparative model evaluations in general and strategy classifications in particular. These are (1) failure to consider systematic error and (2) neglect of global model fit. Using adherence measures to evaluate competing models implicitly makes the unrealistic assumption that the error associated with the model predictions is entirely random. By means of simple schematic examples, we show that failure to discriminate between systematic and random error seriously undermines this approach to model evaluation. Second, approaches that treat random versus systematic error appropriately usually rely on relative model fit to infer which model or strategy most likely generated the data. However, the model comparatively yielding the best fit may still be invalid. We demonstrate that taking for granted the vital requirement that a model by itself should adequately describe the data can easily lead to flawed conclusions. Thus, prior to considering the relative discrepancy of competing models, it is necessary to assess their absolute fit and thus, again, attempt falsification. Finally, the scientific value of model fit is discussed from a broader perspective.

This paper describes how statistical methods can be tested on computer-generated data from known models. We explore bias and percentile tests in detail, illustrating these with examples based on insurance claims and financial time series.

Selected risk programming solutions (i.e., profit maximization, Target-MOTAD, and MOTAD) are tested in an economic environment outside the data set from which they were developed. Specifically, solutions are derived from either a longer 10-year (1965-74) or shorter 6-year estimation period (1969-74), and then, they are tested for consistent risk-income characteristics over a later 10-year period (1975-84). Risk solutions estimated from earlier periods perform well in the later test period in spite of different economic conditions between time periods. However, favorable performance may be related to the specific example used in this analysis. Further testing for other farm situations is needed before general conclusions can be reached.

The field of disease ecology – the study of the spread and impact of parasites and pathogens within their host populations and communities – has a long history of using mathematical models. Dating back over 100 years, researchers have used mathematics to describe the spread of disease-causing agents, understand the relationship between host density and transmission and plan control strategies. The use of mathematical modelling in disease ecology exploded in the late 1970s and early 1980s through the work of Anderson and May (Anderson and May, 1978, 1981, 1992; May and Anderson, 1978), who developed the fundamental frameworks for studying microparasite (e.g. viruses, bacteria and protozoa) and macroparasite (e.g. helminth) dynamics, emphasizing the importance of understanding features such as the parasite's basic reproduction number (R0) and critical community size that form the basis of disease ecology research to this day. Since the initial models of disease population dynamics, which primarily focused on human diseases, theoretical disease research has expanded hugely to encompass livestock and wildlife disease systems, and also to explore evolutionary questions such as the evolution of parasite virulence or drug resistance. More recently there have been efforts to broaden the field still further, to move beyond the standard ‘one-host-one-parasite’ paradigm of the original models, to incorporate many aspects of complexity of natural systems, including multiple potential host species and interactions among multiple parasite species.