¹ See Bernard, H. Russell, Research Methods in Cultural Anthropology (Sage Publications, Newbury Park, CA, 1988)Google Scholar and Miller, Delbert C., Handbook of Research Design and Social Measurement, 5th ed. (Sage Publications, Newbury Park, CA 1991)Google Scholar for detail on qualitative and quantitative data collection methods and instruments.

² MRS members unfamiliar with qualitative data collection techniques would find it useful to review the segment of the tape of each of the six meetings which dealt with the same topic (the merit review criteria might be the most enlightening in this case) to gain an understanding of qualitative data collection techniques. The participants were presented with the same information in each case; the reviewer could observe differences and similarities in responses among the groups. That also might give the reviewer a better context within which to understand the analysis presented in this report.

³ From a decision theory standpoint, of course, this is the same process. The criteria simply make explicit what is implicitly considered by the reviewers as they ask “can it be accomplished” and “do I care.” It was interesting, however, that this group saw those as two, distinctly different processes.

⁴ It is interesting that not one participant recognized that the current process favors those with excellent writing skills over those with excellent scientific abilities. It is true, however, that our education system emphasizes written communication over oral.

⁵ For a recent quantitative approach to an analysis of the return two of these portfolio management models provide (peer review and formula funding), see Cowling, Ellis B., Sigmon, John T., and Purman, Charles E., “Maximizing Benefits from Research: Lessons from Medicine and Agriculture” in Issues in Science and Technology (Spring 1996) p. 29Google Scholar. Robert Mullan Cook-Deegan compares “expert choice” models with peer review in a more qualitative approach in “Does NIH Need a DARPA?” in Issues in Science and Technology (Winter 1996–1997) p. 25Google Scholar.

⁶ The assumed model of the evolution of critical scientific thought that underlay the discussion surrounding the value of peer review in this context was particularly interesting. That model closely resembled that of a laissez faire capitalist system: that from the apparent randomness of individual activity, system order would emerge and all needs would be served.