Testing the proposed criteria

It was considered that the best way to test the criteria was to complete a template based on the criteria established by de Vries et al. ⁽ Reference de Vries, Antoine and Burzykowski ^{5 )} (see Table 1) using examples of markers that reflect:

1. (1) A broad range of interests in nutrition research (see Fig. 1);

2. (2) The use of different tools, including both questionnaires and laboratory tests;

3. (3) Both long-established and newer markers and tools;

4. (4) Commonly used and not commonly used markers.

Table 1 Template to aid the evaluation of candidate markers for their usefulness in nutrition research according to previous step 2 of the Marker Initiative on Nutrition Research (de Vries et al. (2013)⁽ Reference de Vries, Antoine and Burzykowski ^{5 )})

Hence, the markers selected are not necessarily well validated or widely accepted. Markers covering the fields of nutrient intake, nutrient status, physiological function, metabolism, cognitive function and disease risk were all evaluated (Table 2).

Table 2 Markers used to assess the proposed criteria according to their specific field of application

FADS1, fatty acid desaturase 1.

Individual members of the multidisciplinary Expert Group completed the draft template (Table 1) using the criteria proposed in step 2 (de Vries et al. ⁽ Reference de Vries, Antoine and Burzykowski ^{5 )}) (henceforth ‘proposed criteria’, see Fig. 2) for each of the markers selected and then the completed template was discussed amongst all members and modifications made until consensus on the utility of the criteria for each marker was reached. One completed template is included (Table 3), while the completed templates for each of the thirteen markers are included in the online Supplementary material.

Table 3 Example of a completed template: use of response to vaccination as a marker of immune competence

Refining the criteria and developing a new evaluation template

As experts evaluated the proposed criteria and during the subsequent discussions, a number of pertinent points emerged regarding the ease of use, utility and relevance of the criteria and also the exact wording used to describe/define criteria. Although several of the core components that form part of the criteria are clearly defined (for example, sensitivity, specificity), others are not (for example, robustness), making it difficult to address these less well-defined criteria. Furthermore, in the absence of clear definitions, different individuals interpret the meaning of these terms or criteria differently, resulting in a less robust and less reproducible (from individual to individual) evaluation. It was recognised that, despite that fact that some markers are widely used, they fail to meet some of the criteria; for example in some cases assays may be poorly standardised. Furthermore, some measures are often used as endpoints themselves rather than as markers of other endpoints (outcomes); for example verbal memory, a marker of cognitive function, is often reported as an endpoint in its own right. Also, some markers have remained in use over a very long period of time, perhaps because they become validated or well accepted or, in some cases, because they are easy to use. On the other hand, some markers cease to be used after a period of time while new markers can emerge. Thus, there is a certain level of ‘turnover’ of markers. This has been hastened by the emergence of new technologies, typically ‘omics’-based, that have enabled the simultaneous measurement of clusters or patterns of markers. Some of these seem likely to replace existing single-measurement markers, although validation of the patterns, access to the technology and cost remain barriers. An example of how the marker field is developing is the proposal of a composite biomarker called the ‘vascular health index’ based on the integration of a meticulously selected cluster of biomarkers all related to vascular health measured with different types of analytical techniques⁽ Reference Weseler and Bast ^{6 )}.

Many markers are evaluated in a static setting, for example in fasting blood samples. This separates the sample and its component markers from the reality of human physiology, which is the need to respond appropriately to ‘daily stressors’ which may be metabolic, immune, physical (for example, exercise, temperature) or psychological. Thus, it may be desirable to include a challenge test in a study protocol and to evaluate the dynamic change in the marker in response to the challenge. Some of the examples of markers used to test the criteria do include a challenge test (for example, response to vaccination); again it seems that in the future more studies will incorporate challenge models to simulate events which could occur in the natural environment⁽ Reference van Ommen, van der Greef and Ordovas ^{7 ,} Reference Stroeve, van Wietmarschen and Kremer ^{8 )}. One area where challenge models have become widely used in recent years in order to study the dynamic change in a marker is the evaluation of inflammation. For example, both high-fat and high-carbohydrate meals induce an acute elevation in a number of biomarkers of inflammation⁽ Reference Esposito, Nappo and Giugliano ^{9 )} and such challenges have been used to assess the effect of including fibre⁽ Reference Esposito, Nappo and Giugliano ^{9 )} or vitamin C⁽ Reference Peluso, Villano and Roberts ^{10 )} in the meal on the acute inflammatory response that is elicited. Exposure of the skin to UV irradiation induces inflammation and controlled exposures have been used to assess the effects of including n-3 fatty acids⁽ Reference Pilkington, Massey and Bennett ^{11 )} or green tea catechins⁽ Reference Rhodes, Darby and Massey ^{12 )} in the diet on a range of biomarkers of inflammation. Intramuscular injection of bacterial endotoxin has been used to assess the effect of dietary n-3 fatty acids on inflammation⁽ Reference Michaeli, Berger and Revelly ^{13 )}.

It also emerged that a marker may not be equally useful across different applications. For example, a marker that is informative in the controlled setting of a small intervention study may be much less informative, or even unfeasible, in the setting of a large observational study.

The proposed criteria included ‘Must respond to a dietary intervention’. However, upon further discussion it was considered that whether a measure (i.e. a marker) is sensitive to nutrition does not make it a better or worse marker. This will depend on what the marker is designed to measure. Further, when considering whether a marker is influenced by a dietary or other intervention, then the extent of the effect seen (or foreseen) needs to be taken into account. This poses a challenge because studies are typically powered to show a statistically significant change in the marker being used. Even if that marker has an association with an endpoint, a statistically significant change in the marker may not be of clinical significance or even biological significance. Conversely, a change that is clinically significant may not be statistically significant in a study setting. Thus, it would seem prudent when planning a nutrition study to consider both clinical and statistical significance of the change being sought. Such considerations of study design, including effect size, are discussed elsewhere⁽ Reference Welch, Antoine and Berta ^{14 )}. It was concluded that the two criteria listed in the section ‘Reflect/mark an endpoint’ were essentially addressing the same point: that a relationship exists between the marker and an endpoint of interest. It was difficult for experts to adequately complete the section on ‘Analytical aspects’ because the different criteria asked about in this section were not well separated. Thus, by completing the draft template, based upon the proposed criteria, a number of the components of the proposed criteria were identified for change or improvement. It was also identified that providing information on normal values or ranges in different population subgroups and thresholds used to make different conclusions would be very valuable and was not explicitly requested in the draft template. It was also felt that sections in a new template to add other relevant information, for example to record inconsistencies in the literature, to make a clear conclusion about the usefulness of the marker under consideration, including any important limitations, and to record references used would all be valuable.

The above considerations led the Expert Group to conclude that the proposed criteria could be improved upon and therefore the draft template (Table 1) was revised to produce a new template reflecting these improvements (Table 4). This retains the general features of the proposed criteria (as described in the draft template), but the template is formatted in a way that is easy for the end-user to complete with a clearer indication of the nature of the information that is required for each section. The section ‘Methodological aspects’ (previously termed ‘Analytical aspects’) explicitly separates the most important components (validation; sensitivity; specificity; technical aspects other than sensitivity and specificity; biological variation), providing an opportunity to consider these individually. The section ‘Reflects the biological purpose of the marker’ (previously ‘Reflects/marks an endpoint’) combines the two previous criteria (‘Significant association between marker and endpoint in a target population’ and ‘Marker changes consistently with a change in the endpoint’) into a single reworded criterion (‘A change in the marker is linked with a change in the endpoint in one or more target population(s)’). This is because the two previous criteria address the same point, both stating that a relationship exists between the marker and an endpoint of interest. The section ‘Relevance to nutrition research’ (previously termed ‘Must respond to a dietary intervention’) expands upon and presents a change in focus from the proposed criteria. Now information on the normal range of values can be entered and the requirement that the marker must respond to a dietary intervention is replaced by a question seeking the evidence that nutrition can influence the marker and, if so, the extent of the reported effect. The reason for this change in focus is that whether a measure is sensitive to nutrition or not does not make it a better or worse marker, although it may make it more or less attractive to researchers and other stakeholders. Finally, in this section a question about other factors that might affect the marker is now included. A section ‘Other relevant information’ is included and there are cells for ‘Conclusion’ and to record ‘References’. It is considered that these changes will make the criteria and associated template more useful and more robust.

Table 4 Refined template to aid the evaluation of candidate markers for their usefulness in nutrition research

* Appropriate is used here to indicate that the required sensitivity and specificity of measurement may differ between study contexts, for example between a large epidemiological study and a much smaller randomised controlled trial.

It is anticipated that once a particular marker has been assessed using the criteria and the associated template, the information will be used as the basis for scoring the marker in order to determine its usefulness as a research tool. Such scoring requires a suitable methodological approach (see next paragraph).

Towards developing a marker scoring system

Examples of scoring systems may be found in Albers et al. ⁽ Reference Albers, Antoine and Bourdet-Sicard ^{15 ,} Reference Albers, Bourdet-Sicard and Braun ^{16 )} where markers of immune function were evaluated by scoring them against a range of predetermined criteria; in Albers et al. ⁽ Reference Albers, Bourdet-Sicard and Braun ^{16 )} these related to clinical relevance, biological sensitivity and feasibility. Table 5 is the generic marker scoring template now proposed. Researchers would score any marker under consideration according to the different criteria listed in the upper section of Table 4 as proven (+++), strong (++), medium (+) or low (0). Additionally, an arbitrary marker score would be based on subjective expert judgement on the usefulness of a marker based on carefully considered evaluation of individual criteria. This would enable researchers to evaluate and to compare different candidate markers within the same field of nutrition research in order to identify their relative usefulness. The criteria for a ranking of proven, strong, medium or low are likely to vary according to the research setting (for example, epidemiology, intervention, mechanistic investigation), the research field (for example, immune function, cognitive function, metabolism and metabolic dysfunction), and the type of tool used to assess the marker and therefore the ranking criteria need to be determined in a setting-, field- and tool-specific manner. Examples of such criteria and their use in evaluating immune markers may be found in Albers et al. ⁽ Reference Albers, Bourdet-Sicard and Braun ^{16 )}. It is expected that researchers would develop scoring definitions and then score and rank potential markers as part of study planning.

Table 5 Generic scoring system to evaluate and compare candidate markers within the same field of nutrition research