Does it seem to you that news headlines about nutrition research and your health repeatedly contradict what you thought you knew before? Sadly they often do. A quote from Harvard University Professor Frank Sacks addresses this very eloquently:

For the media, established knowledge is boring. The media wants something new. But that’s not necessarily good for your health. You’re better off with established scientific knowledge, and if it’s boring, that’s fine. You won’t be healthier because the media sells more newspapers.

In other words, science progresses slowly, as either previous knowledge is confirmed by new studies or it is shot down and gradually replaced by a new body of knowledge. This happens not as a result of a single study but a body of work that depends on consensus, which takes time. When a conflicting finding is reported, the process involves further analysis and confirmation by other scientists. If all these confusing headlines are based on “research,” how are you supposed to know what to believe and how to act on information?

This reference page provides a quick guide to some of the more commonly used methods in epidemiological (“wholistic”) nutrition research and how those methods impact the quality of what we can learn from them. The goal is to provide you with a better idea of how to question new findings yourself to ask whether the news is worthy of any behavioral change. It’s often not! Note that this summary does not include biochemical, animal, or in vitro (cell culture) studies or study designs. You can find an interesting read on the lack of application of most animal research to human medicine here.

Nutrition Research from the Bottom Up

In a context as admittedly complicated as this, sometimes a picture is, indeed, worth a thousand words. To maximize understanding and minimize my tendency to carry on, I present you with the following image. While I hope much of it is self-evident, I’ll provide a bit of guidance to get you started.

The Evidence Quality Pyramid

 

A pyramid sliced horizontally, with each slice representing different types of evidence provided by epidemiological studies of nutrition. The quality of evidence improves from bottom to top.

This graphic presents a range of epidemiological nutrition research evidence quality from least (bottom) to greatest (top). Use the table of contents (on right) to find more detailed information.

The Bold Arrow

Aside from the eye-grabbing colors, the big black arrow should “point out” the gist of this infographic’s meaning. It places nutrition research evidence quality  on a continuum of levels ranging from least impact (bottom) to greatest (top). Although simplistic, this should suffice to orient you to the rest of the image.

Three Brackets

Just to the right of the arrow are three brackets labeled:

Each of these is a generally describes types of nutrition research study designs you may see mentioned in news articles. Again, the quality of evidence provided by each design types is indicated by position in the graphic.

Colorful Steps

Each colorful step of this pyramid is labeled with a different type of nutrition research study design. With the exception of the bottom step, every other step represents a type of research design. Evidence from any of these levels might find its way into news headlines. It is key that you understand that not every type of study is equally powerful (refer to the bold arrow beside the pyramid).

A Primer for Nutrition Research Study Designs

Note: Not represented on the pyramid or discussed on this page are in vitro (i.e., cell culture) and animal studies that are better described as biochemical (rather than epidemiological) nutrition research designs. This is not meant to impugn their value as sound evidence.

Background information, expert opinion

This bottom level of the pyramid is not included in any of the three brackets described to the left of the pyramid. This is because this type of “evidence” isn’t a type of evidence at all. Even when made by reputable experts, these statements simply do not qualify as scientific evidence. This includes such categorizations as editorials, opinions, viewpoints, and perspectives. More suspect, in general, are weblogs purporting to be judges of the quality of scientific evidence. Considerthe qualifications of any blog author.

  • To consider as you read:
    • Beware of what you take away from these types of publications.
    • These are cited around the internet, particularly by individual blogs, as if they represent high-quality nutritional advice, guidance, or information—if it helps make the site author’s point

Observational Studies

This bracket includes three types of epidemiological research studies. Although results of these studies do qualify as scientific evidence, their level of quality is generally considered lower than studies included in the top two brackets of the pyramid. This is because they are not experimental in nature. In general, observational studies aim to investigate links and “associations”5“Associations” are not necessary the same as “cause and effect” relationships. Associations between two things may easily be due to something else entirely. Assuming one equates to the other is bad science and should be avoided at all costs. between possible contributing factors (called exposures) and disease outcomes.

Individual Case Reports

  • As the name implies, these are detailed stories, usually about an individual patient (although may be up to 3)
  • Other than background information and expert opinions, they are considered the lowest quality evidence of the types considered here
  • Positives
    • Might provide basis for future identification of new conditions, diseases, or their treatments (thus, their low position on the pyramid)
    • Highly educational, used in the training of medical professionals
  • Negatives
    • Lowest quality of published evidence
    • Neither experimental, systematic, nor generalizable
    • Not systematic
    • Many reputable journals don’t even publish case reports
    • Their stories can have explanations other than those described
    • To get published such reports might emphasize what’s highly unusual or unintentionally mislead readers
  • To consider as you read:
    • Is the patient described in considerable detail so that findings might be applied to similar individuals?
    • Are the findings reported in carefully noted, unbiased terms?

Case-Control Studies

  • Case-control studies are designed to determine whether an outcome of interest (be it a disease or condition) is associated 6“Associations” are not necessary the same as “cause and effect” relationships. Associations between two things may easily be due to something else entirely. Assuming one equates to the other is bad science and should be avoided at all costs. with a a given risk exposure (e.g., eating red meat or dairy)
    • The “case group” is identified as those known to have the disease or condition of interest to the study
    • The control group is identified as subjects who do not have the disease of condition of interest to the study
    • An “association” is defined by a measurably higher exposure observed in the case group compared to the control group
    • Any “association” found is merely that; it is not necessarily define  a cause and effect relationship!
    • For illustrative purposes, consider this graphic from an Epidemiology and Biostatistics course at Tufts:

Diagrammatic process for case control studies, which are retrospective in nature. The investigator, in the present, defines two groups of a study sample, cases and controls. The cases have expressed an outcome (disease or marker), while the controls have not. The investigator looks back in time to find the frequency of exposure to the variable of interest and compares the frequency of exposure in the cases vs. the controls to determine the association between them (though NOT necessarily the cause and effect).In the case control study, an investigator forms a study sample by selecting a group of case subjects who already meet the definition for an outcome (e.g., have high blood pressure) and a group of control subjects who do not. The investigator then looks back over time to determine the frequencies of exposure of interest (e.g. stopped smoking) in the two groups. An association is found if the frequency of exposure is significantly different for one group than the other.

  • In the case control study, an investigator forms a study sample by selecting a group of case subjects who already meet the definition for an outcome (e.g., have high blood pressure) and a group of control subjects who do not. The investigator then looks back over time to determine the frequencies of exposure of interest (e.g. stopped smoking) in the two groups. An association is found if the frequency of exposure is significantly different for one group than the other.
  •  Positives:
    • They are often relatively faster and less expensive than experimental studies (easier to do), because:
      • They rely on subject data from the past (i.e., are retrospective; see graphic above)
      • They don’t treat the subjects or attempt to alter the course of their condition in any way
    • Accordingly, their results sometimes provide the basis for subsequent experimental studies that might examine treatment or prevention
    • Good tool for rare diseases or conditions or those that take long periods of time to develop
  • Negatives:
    • Significantly, these research designs are subject to bias (easier to do wrong)
      • Case-control studies depend on careful matching of case and control groups; while often difficult, it is necessary for generating quality evidence
      • Retrospective studies usually depend to some extent on the subjects’ memories, the quality of such may vary widely particularly regarding what we’ve eaten or what we’ve done
  • To consider as you read:
    • Are the criteria for inclusion in the case group explicitly defined? Also, if there are eligibility criteria, are they clearly defined?
    • Do the control group subjects come from the same population as case group subjects? Also, have they been selected independently of their exposure to the risk of interest to the study?
    • Are the data gatherers blind to case/control group status, or at least blind to the hypothesis of the study (e.g., association of risk “x” with outcome “y”)?
    • Do the researchers address possibly confounding factors in their discussion of the results? (e.g., healthier eaters pay better attention to many lifestyle features)

Cohort Studies

  • In general, cohort studies track two or more groups of subjects forward, from exposure to outcome. This can be done looking back over time—retrospective cohort studies—or looking forward with time—prospective cohort studies.
  • To introduce this observational design, consider a pair of graphics from a Tufts University Epidemiology and Biostatistics course:

This diagrammatic view of a retrospective cohort study shows that the investigator forms two groups on the basis of their exposure to a particular variable of interest. S/he then compare the frequency of disease or other marker outcome at the time of the investigation to find associations.

In contrast to the case control study design (above), the investigator in a retrospective cohort study selects groups out of a large cohort based on their past exposure to a variable of interest (e.g., stopped eating red meat). The difference in frequencies of new cases of the outcome (e.g., high blood pressure) that developed in the exposed vs. non-exposed groups are compared at the time of investigation to determine an association.

  • Potential drawbacks of retrospective cohort studies
    • Not good as good for very rare diseases as case-control studies
    • Confounding factors (those that confuse the data/results) not easily excluded using records from the past
    • Often difficult to appropriately identify exposed and comparison groups from the cohort

This diagrammatic view of a prospective cohort study shows that the investigator screens for members of a large cohort who do not currently express the outcome of interest. This cohort is then followed over time and divided on the basis of whether they have experienced the exposure of interest. At some time later, the investigator compares the frequency of new cases of the outcome of interest in the exposed and unexposed cohort members to seek an association between the two.

The more common prospective cohort study investigator selects healthy subjects out of a large cohort at the time the study begins (i.e., they don’t express the outcome of study) that either have or have not been exposed to the variable of interest (e.g., stopped eating red meat) have a significantly different frequency of the outcome of interest (e.g., high blood pressure) at some future point in time. Again, associations are determined by significant differences in new cases between the groups.

  • Potential drawbacks of prospective cohort studies
    • Can be much more expensive and time consuming
      • Large number of subjects in cohorts
      • Long followup period
    • Also, not as useful as case-control studies for rare diseases/conditions or those that time long periods to develop
  • Advantages of cohort studies
    • Considered by some as the “gold standard” of observational study design”
    • Less subject to bias due to selection of subjects more/less likely to have develop the outcome, unlike case-control studies
    • Less subject to the subject recall bias of case-control studies
    • Useful in establishing timing of exposure relative to outcome
      • Can be useful in supporting cause and effect relationships
  • To consider as you read:
    • Are both of the cohort groups similar populations?

Experimental Studies

This bracket of the pyramid indicates two types of research studies. Though similar, the fundamental difference between the types of observational studies just described and these is the inclusion of some sort of intervention or change—in our case usually a dietary change of some kind.

Randomized (Controlled) Clinical Trials

  • Clinical trials might be designed for a variety of reasons, for example to compare:
    • A new dietary approach (e.g., a ketogenic diet) to one that is already available (e.g., the standard American diet (SAD);
    • Two different already-available dietary patterns to each other (e.g., a ketogenic diet to the Mediterranean diet); or
    • A dietary pattern to a drug of some sort (e.g., a DASH diet to a blood pressure-lowering drug).
  • Three general approaches to clinical trials in relation to a particular condition or disease:
    • Prevention
    • Screening or diagnosis
    • Treatment or improving quality of life
  • Common features of randomized, controlled clinical trial designs:
    • Investigators write up a research plan or protocol before the study begins. The plan or protocol clearly defines:
      • The primary research question
      • The population from which participants for the study will be drawn (eligibility criteria)
      • Rules for dividing participants into control vs. experimental subjects, specifically including randomization
        • This means that any given subject is assigned to the control or experimental group on the basis chance alone (e.g., by lottery), not based on any characteristic relevant to the trial
      • Rules for excluding potential participants who might confuse interpretations of the results (exclusion criteria)
      • What intervention or change the experimental group will be given
      • Whether investigators, data analysts, and subjects will be aware of or be blinded to control/experimental group membership
        • A single-blinded study usually means just one of either the investigatory staff or the subjects are aware who gets the intervention (usually investigators)
        • A double-blinded study usually means neither the investigatory staff nor the subjects know who gets the intervention until the trial and its analysis are complete
        • In nutrition research, participants are often aware of their group status based on the rules they follow (i.e., follow this diet or that)
      • How results will be analyzed and interpreted
    • The study progresses for a defined length of time, usually much shorter than either type of cohort study
    • At the end of the study, investigators evaluate the data for the impact of the intervention or change on a particular measurable outcome
    • Despite their quality, ethical and practical concerns can prevent their use
  • To consider as you read:
    • Are all of the participants in the trial from the same population?
    • Is there only one variable that’s different for participants in the control group vs. the experimental group?

Critical Appraisal

As we move away from evidence derived from intervention-based studies, we move into a more analytical bracket of research methods. This more top-down view of scientific evidence is a component of evidence-based medicine. A particularly useful description of this process of critical appraisal comes from the Center for Evidence-Based Medicine at Oxford:

Critical appraisal is the systematic evaluation of clinical research papers in order to establish:

  1. Does this study address a clearly focused question?
  2. Did the study use valid methods to address this question?
  3. Are the valid results of this study important?
  4. Are these valid, important results applicable to my patient or population?

If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of it.

Evidence from critical appraisal studies is generally considered to be of higher quality than the individual studies it aims to appraise, although one should be careful to avoid making assumptions about the quality of any study merely due to its type. In general, critical appraisals combine the data from a body of individual studies and provide critical commentary on the utility of those data for use in providing healthcare. A relevant example of critical appraisal would include “A critical appraisal of probiotics (as drugs or food supplements) in gastrointestinal diseases.”7Passariello, A., Agricole, P., & Malfertheiner, P. (2014). A critical appraisal of probiotics (as drugs or food supplements) in gastrointestinal diseases. Curr Med Res Opin, 30(6), 1055-1064. doi:10.1185/03007995.2014.898138

Systematic Reviews

Image depicting individual studies as components being droped into a sieve that processes them using the steps in the text, into a systematic review

Diagrammatic systematic review, edited from http://bitly.com/2DZQPZg

  • The Cochrane Library is a reputable source of systemic reviews for healthcare. These are steps that review authors must take to be included in the Library:8http://www.cochranelibrary.com/about/about-cochrane-systematic-reviews.html
    1. Identification of relevant studies from a number of different sources (including unpublished sources);
    2. Selection of studies for inclusion and evaluation of their strengths and limitations on the basis of clear, predefined criteria;
    3. Systematic collection of data;
    4. Appropriate synthesis of data.
  • Evidence from systematic reviews is considered to be higher quality than that from individual clinical studies, for example. This results from the inclusion of evidence from multiple individual studies; the more studies included, the higher the quality of evidence.
  • To consider as you read:
    • Is each study included in the systematic review looking at the same outcome? Each study should have examined the same variable (e.g., effects of probiotic treatment in gastrointestinal disease referred to above, not including any studies of prebiotic treatment)
    • Are results of the reviews applied to the same groups being studied in the included studies? No new grouping of participants should be applied (e.g., review of effects on participants aged over 65 years from individual studies where ages of participants weren’t segregated)

Meta-Analyses

  • These studies have the potential to generate the highest quality evidence in epidemiological nutrition research
  • The ultimate strength of meta-analyses rests in the strict statistical analysis of data that is pooled from as many individual studies as fit the criteria for inclusion. It can thereby greatly expand the number of individuals considered and significantly refine the recommendations for medical practitioners’ use
  • Advantage
    • Recommendation can more reliably be applied to the general population, having pooled data compiled from numerous different populations
  • Disadvantage
    • Requires complicated statistical analysis of data after a lengthy process of collecting as many published and unpublished studies as can be found
  • To consider as you read:
    • Is each of the pooled studies of the same type? (e.g., random clinical trials)
    • Are unpublished studies, as well as published, included to help minimize publication bias?
Resources:
  • Coggon, D., Rose, G., & Barker, D. (1997). Epidemiology for the uninitiated (4th ed.): BMJ Books.
  • Himmelfarb Health Science Library. (2011). Study Design 101. Retrieved  December 23, 2017 from https://himmelfarb.gwu.edu/tutorials/studydesign101/index.html
  • Kanchanaraksa, S. (2008, 2008). Cohort Studies. Retrieved December 28, 2017 from http://ocw.jhsph.edu/courses/fundepiii/PDFs/Lecture13.pdf
Follow by Email
Facebook
Facebook
Google+
Google+
http://wholefoodhealingforaging.com/nutrition-research-eaters">
Instagram
RSS