Reports on nutrition science seem to constantly ping-pong back and forth: One day eating red meat is “bad” for you. The next it’s not. For those seeking to lead a healthful life, it’s hard to know what to believe.
Now a new set of studies attempts to cut through some of the noise. The research quantifies the evidence for the health risks associated with common behaviors, such as smoking, eating red meat, or consuming few vegetables—and assigns each risk-outcome pair a rating of one to five stars based on the strength of the data involved. The studies were published on Monday in Nature Medicine.
The new metric assigned a five-star rating to the link between smoking and lung cancer, meaning there is extremely strong evidence of an association. It did so for the link between high blood pressure and ischemic heart disease as well. But the links between consuming unprocessed red meat and colorectal cancer had only a two-star rating, meaning there is much weaker or conflicting evidence of an association. The same was true for the link between unprocessed red meat consumption and breast cancer, diabetes and ischemic heart disease. The link between eating vegetables and lower risks of ischemic heart disease and hemorrhagic stroke also garnered just two stars.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
The researchers hope such ratings—available as an online tool—will help consumers make decisions about their own risk, officials develop health guidelines and research funders decide where to allocate resources for further study.
“We wanted to react to people out in the scientific community and in the media and just generally consumers who are puzzled by how studies go from one result to another,” says Christopher Murray, director of the Institute for Health Metrics and Evaluation (IHME), based at the University of Washington, and chair of health metrics sciences at the university, who headed the team that conducted the studies. “We wanted to come up with a way to summarize the available evidence and say, ‘Here’s, on average, what we think is going on’ [but also] ‘How sure are we that that result isn’t going to change?’”
The findings were part of the Global Burden of Disease study, a more than 30-year-old research project led by IHME and funded by the Bill & Melinda Gates Foundation to quantify the magnitude of all major diseases, risk factors and intermediate clinical outcomes in a standardized way.
Importantly, the new ratings donot show that eating red meat isn’t associated with certain health problems or that eating vegetables does not help protect against them. Rather they suggest the evidence supporting these associations is much less strong than, say, the evidence that smoking causes lung cancer. Low star ratings indicate findings could change as better studies are done and more evidence accumulates, the researchers say. “We’re not saying there’s no risk associated with red meat consumption,” says Jeff Stanaway, an assistant professor at IHME. Rather, a one- or two-star rating is “telling us either that the evidence is very mixed or that the association is rather weak.”
But some experts say the findings merely reinforce what was already well known among nutrition scientists: nutrition research is hard to do. “It’s extremely difficult to know what people are eating. There are standard ways of collecting that information, and all of them are flawed,” says Marion Nestle, emerita Paulette Goddard Professor of Nutrition, Food Studies and Public Health at New York University, who wasn’t involved in the new papers. Most nutrition studies, she says, rely on self-reporting by participants, and most people either can’t remember exactly what they have eaten or tend to overestimate the amount of healthy versus unhealthy foods in their diet. The authors of the Nature Medicine papers “just discovered that it’s difficult to do nutrition research,” Nestle says. “What do we know now that we didn’t know before?”
Others claim the new rating system wasn’t nuanced enough to capture the subtleties of nutritional studies. “The star rating, I think, is a bit simplistic and arbitrary,” says Frank Hu,a professor and chair of the department of nutrition at Harvard University’s T. H. Chan School of Public Health. “We’re not talking about rating hotels or restaurants.”The papers’ rating system assumes that the differences between studies are the result of uncertainty in the risk relationship itself rather than of meaningful differences between study populations, he says. But some of the variation in findings on red meat stems from the fact that populations in different countries consume very different amounts: people in Asian countries consume less red meat than Americans, and when they do, it’s often a marker of wealth, Hu adds. “This doesn’t mean that the heterogeneity or the difference in population is due to uncertainty of the findings,” he says.
Another limitation of trying to evaluate the risks of individual foods is that it’s very difficult to study one food in isolation. “People don’t eat hamburgers,” Nestle says. “They eat hamburgers, fries and soda.”
Nestle and Hu both worry the ratings could be used to cast doubt on dietary advice that, while not incontrovertible, is supported by a wealth of research: namely, that diets high in vegetables and lower in red meat are healthier. In the past, cigarette companies used a doubt-casting strategy to sow uncertainty over the risks of smoking, and some researchers fear the meat industry could use these low star ratings in a similar way.
“It’s important to recognize that a low-star rating doesn’t mean that there is no connection between a dietary habit and a specific health outcome—in most cases, it suggests that there may be a link, but the body of evidence is not yet clear,” Murray and his colleagues wrote in an e-mail to Scientific American. “A person who wants to minimize health risks should still take those low-star ratings seriously. For example, in the case of red meat, someone who wants to assume no level of risk shouldn’t eat it. The Burden of Proof analysis is intended to help people make informed choices. It is intended to both inform and complement—but not replace—expert guidelines and physicians’ advice.”
While the research may not be surprising to nutrition experts, “the general public often only hears about nutrition studies via news reports and social media that don’t include this nuance,” the researchers wrote. “Our hope is that the Burden of Proof system will help those who aren’t experts, including consumers and many policymakers, better understand our level of confidence in the links between specific risks and health outcomes.”
With regard to Hu’s claim that the star rating system doesn’t capture differences in geography or population, the researchers say that decision makers in a particular country or region could choose to lend more weight to studies conducted in their own populations. But from a global perspective, there is real disagreement between studies that their analysis tries to capture.
Taken together, the risk ratings could suggest that existing science simply doesn’t tell us much about nutrition. But numerous expert committees have reviewed the research on various food groups and concluded that enough is known for experts to make recommendations. Rather than applying a “burden of proof” approach—in which everything must be conclusively proved beyond any reasonable doubt—Hu says researchers should apply the precautionary principle when there is evidence of harm.
And there may be: in 2015 the World Health Organization’s International Agency for Research on Cancer classified red meat as probably carcinogenic, “based on limited evidence that the consumption of red meat causes cancer in humans and strong mechanistic evidence supporting a carcinogenic effect.” It classified processed meat—meat that has been salted, cured or smoked, for example—as carcinogenic to humans based on “sufficient evidence” that consumption causes colorectal cancer. (Processed red meat carries different risks than unprocessed red meat, however, and the IHME researchers have not released a separate star rating for the former.)
In 2019 the Annals of Internal Medicine published guidelines from the Nutritional Recommendations (NutriRECS) consortiumsuggesting that Americans should continue their current levels of consumption of red and processed meat. The article describing the guidelines was widely criticized for flaws in methodology and for not disclosing one of the authors’ ties to the beef industry (a common source of bias in nutrition research). The journal subsequently issued a correction disclosing that author’s conflict of interest.
Given the variability in nutrition research, can we really know anything about what foods are healthy or unhealthy? “I think there’s an enormous amount of information available—phenomenal now—that vegetables are good for you and red meat is better for you in small amounts or smaller amounts than a lot of people eat,” Nestle says. “Is the research behind that incontrovertible? No, obviously not.” But she thinks science writer Michael Pollan captured the best nutrition advice well when he wrote in his 2008 book In Defense of Food: An Eater's Manifesto, “Eat food. Not too much. Mostly plants.”