Post by S. Stanley Young and Warren Kindzierski
Summary
The World Economic Forum, supported by academia food researchers, wants you to believe that meat is unhealthy compared to a diet of soy, tofu, insects and fungi. The statistical work on food research presented here shows that this is not the case. Food frequency questionnaires (FFQs) are used in population cohort studies. Years later, this information along with observations of health outcomes are combined in statistical analyses. These analyzes easily lead to more than 20,000 food-disease associations tested in a typical FFQ study – known as the multiplex trial. Researchers can then search and filter and only report the results they want, but many of these results can be false. Red meat is not healthy. It is the belief in deceptive statistical practices and false claims from academic food researchers to be unhealthy.
Introduce
Kip Hansen’s recent Article WUWT died about the nonsense behind meat being a problem of climate change. The World Economic Forum (WEF), with the support of academics, wants you to believe that meat is unhealthy compared to soybeans, tofu, insects and fungi protein rations.
WEF confirm that in the future “… meat will be a specialty dish, not a staple food for the sake of the environment and our health.” Scholars say eating red meat reason mortality, many types of cancer (colorectal, breast), type 2 diabetes, and the list goes on. Does this mean?
There is a saying… hard math. Well, as will be shown, statistics seem to be even harder for academic food researchers. An inside look at the statistical workings of food research (nutritional epidemiology) is one way to express this and address negative health claims about red meat.
Story
Many food claims – beneficial or harmful – are made based on observational studies of large groups of people known as cohorts. These groups are provided with a food frequency questionnaire, FFQ. The FFQ asks questions about the different types and portions of food consumed. Years later, food researchers asked about their health status.
They then performed a statistical analysis of the association between food and disease with the collected data. Surprising food-disease associations end up as published research claims. But are these claims true?
Unhealthy red meat is arguably deserving of special attention by the WEF. Kip Hansen’s Article WUWT pointed out a review of FFQ studies on red meat completed by Bradley Johnston researchers in 2019. It was an international collaboration that examined red meat consumption and 30 different health outcomes.
The Johnston team reviewed the published literature, selected 105 FFQ studies, analyzed them, and presented their findings in the journal Nature. Annals of Internal Medicine. They take the opposite view of the WEF – studies involving red meat are unreliable. Their findings have created a firestorm among food researchers, who are mostly academics. More on that later.
Analysis
Statistically confirming the same claim in another study is the foundation of the science. This is called replication. Given the potential importance of the Johnston study, it was recently independently evaluated in a National Association of Scholars report.
In the report, 15 out of 105 FFQ studies were randomized and counted for specifics. This included counting the number of foods, the number of health outcomes, and the number of adjustment factors in each of the 15 studies.
Food researchers use different techniques to manipulate the FFQ data they collect. The researcher’s flexibility allows food categories from the FFQ to be analyzed and presented in some ways. This includes individual foods, food groups, a nutrient index, or a food group-specific nutrient index. It was found that between 3 and 51 (mean of 15) foods were used in 15 studies.
The number of health outcomes only ranged from 1 to 32 (median 3) in 15 studies. Modulators can change the food-disease association. Nutritionists almost always include these factors in their analysis. These factors ranged from 3 to 17 (mean of 9) in 15 studies.
With these quantities, the analytical search space can be estimated. This is the number of foodborne illness associations examined in an FFQ study. It is estimated to be estimated as ‘number of foods’ ´ “number of health outcomes” ´ “2 is raised by the power of the number of modifiers”.
The typical (mean) analytic search space estimated in 15 studies is more than 20,000 won. A large analytical search space means that many possible links can be checked. Food researchers could then search through their results and select and report only surprising, but possibly false, results like we do today.
Now the elephant in the room… many of these types of analysis are likely done by researchers with incomplete understanding of statistical methods.
The p-value is a number calculated from a statistical test. It describes the likelihood (probability) you find a surprising outcome. It is a number between 0 and 1. The smaller the number, the more unexpected (the greater the probability).
The usual threshold for statistical significance for most sciences is a p-value of less than 0.05. Researchers can claim a surprising result if the p-value in a statistical test is less than 0.05.
However, a false discovery (chance) can occur about 5% of the time when multiple tests are performed on the same data set using a threshold of 0.05. Five percent of the 20,000 possible links tested could lead to 1,000 false positives in one study.
Performing multiple tests on a data set is called many tests. Assume 20,000 associations are tested on the FFQ study dataset on red meat. Usually, only a few dozen results from all of these tests will eventually be presented in a published study.
Of course, some results will be surprising. For example, an extravagant claim that red meat can lead to complications related to erectile dysfunction. Otherwise, their research may not be accepted for publication.
With multiple trials with 1,000 possible false positives and only a few dozen results presented, how do you know if the claim that red meat leads to complications of erectile dysfunction is true or just the result? wrong?
Without access to the original dataset to test or claim a claim, you can’t! The Johnston team was right in calling the FFQ studies on red meat unreliable.
Fire storm signal. Nutrition thought leaders – from Harvard – grant the title to the chief editor of Annals of Internal Medicine to pull out Johnson’s paper before it appeared in print. Editor hold on. The food research crowd did not prevail.
Implications
Too many nutrition thought leaders, mostly academics, argue that repeated testing is not a problem in food research. They teach it’s not a problem. They were wrong, it was a big deal.
There are no problems for them, but major misinformation problems for others when false findings are asserted as true outcomes. John Ioannidis from Stanford and others has called many trials one of the biggest contributors to falsely published research claims.
The FFQ studies using multiple tests and affirming red meat as unhealthy are primarily academic exercises on statistical flimflamming. Red meat is not healthy. It is the belief in deceptive statistical practices and false claims from academic food researchers to be unhealthy.
There are more than 50,000 food-diseases published research since the FFQ was introduced in the mid-1980s. Basically all of these studies involve lots of trial and error.
S. Stanley Young is the CEO of CGStat in Raleigh, North Carolina and the Director of National Scholars Association’s Teleport Sand Project. Warren Kindzierski is a retired university professor in St Albert, Alberta.
Related
