Analysis of the retraction request by Drs Behrens and Hayek for FAO’s “Pathways” report

Date of FAO Report: 2023/12/08; Date of retraction request: 2024/04/09; Date of this analysis: 2024/06/03

N. Lupton¹, F. Correddu², M. Lunesu², J.-F. Hocquette³, P. Ederer¹, G. Pulina^2*

1, GOALSciences, Switzerland
2, Università degli Studi di Sassari, Italy
3, INRAE, French National Research Institute for Agriculture, France
* corresponding author: gpulina [at] uniss.it 

Drs P. Behrens and M. Hayek, requested from the Food and Agriculture Organisation’s (FAO) that its 2023 report titled “Pathways to Lower Emissions” be retracted, revised, and re-issued due to a purported distortion of the findings, among others by their own studies from Behrens. This present analysis finds that this call for retraction and revision is not merited, for the following four reasons:

1. Only a small part of the overall FAO argument is dealing with the portions which are concerning to Drs Behrens and Hayek. The focus of the FAO report was not large-scale reduction of animal protein consumption and its effects, but rather on the necessary and possible improvements within the animal livestock sector specifically (FAO, p vii, 2023).
2. While the FAO could have deployed some minor methodological improvements, these are not as substantial that a retraction and revision is necessary. The essence and substance of the report are neither compromised nor would be significantly changed during such a revision
3. The main data on which Drs Behrens and Hayek are basing their request for retraction are themselves significantly flawed, and would lead to a significant misrepresentation of the subject matter.
4. The request for retraction is methodologically flawed in itself and can therefore not be considered a scientifically valid basis for a revision. The following will outline the details to the above assessment by analysing each argument of Drs Behrens and Hayek one by one.

The criticisms by Drs Behrens and Hayek are presented and analysed in the order in which they have been raised in their correspondence to the FAO. This shall be compared to what was stated by the FAO in the abovementioned report.

Misuse and conflation of the results obtained in the article by P. Behrens and his team (Behrens et al., 2017) in section 3: Scaling-up mitigation options, subsection 1: Changes in consumption of terrestrial animal source food (pg. 17-19) (FAO, 2023).

Behrens 2017 concluded that global emissions could be reduced by 0.19-0.53 Gt CO2 eq∙a-1, if 37 countries representing 64% of the global population, switched from their average diets to their relevant National Recommended Diet (NRD). The data used for this study was from 2011, and this renders this particular study outdated for the use intended by the FAO. Furthermore, the FAO seemed to assume that these reductions (reported in CO2 equivalents per year - CO2 eq∙a-1) would affect the different greenhouse gases (GHG) equally.

On pp 18-19, the FAO 2023 report stated: “Behrens et al. (2017) have analysed the potential impact of large-scale adoption of NRDs on GHG emissions. In HICs, such shift could translate to a reduction in GHG emissions of between 13 and 17 percent. However, the expected reduction in the middle-income countries is marginal (4.4 percent) with increasing emissions in some countries due to increased consumption of nuts, fruits and vegetables, partly grown in greenhouses. Taken together, these GHG reductions could amount to a decrease of 0.19 to 0.53 Gt CO2 eq per year for the 37 countries considered, representing a 2 to 5 percent reduction in emissions associated with the entire global food system.”

Re outdatedness: The global population has increased from 7.07 billion in 2011 to 8.05 billion people in 2023 (FAO, 2024). Behrens et al. (2017) used data from 2011 (global population, emissions and average diets). The FAO’s models and systems (like GLEAM etc.) use 2015 as a base year, and are thus not directly comparable to the study by Behrens 2017. However, while the absolute amount of emissions has broadly risen since 2011 proportionately to the increase of the population, the relative structure of the emissions will have changed only marginally. Therefore, regardless of the base year, the relative percentage of potential reductions also remain broadly the same.

Re equal affecting different GHG: While emissions and reductions potential are reported as CO2 equivalents, this does not automatically imply that the main GHGs were equally treated regarding their composition, properties, formation and heating indexes. This manner of reporting summarises the emissions in a quantitative fashion and is supposed to make the abovementioned differences between methane (CH4), carbon dioxide (CO2) and nitrous oxide (N2O) comparable. For transparency purposes, a clarification on the FAO methodology used to quantify the reductions in each individual GHG, followed by a conversion to CO2 eq∙a-1 would be advisable.

Ignoring of more ambitious dietary change (i.e. other than nationally recommended diets). 

The study Behrens et al. (2017) “focuses only on nationally recommended diets (NRDs) - most of which do not currently factor in environmental considerations in their design, and so are not reliable as an indicator of the emissions mitigation potential of ‘sustainable and healthy diets’.” The validity of NRDs as potential benchmarks for healthy diets based on their political nature is also questioned: “NRDs are not a reliable benchmark for sustainability due to their highly political nature, and the capacity for interest groups like the livestock industry to influence them - such as the US livestock industry successfully “influencing government officials to drop sustainability from official US dietary guidance” (Rose et al., 2021).  Also: “most dietary guidelines based on both health and environmental factors - such as the Eat-Lancet diet and the Nordic Nutrition Recommendations - recommend considerably lower animal product consumption. Research has shown that many NRDs are deficient in meeting both health and sustainability goals.” Using a better and coherent approach for answering this question, (Clark et al., 2020) estimate the opportunity for dietary change alone (plant-rich diets) is 3.08 GtCO2eq per year.

P 18, “Dietary changes that consider nutritional, health and environmental concerns about food consumption often derive from the national recommended diets (NRD). Typically, NRDs recommend reduced intake of sugars, dairy products, meat and oils, with large variations across individual countries.”

Re NRD’s in general: Drs Behrens and Hayek point out the obvious and non-contested fact that NRD’s often are politically motivated and are influenced by various industries and civil society groups within the food-sector as government policies are reflected in these guidelines. Culture also plays a role in these guidelines. This point is also not limited to the livestock sector, as other political actors with vested interests can and do influence the NRD’s. However, all of this does not render NRD’s a per se unsuitable candidate for analysis. Furthermore, to declare ‘most’ NRDs obsolete is cavalier and presumptuous.

Re suitability of NRDs: NRD’s are specific to their country of origin, and its socioeconomic background. In low-income countries, where poverty and malnutrition are prevalent, NRDs should (and often do) recommend an increase in foods like animal-source protein and fruits and vegetables in order to address protein-, vitamin- and mineral deficiencies (Chen et al., 2022). One-size-fits-all approaches are unlikely to do justice to these local circumstances, and are therefore less suitable for analysis.

Re EAT Lancet: The suggested use of the EAT-Lancet diet as a global standard for both nutrition and environmental health is inappropriate. This diet is highly contested from both nutritional- (Vieux et al., 2022), affordability (Chen et al., 2022; Hirvonen et al., 2020) and even environmental benefits (Perry, 2019) points of view. The EAT Lancet diet cannot be considered a suitable NRD by which to evaluate the emission reduction potential.

Re 3.08 GtCO2eq opportunity: Upon review of the article (Clark et al., 2020) by a number of analysts, this number could not be found in either the article or the supplementary texts. An explanation of where this value is to be found and how it was calculated is needed so that it could be considered in a revision.

Ignores the opportunity costs of land spared through dietary change, as a study by “researchers in Science found an opportunity of 3.10 Gt CO2 eq⋅a−1 using robust and appropriate modelling (increasing to 6.22 Gt CO2 eq⋅a−1 if the land that is spared is used to draw down carbon).”

P 25, “Furthermore, the potential for carbon sequestration is influenced by existing carbon stocks, making global quantification challenging and subject to change over time. Consequently, estimates of sequestration potential varies widely, further complicated by concerns over the reversibility of sequestration efforts (Godde et al., 2020) and variations in soil carbon estimation methodologies. The lack of representative evidence from different parts of the world underscores the need for long-term experiments to validate assumptions. The maintenance of grazing land is deemed more crucial than restoring degraded land, as the absence of former is likely to cause significant losses in soil carbon stocks.”

Re 3.10 to 6.22 GtCO2eq opportunity: These values are also supposedly provided by Clark et al. (2020) published in Science. However, these values cannot be found in this article upon review, so that an explanation is due which source or methodology justifies these values.

Re carbon sequestration: While the effect of carbon sequestration due to dietary change hasn’t explicitly been discussed by the FAO, carbon sequestration in livestock systems and grasslands has been discussed in a broad sense with the optimistic estimation of 0.6 Gt CO2 per year (FAO, 2023), from estimations ranging from 0.037-0.8 Gt CO2 per year (Godde et al., 2020). However, in contrast to the FAO’s optimistic estimations, the authors of Godde et al. (2020) recommend that other mitigation strategies be used as well, for example, prevention of deforestation and the reduction of food waste.

Re land-use change in general: land use change should not merely be considered from the perspective of emission reduction potential. Rather, an integrated approach should be considered, taking into account plant- and animal biodiversity, soil quality, water cycles, landscape maintenance, etc.

Contains some major methodological errors. Comparing total food systems emissions from Tubiello et al. (2021) with estimations for emissions mitigation potential from dietary change from Behrens et al. (2017) study is not appropriate, for the following reasons:

Tubiello et al. (2021) is incomparable to Behrens et al. (2017). Tubiello et al. (2021) deliberately expands the boundaries and definitions of emissions that count as “within the food system” beyond what previous studies have counted, including Behrens et al. (2017). A major stated purpose of this research paper was to discover and monitor a growing category of emissions: pre- and post-production emissions. As such, this research added emissions from sources such as food waste disposal and food processing. Behrens et al. (2017) did not consider these emission sources within the systems boundaries for their study.

Tubiello et al. (2021) consider and update estimates of land use change, which were not included in the EXIOBASE database of emission sources from food used by Behrens et al. (2017).

P 39, “The most recent estimate of livestock emissions using GLEAM 3, 6.2 Gt CO2eq, is based on the reference year 2015. This estimate constitutes approximately 38 percent of the total emissions from agrifood systems, which FAO estimates at 16.3 Gt CO2eq for 2015 (FAO, 2022a; Tubiello et al., 2021) (see Table 1). At the same time, it is lower than the previous global assessment of 7.5 Gt CO2eq for 2010, produced using GLEAM 2 (FAO, 2019a), or 7.1 Gt CO2eq estimated in Gerber et al. (2013) for 2005. The main difference is linked to the updated methodologies for estimating direct and indirect GHG emissions, as well as emissions associated with the LUC, including deforestation and pasture expansion.”

Re incomparability: Tubiello et al. (2021) indeed expanded the scope of emissions considered within the food system to include emissions during pre- and post-production stages, including emissions from food waste and food processing. These additional emissions were not considered in the study by Behrens et al. (2017), and so the reductions estimated in these studies are not directly comparable on an as-is basis. However, this constitutes only a minor methodological flaw without impacting the overall message.

Re different base years: Tubiello et al. (2021) used more recent data from the year 2020, compared to Behrens et al. (2017) who used data from the year 2011. This renders Behrens et al. (2017) as a less appropriate source for emissions projections. According to FAOSTAT, the global human population stood at 7.07 billion in 2011 and increased to 7.91 billion people by 2021 (FAO, 2024). That, coupled with the changing of many NRDs to include guidelines and recommendations for lower consumption of meat and dairy products, would likely result in somewhat different emissions projections compared to Behrends 2017. However, as mentioned before, the overall difference is unlikely to be large and does not change the essence of the analysis.

Double counting of emissions from meat consumption.

The FAO factors in a projection of an extra 2,871 Mt CO2eq in Figure 12, driven by increased global meat consumption. But Behrens et al. (2017)’s net emissions savings due to dietary changes also factor in increases in emissions in some countries due to increased meat consumption, mainly in lower income countries, to bring them in line with their national dietary guidelines - which significantly offset and therefore obscure some of the emissions savings caused by the countries reducing their meat consumption. The emissions from these projected increases in meat consumption are therefore being double counted in the graph - once in the additional emission (BAU) bar, and again in the Behrens et al study, in a way which further diminishes the emissions reduction potential from dietary change.

<enter graph from the FAO p 31>

P 30-31, “With no change in productivity, the increase in demand for TASF by over 20 percent compared to 2020 would have to be met by an equivalent rise in the overall number of animals. This would result in a proportional increase in both upstream and downstream emissions, elevating baseline livestock emissions from 6.2 Gt CO2 eq in 2015 to 9.1 Gt CO2 eq by 2050. This BAU scenario outlined here assumes no efficiency improvements throughout the production chain. Consequently, it explores the extent of the absolute emissions change associated with a specific intervention.”

Re the double counting: Figure 12 focuses mainly on the emissions from livestock systems, whereas the study by Behrens et al. (2017) considers dietary changes across the board, including an increased consumption of fruits, vegetables, and nuts. This recommended increase, found in many NRDs, is not always related to the substitution of animal-based protein foods with plant-based alternatives.

Thus, if emissions, and their reductions, due to the increased consumption of plant-based foods are to be considered in this figure, it should only be in cases where these plant-based foods would replace animal-based foods, either as protein or energy. Since the study by Behrens et al. (2017) does not consider mere increases in plant-based food consumption due to the lower consumption of animal-based foods, it is an inadequate study for this purpose.

However, the purpose of the BAU bar was to contrast the “business-as-usual” scenario with the possible reductions and improvements within the livestock sector. These improvements include the increase and decrease in animal consumption due to a changing human population size and a shift towards more appropriate diets across the board. Considering the scope of the study by Behrens et al. (2017), the double counting of emissions due to increased meat consumption is likely very minor. Furthermore, if we use the FAO’s methodology in considering a global shift towards NRDs as a way of reducing global emissions in the livestock production sector, then no double counting has taken place as the BAU model assumes that this shift towards NRDs does not take place.

“Replacing meat with calorically equivalent greenhouse vegetables or out-of-season fruits flown from afar could potentially reverse many GHG emissions offsets".

The only evidence cited for this claim by the FAO is Fresán and Joan Sabaté (2019) which in turn cites Vieux et al. (2012) - a study which only references greenhouses once to say that “Jungbluth et al. (2000) observed that the greatest environmental impacts were associated with fresh food that is flown from another country and with greenhouse production and meat consumption” - a very out of date study. In fact, Fresán and Joan Sabaté (2019)’s main finding is that “Greenhouse gas emissions resulting from vegan and ovolactovegetarian diets are ∼50% and ∼35% lower” than current diets - information omitted by the FAO report. Additionally, fruit and vegetables transported from other countries is not a meaningfully large source of greenhouse gases, except in a few cases of berries or asparagus. Most food is shipped by ocean, as well as truck and rail. Altogether, only 0.16% of food is air-flown.”

P 18, “the actual reduction of GHG emissions resulting from dietary changes depends on how the animal protein is substituted and how these substitutes are produced. For instance, replacing meat with calorically equivalent greenhouse vegetables or out-of-season fruits flown from afar could potentially reverse many GHG emissions offsets (Fresán and Sabaté, 2019).”

FAO makes a valid general point, and mentions air-flown vegetables as one out of many possible examples. It is perhaps an unfortunate choice of example, but that does not invalidate the overall message. The criticism by Drs Behrends and Hayek are taken out of context and out of proportion.