EDGE Lab Members & Peers Publish Analysis to Advance Metrics Around Food Insecurity

EDGE Lab PI Kathy Baylis, Erin Lentz (University of Texas, Austin), Hope Michelson (University of Illinois, Urbana-Champaign), along with EDGE Lab member Chungmann (Manny) Kim (University of Illinois, Urbana-Champaign) were commissioned by the Integrated Food Security Phase Classification (IPC) to study the accuracy of their food insecurity and malnutrition classification metrics. The IPC is an innovative multi-partner initiative for improving food security and nutrition analysis and decision-making. Details of their analysis can be found below, courtesy of IPCinfo.org:

The Integrated Food Security Phase Classification is the global point of reference for classification of food crises and famine and has been used in over 30 countries over the past fifteen years.

In 2022, the IPC commissioned, for the first time, an external study of the accuracy of its acute food insecurity and acute malnutrition phase classifications and population estimates. Because IPC outcomes and the underlying indicators that are used to generate IPC outcomes are latent constructs, there is no true measure of ‘accuracy.’ Thus, the first task of this study was to determine a methodology suitable to this challenge. In this study, the analysis examines IPC outcomes through the lens of internal consistency. Our approach is informed by a review of the methodological literature, the data available and findings from key informant interviews.

Drawing on multiple methods ranging from descriptive statistics to regression analyses, we compare IPC outcomes against a series of reference points informed by the IPC Technical Manual V3.1. We have rich data for acute food insecurity, and results focus on these findings. We show proof of concept findings for acute malnutrition.

To examine consistency of acute food insecurity analysis, we ask and answer four main questions.

• First, we ask what the underlying food security indicators suggest to Technical Working Groups about phase classification and population outcomes. We find that food security indicators are not highly concordant, pointing to the importance of the IPC consensus process.
   
• Second, we ask how Technical Working Group consensus processes use available information. We find that classifications of severity are highly consistent with the technical guidance provided in IPC Technical Manual V3.1 which serves as our benchmark. We also find that Technical Working Groups, on average, are under classifying populations in food crisis or above relative to our reference points; that is, on average they appear to be identifying fewer people as urgently food insecure (or worse) compared to what our reference analysis would point to.
   
• Third, we ask what could be influencing consensus outcomes. We find little evidence that any specific indicators are systematically disregarded or ignored. Consensus appears idiosyncratic: the treatment of food security indicators varies by several factors, raising questions about the comparability of the consensus outcomes. We also find that that Technical Working Groups tend to classify fewer people as food insecure when underlying food security indicators are noisier.
   
• Fourth, we ask how acute food insecurity projections relate to later analysis of the current status. We do not observe evidence of systematic over or under-prediction of the projections relative to the realized current study analyses.

We close with short and long-term recommendations, including reviewing food security indicator thresholds in the technical guidance, investigating pairing automated analyses with documentation to support consensus, comparability, and transparency, and undertaking a follow-up accuracy study in five years.