Humane League Labs Report E009R01

Published November 7, 2018; Revised November 12, 2018

A pdf version is also available.

Abstract

We consider the self-reported dietary measurement instruments currently recommended for animal advocacy research and possible alternatives to using these instruments. Reviewing the biases associated with self-reporting and comparing these self-reports with direct biochemical measurements of diet, we conclude the animal advocacy research community should strive for more accurate measurements of diet change and move away from self-reported measurements. Having identified this problem, we provide an overview of current and developing alternative measurements of diet change. First, we review commercial sources of food purchasing data. Second, we consider efforts to collect data with retailers and institutions through relationship-building with those organizations. Third, we consider predictive biomarkers as a developing alternative for directly measuring the diets of individuals. Finally, we relate the needs of animal advocacy researchers to the broader issues of measuring and understanding the food system.

Introduction

Many interventions in animal advocacy seek to reduce the demand for animal products in order to reduce the number of animals used for food. In particular, interventions like leafletting, online advertising and pledge programs often urge individuals to adopt more plant-based diets. Intervention research seeking to measure these changes has thus far almost exclusively used self-reported outcomes, whereby individuals are asked to complete a survey on their dietary behaviors or intentions. In this report, we examine the biases associated with self-reports of change in diet and the data validating these self-reports against biochemical measures of diet. In light of this evidence, we explore alternatives to self-reporting in hopes that popularizing these alternatives may lead to more animal advocacy research using non-self-reported measures of diet.

Before considering the details of self-reported diet change and its alternatives, we should consider more precisely measurement goals of intervention research. Ultimately, intervention research seeks to measure the change in the number of animals required for use in food caused by an intervention. Of course, the number of animals affected is not readily measured and instead several proxies may be used. Self-reports of diet are designed and validated to measure the amount of food actually eaten by an individual. By contrast, the commercial and collaborative data sources explored in this report measure food disappearance, which includes the significant fraction of food that is never consumed due to waste, spoilage or other loss. The prevalence of dietary patterns, like veganism and vegetarianism, is another proxy to consider. However, interventions encouraging individuals to eat fewer animal products (termed "reducetarian", "flexitarian" or "semi-vegetarian" diets) are widespread in animal advocacy and could represent significant change in diet. As such, measuring the incidence of dietary patterns may be insufficient to evaluate these interventions and this proxy is not further considered here. While no measurement is perfect, intervention research should strive to use proxies that can be used to approximate the effect of interventions on animal lives.

Self-reported dietary measurements

Both Animal Charity Evaluators ("Dietary Assessment" 2016) and Faunalytics ("Food Frequency Questionnaires" n.d.) have promulgated best practices for self-reported dietary measures using food frequency questionnaires (FFQs) and the Automated Self-Administered 24-Hour (ASA24) dietary assessment tool. In an FFQ, the respondent is asked to indicate the frequency with which they consumed particular food items over a period of time or their intention to consume those food items in the future. In the ASA24, respondents are asked to comprehensively recall the last 24 hours of their diet through a series of prompts. (For an example, see "ASA24 Demonstration Site" n.d.) Both approaches are prone to significant biases, both intrinsic to survey research and unique to their use in animal advocacy research. Examining validation data comparing actual and self-reported diet confirms these biases are strong in self-reported dietary measures.

Biases

All surveys are subject to response bias and those used for diet measurement are no exception. FFQs rely either on respondent recall, which is prone to memory error, or respondent's prediction of their future behavior, itself unreliable. ASA24 mitigates the risk of memory error by using a shorter recall period; however, this is at the expense of higher variance since the diet of the last 24 hours might not be representative of the respondent's average diet. In either, approach pre- and post-treatment surveys can be employed to compare the respondent's diet before and after an intervention. Additional measurements can further reduce variance and provide a better estimate of long-term consumption behaviors. However, this approach introduces non-response bias if some respondents do not follow-up and the possibility of reactivity, where the repeated measurements cause respondents to alter their behavior. (Thompson and Subar 2013)

Social desirability bias, where respondents are inclined to provide answers they believe to be socially desirable or desirable to the researcher, is a particular concern to animal advocacy research. Animal advocacy interventions often rely on arguments surrounding the negative moral, health, environmental and social justice implications of consuming animal products. As such, it is reasonable to believe social desirability bias in self-reports is large. While propensity for social desirability bias can be measured and then mitigated via statistical modeling, without validation one cannot be assured of the success of this approach.

Validation

With these biases in mind, researchers have worked to validate self-reported diet measures against recovery biomarkers, a small group of molecules that can be used to biochemically measure some aspects of diet. ("Learn More About Biomarkers" n.d.) In particular, sodium, potassium, protein and energy consumption can all be measured with high accuracy. By comparing the self-reported and direct biochemical measures of diet, we can understand the errors introduced by self-reporting.

Freedman et al. (2014) collated the results of five validation studies measuring protein and energy consumption. Each study used a combination of food frequency questionnaires and multiple 24-hour recall surveys. While the ASA24 is not directly tested in any of the validation studies, its progenitor, the Automated Multiple Pass Method (AMPM) developed by the United States Department of Agriculture, is used in four of the five. ("ASA24® Evaluation & Validation" n.d.) The AMPM is administered by a trained interviewer with five rounds of questions and careful prompting on serving size (Raper et al. 2004); it represents an ideal self-reporting of diet. As such, we might expect the accuracy of the AMPM to be an upper bound on that of the ASA24.

The results of this analysis are reported in terms of the Pearson correlation coefficient (Freedman et al. 2014 Web Appendix 3), which represent the strength of correlation between two variables, with zero indicating no correlation and one perfect correlation. Overall, these coefficients are low (Freedman et al. 2014 Table 7). For energy consumption measured using the combined results of three 24-hour recalls, coefficients averaged across the studies were 0.28 for men and 0.34 for women. FFQs fared worse, with coefficients of 0.16 and 0.24 for men and women, respectively. For protein consumption, the numbers are somewhat better with coefficients of 0.48 and 0.49 for men and women using 24-hour recalls and 0.28 and 0.29 using FFQs. However, even in the best cases, the self-reported dietary measurements explain less than a quarter of the variance in actual diet.

These results also imply an increase in variance resulting from the use of self-reports (See Section 7). This increase in variance reduces the statistical power of intervention research using self-reported outcomes, necessitating a larger sample size. (Sethu 2016) To illustrate this, consider an experiment evaluated with a two-sample $t$-test, aiming to detect a medium effect size of 0.5 with 80% power and false positive rate $\alpha =0.05$. Self-reporting as above with 24-hour recalls would increase the number of required participants 1.4-fold in the best case scenario of measuring protein consumption in women. In the most extreme case, 5.2 times as many participants are required to measure energy for men using self-reports rather than direct measurement. Especially in mass outreach like leafletting or online advertising, where a small effect may still be of interest, this loss of power poses a barrier to successfully detecting effects. Given these results, which use more rigorous survey instruments, applied in triplicate, and without significant social desirability bias, we should assume the self-reported surveys common in the animal advocacy literature are even less accurate.

Commercial marketing data

A more direct alternative to self-reporting, is measuring what consumers purchase. In the United States, there are three major streams of consumer food purchasing data available to researchers: retail scanner data, measuring the sales across retailers; the National Consumer Panel, surveying a panel of consumers to measure their purchasing; and customer loyalty cards, which track the purchases of individual consumers at particular retailers. Comprehensive retail scanner data and access to the National Consumer Panel is available through two companies, Nielsen and IRI, while smaller companies handle scanner data for just a few retailers (eg, 84.51) or specialized sectors (SPINS for organic and natural foods). (Martin 2014) Both data sets are available for purchase through negotiation with either company or, for academics, at a significant discount through a partnership with the Chicago Booth School's Kilts Center for Marketing. ("Nielsen Datasets Consumer Panel Data" n.d.; "Nielsen Datasets Retail Scanner Data" n.d.) The market for customer loyalty card data is still developing, with numerous vendors representing different retailers. These sources and their strengths and weaknesses are described below, with weaknesses shared by all three explored in a final section.

Retail scanner data

Retail scanner data is collected at the point of sale (ie, cash register) of 35,000 stores across the United States, describing each item purchased by consumers with a Universal Product Code (UPC) and the purchase price. The data available from Nielsen or IRI is aggregated across regions and product types, although this is likely negotiable. However, the academic data set is available disaggregated, but anonymized at the retailer level, and starting in 2004.

While scanner data includes a wide variety of food retailers, some major retailers do not participate (eg, Trader Joes and Aldi). ("Nielsen Datasets Retail Scanner Data" n.d.; Martin 2018b) Comparison to other data on US retailers and their sales suggests IRI retail scanner data covers only 41% of retailers and 55% of food and alcohol sales dollars, although some of this discrepancy is attributable to different inclusion criteria of retailers. (Muth et al. 2016) Furthermore, private label products, which are provided and sold by only a particular retailer, are often omitted from retail scanner data or aggregated in such a way as to impede the calculation of prices per unit. (Muth et al. 2016)

National Consumer Panel

The National Consumer Panel (NCP; formerly known as the HomeScan Consumer Panel) is a joint project between IRI and Nielsen whereby a panel of households is recruited and incentivized to use a bar code scanner to report all items purchased each week. ("National Consumer Panel" n.d.) About 40-60k individuals are represented in the panel, with data beginning in 2004 and tracking the same set of products as the retail scanner data. ("Nielsen Datasets Consumer Panel Data" n.d.) While this method ultimately relies on participants voluntarily scanning items, recall and prediction biases associated with self-reported measures are mitigated and social desirability bias would at least require more direct action (ie, deliberately not scanning items).

However, the NCP is less suitable for intervention research. First, the panel is already under intense scrutiny, with surveys and "special studies" routinely run on the panel. ("Join NCP Today" n.d.; Hunter 2018) These may exacerbate reactivity and render results non-generalizable. Second, the panel has a retention rate around 50%, suggesting a strong selection bias, and may not succeed in being representative of the US population. (Hunter 2018; Muth et al. 2016) Third, products sold by weight (so-called "random weight products") may not have the associated weights recorded in the data set, producing another uncertain step in analysis. (Muth et al. 2016) Fourth, some animal advocacy interventions are not readily applied to individuals spread across the country; for example, billboards or even direct mail would be difficult to distribute to panel participants cost effectively. That said, alternatives to the NCP are under development, especially those reducing the burden on participants by using optical character recognition (OCR) to rapidly scan receipts, rather than arduously scanning item by item. (Hunter 2018)

Customer loyalty card data

Customer loyalty cards are issued by retailers to their customers and often include discounts and other rewards for purchasing products at that retailer. Loyalty cards can then produce longitudinal data on the detailed purchasing habits of a particular customer at that retailer. However, these programs are still subject to selection and reporting bias, where only some customers opt to enroll in a loyalty card program and may not always use their card. That said, anecdotal reports suggests even data from a single retailer is well correlated to national aggregate trends. (Hunter 2018)

Common challenges

Despite the "Universal" of UPCs, coding and accuracy is still a concern as many UPCs are not readily matched to their corresponding products. (p82 Improving Data to Analyze Food and Nutrition Policies 2005) Other UPC codes need to be anonymized so as to maintain the anonymity of retailers carrying private label products. ("Nielsen Datasets Retail Scanner Data" n.d.) Loyalty card data produced via OCR may not even include UPCs.

Unlike self-reported measures of diet change, commercial marketing data does not include all food (consumed) away from home (FAFH), which accounted for 34% of calories in US diets in 2013-14 (p98 Saksena, Okrent, and Anekwe 2018). (However, some FAFH, like a meal at a friend's home, may still be included in grocery scanner data.) Additionally, scanner data is available only for brick and mortar stores, not online retailers, although Nielsen and IRI have separate data products covering some of these channels. (Martin 2018a)

Further complicating matters for animal advocates, it is crucial that changes in diet be translated to number of animals effected, which necessitates estimating the fraction of animal products in some items. This data is not readily available and is at odds with the internal organization of the marketing-oriented data vendors, which cater to providing data on individual products, rather than all products containing chicken meat, for example.

Direct collaborations with restaurants, grocery stores and institutional dining services

Given the challenges and expenses present in commercial marketing data, exploring more direct collaborations with restaurants, grocery stores and dining services at colleges and universities is a developing avenue for obtaining data on dietary change. While the exact data available, feasibility of forming such relationships and costs are less clear, at least one group of researchers has had important success. By collaborating with a dining hall at Occidental College, Joshua Tasoff and collaborators procured "a rich dataset of individual-level food consumption data of three meals per day, 20+ weeks per year for the ~2,000 students at the college." (Stuessy 2017)

Such data would be more powerful than that provided by retail scanner data, which cannot track changes across individuals. College campuses are a particularly valuable source of data as they tend to be geographically isolated, with students consuming many of their meals and spending much of their time on campus. This allows interventions to be readily applied and measured. Furthermore, the college demographic has always been a prime target for animal advocacy intervention as college students tend to have agency in their food choice and be eager to explore new world views. The Menus of Change University Research Collaborative is actively working to facilitate collaborations between researchers and universities in the US, with sustainability goals that align relatively well with animal advocacy outcomes. (America and University 2018)

Such collaborations could also be forged with public institutions like schools and hospitals or anywhere that serves food to individuals. Academics and researchers at non-profits may be especially well positioned to build mutually beneficial good-will collaborations. In forthcoming research, Humane League Labs will be working to expand collaborative programs to other US university and college campuses, which could provide data of unprecedented quality to animal advocacy researchers.

Predictive biomarkers

As discussed in Section 2.2, recovery biomarkers are available to precisely measure a handful of dietary parameters. Recovery biomarkers are immediate byproducts of the parameters they measure and are thus less prone to variation between individuals. Predictive biomarkers are a developing approach that may offer measurements of other aspects of diet like animal product consumption. Predictive Biomarkers for animal products in the diet are usually substances produced during the metabolic breakdown of food, called metabolites, and found in blood, saliva or urine. Food additives and contaminates, like antibiotics in cows and polychlorinated biphenyls (PCBs) in fish, also have potential as biomarkers. (Turunen et al. 2010; Ji et al. 2010) While dietary biomarkers have been studied for decades, the field of metabolomics has recently accelerated development by measuring the levels of thousands of different metabolites and considering the patterns in which they occur. (Cheung et al. 2017; Schmidt et al. 2015) Using metabolomics, numerous predictive biomarkers correlating with different foods, food groups and dietary habits have been discovered.

Distinguishing diets like veganism, vegetarianism and meat-eating has been relatively successful (Schmidt et al. 2015), while efforts to measure consumption of dairy and eggs (Münger et al. 2018) and meat products from chickens, cows and pigs (Dragsted 2010) are still in early stages. Red meat has received the most interest for its role in disease, and we might expect red meat biomarkers to be among the first developed. Meanwhile, biomarkers for animal products from many different species, like fish products, will likely be developed later. The development of dietary biomarkers must also overcome the numerous confounding genetic and lifestyle factors, like metabolic rate, smoking, exercise, and differences in cooking conditions. Current approaches also tend to focus on short-term biomarkers indicative of consumption only within the few days before testing, rather than consumption over weeks or months (Gibbons and Brennan 2016). With these complications and others, estimates of food consumption from biomarkers may be too imprecise to be useful for some time. However, with active coordination by the Food Biomarkers Alliance (Brouwer-Brolsma et al. 2017), this field is rapidly developing and perhaps in the future animal advocacy researchers could utilize such biomarkers in measuring diet change.

Conclusions

We've explored current and developing alternatives to self-reporting of dietary data. Obtaining this crucial outcome data still poses a barrier to animal advocacy research. The challenge is not unique to the field and a National Academy of Sciences report also found "gaps that limit the information that policy makers and researchers have to address current and emerging issues in food and nutrition." (p4 Improving Data to Analyze Food and Nutrition Policies 2005) These challenges, however, may not be intrinsic, but appear to be the product of lobbying by the food and marketing industries. For example, the Argus Leader, a South Dakota newspaper, has been embroiled in a 7-year lawsuit to obtain data on the use of the publicly funded SNAP program, which offers nutrition assistance to millions of Americans at the annual cost of $68 billion taxpayer dollars. In the last months, lobbyist from the Food Marketing Institute have successfully thwarted the release of data describing SNAP purchases from 2005-2010, arguing the decade-old information would cause competitive harm. (Ellis 2018)

In light of these efforts, animal advocates should align themselves in the fight for greater transparency in the food system, not just in how animals are used for food but also in how marketers work to perpetuate demand for animal products. In addition to these efforts, researchers should work to build collaborations with independent and conscientious business partners to establish direct relationships for collecting dietary data. Lastly, we should keep an eye to technological developments that may aid our research and advocate for the development of tools well-suited to animal advocacy research.

Technical appendix

In Freedman et al. (2014 Web Appendix 3), true and self-reported diet for a particular study ($k$) are related using a random effects model

$$T_i={\lambda }_0+{\lambda }_1{Q}_i+u_i$$

where,

• $T_i$ is the unobserved true dietary consumption of individual $i$

• ${\lambda }_0$ is the intercept for all individuals

• $u_i$ is the random intercept for individual $i$, assumed independent and normally distributed with mean zero

• ${\lambda }_1$ is the regression coefficient termed the attenuation factor

• $Q_i$ is the observed self-reported consumption of individual $i$

The Pearson correlation coefficient for $T$ and $Q$ is then,

$${\rho }_{T,Q}=\frac{Cov[T,Q]}{\sqrt{{\sigma }_T^2{\sigma }_Q^2}},$$

where ${\sigma }_T^2$ and ${\sigma }_Q^2$ represent the variance of $T$ and $Q$ respectively and $Cov$ denotes the covariance. To calculate ${\sigma }_T^2$, using the properties of variance and the independence of $u$ and $Q$, we find:

$${\sigma }_T^2=Var\left[T\right]=Var[{\lambda }_0+{\lambda }_{1}Q+u]={\lambda }_1^2{\sigma }_Q^2+{\sigma }_u^2+2{\lambda }_{1}Cov[u,Q]={\lambda }_1^2{\sigma }_Q^2+{\sigma }_u^2.$$

To calculate $Cov[T,Q]$, using the properties of covariance and the independence of $u$ and $Q$, we find:

$$Cov[T,Q]=Cov[{\lambda }_0+{\lambda }_{1}Q+u,Q]={\lambda }_{1}Cov[Q,Q]+Cov[u,Q]={\lambda }_1{\sigma }_Q^2.$$

Substituting and rearranging the definition of Pearson correlation coefficient, we have the result presented in Web Appendix 3:

$${\rho }_{T,Q}={\lambda }_1\sqrt{\frac{{\sigma }_Q^2}{{\lambda }_1^2{\sigma }_Q^2+{\sigma }_u^2}}.$$

As we are given ${\lambda }_1$ and ${\rho }_{T,Q}$ (Tables 6 and 7, respectively), we can recover the ratio of the variance of self-reported and true consumption,

$${\left(\frac{{\rho }_{T,Q}}{{\lambda }_1}\right)}^2=\frac{{\sigma }_Q^2}{{\lambda }_1^2{\sigma }_Q^2+{\sigma }_u^2}=\frac{{\sigma }_Q^2}{{\sigma }_T^2},$$

and then the ratio of standard deviations,

$$\frac{{\rho }_{T,Q}}{{\lambda }_1}=\frac{{\sigma }_Q}{{\sigma }_T}.$$

To examine the effects on power, consider a $t$-test and the associated Cohen's $d$ effect size of the form $\Delta /s$, where $\Delta$ is the difference in means between the two treatments and $s$ is the shared standard deviation in both treatments. If we could measure consumption directly, we might detect an effect size $d_T=\Delta /{\sigma }_T$. However, if we used self-reports instead, we'd find an effect size, $d_Q=\Delta /{\sigma }_Q$. (This is ignoring issues of bias, which can be readily corrected, so we assume $\Delta$ would remain constant.) By dividing $d_T$ by ${\sigma }_Q/{\sigma }_T$, we can relate $d_Q$ and $d_T$:

$$\frac{d_T}{{\sigma }_Q/{\sigma }_T}=\frac{\Delta }{{\sigma }_T}\frac{{\sigma }_T}{{\sigma }_Q}=\frac{\Delta }{{\sigma }_Q}=d_Q.$$

Thus we can calculate the reduction in effect size resulting from using self-reported rather than direct measures of diet. For the Python code to perform these calculations, see the Open Science Framework repository at https://osf.io/8zqc3/.

Revision history

• November 12, 2018: Non-substantive copyediting.
• November 9, 2018: Non-substantive copyediting.

License

This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

References

America, Culinary Institute of, and Stanford University. 2018. "Menus of Change University Research Collaborative." The Menus of Change University Research Collaborative. 2018. http://www.moccollaborative.org/.

"ASA24 Demonstration Site." n.d. Accessed September 17, 2018. https://asa24.nci.nih.gov/demo/.

"ASA24® Evaluation & Validation." n.d. National Cancer Institute. Accessed August 22, 2018. https://epi.grants.cancer.gov/asa24/respondent/validation.html.

Brouwer-Brolsma, Elske M., Lorraine Brennan, Christian A. Drevon, Henk van Kranen, Claudine Manach, Lars Ove Dragsted, Helen M. Roche, et al. 2017. "Combining Traditional Dietary Assessment Methods with Novel Metabolomics Techniques: Present Efforts by the Food Biomarker Alliance." Proceedings of the Nutrition Society 76 (04): 619--27. https://doi.org/10.1017/S0029665117003949.

Cheung, William, Pekka Keski-Rahkonen, Nada Assi, Pietro Ferrari, Heinz Freisling, Sabina Rinaldi, Nadia Slimani, et al. 2017. "A Metabolomic Study of Biomarkers of Meat and Fish Intake." The American Journal of Clinical Nutrition 105 (3): 600--608. https://doi.org/10.3945/ajcn.116.146639.

"Dietary Assessment." 2016. Animal Charity Evaluators. December 2016. https://web.archive.org/web/20180813184439/https://animalcharityevaluators.org/for-charities/develop-a-survey/dietary-assessment/.

Dragsted, Lars O. 2010. "Biomarkers of Meat Intake and the Application of Nutrigenomics." Meat Science 84 (2): 301--7. https://doi.org/10.1016/j.meatsci.2009.08.028.

Ellis, Jonathan. 2018. "U.S. Supreme Court Justice Temporarily Delays Release of Food Stamp Data in Argus Leader Lawsuit." Argus Leader. August 7, 2018. https://www.argusleader.com/story/news/2018/08/07/argus-leader-food-retailers-ask-u-s-supreme-court-intervene-lawsuit-snap-benefits/927242002/.

"Food Frequency Questionnaires." n.d. Faunalytics. Accessed August 13, 2018. https://web.archive.org/web/20180813184347/https://faunalytics.org/ffq/.

Freedman, Laurence S., John M. Commins, James E. Moler, Lenore Arab, David J. Baer, Victor Kipnis, Douglas Midthune, et al. 2014. "Pooled Results from 5 Validation Studies of Dietary Self-Report Instruments Using Recovery Biomarkers for Energy and Protein Intake." American Journal of Epidemiology 180 (2): 172--88. https://doi.org/10.1093/aje/kwu116.

Gibbons, Helena, and Lorraine Brennan. 2016. "Metabolomics as a Tool in the Identification of Dietary Biomarkers." Proceedings of the Nutrition Society 76 (01): 42--53. https://doi.org/10.1017/S002966511600032X.

Hunter, Paul. 2018. "New Age Panels: The Next Gold Standard for Shopper Data?" CPG Data Tip Sheet. August 1, 2018. https://www.cpgdatainsights.com/.

Improving Data to Analyze Food and Nutrition Policies. 2005. Washington, D.C.: National Academies Press. https://doi.org/10.17226/11428.

Ji, Kyunghee, Young Lim Kho, Yoonsuk Park, and Kyungho Choi. 2010. "Influence of a Five-Day Vegetarian Diet on Urinary Levels of Antibiotics and Phthalate Metabolites: A Pilot Study with 'Temple Stay' Participants." Environmental Research 110 (4): 375--82. https://doi.org/10.1016/j.envres.2010.02.008.

"Join NCP Today." n.d. National Consumer Panel. Accessed August 22, 2018. https://join.ncponline.com/apex/rewards.

"Learn More About Biomarkers." n.d. National Cancer Institute. Accessed October 4, 2018. https://dietassessmentprimer.cancer.gov/learn/biomarkers.html.

Martin, Sally. 2014. "What's the Best Data Source? Retailer Direct or Syndicated Nielsen/IRI Data?" CPG Data Tip Sheet. October 27, 2014. https://www.cpgdatainsights.com/get-started-with-nielsen-iri/data-source-retailer-syndicated/.

---------. 2018a. "CPG Syndicated Data Markets: Part 1 -- the Big Picture." CPG Data Tip Sheet. April 24, 2018. https://www.cpgdatainsights.com/get-started-with-nielsen-iri/markets-part-1-the-big-picture/.

---------. 2018b. "CPG Syndicated Data Markets: Part 2 - the Nitty-Gritty." CPG Data Tip Sheet. May 29, 2018. https://www.cpgdatainsights.com/understand-your-database/markets-part-2-the-nitty-gritty/.

Muth, Mary K., Megan Sweitzer, Derick Brown, Kristen Capogrossi, Shawn Karns, David Levin, Abigail Okrent, Peter Siegel, and Chen Zhen. 2016. Understanding IRI Household-Based and Store-Based Scanner Data. United States Department of Agriculture, Economic Research Service.

Münger, Linda H, Mar Garcia-Aloy, Rosa Vázquez-Fresno, Doreen Gille, Albert Remus R Rosana, Anna Passerini, María-Trinidad Soria-Florido, et al. 2018. "Biomarker of Food Intake for Assessing the Consumption of Dairy and Egg Products." Genes & Nutrition 13 (1): 18. https://doi.org/10.1186/s12263-018-0615-5.

"National Consumer Panel." n.d. Accessed August 22, 2018. https://www.ncppanel.com/.

"Nielsen Datasets Consumer Panel Data." n.d. Chicago Booth. Accessed August 21, 2018. https://research.chicagobooth.edu/nielsen/datasets#simple1.

"Nielsen Datasets Retail Scanner Data." n.d. Chicago Booth. Accessed August 21, 2018. https://research.chicagobooth.edu/nielsen/datasets#simple2.

Raper, Nancy, Betty Perloff, Linda Ingwersen, Lois Steinfeldt, and Jaswinder Anand. 2004. "An Overview of USDA's Dietary Intake Data System." Journal of Food Composition and Analysis 17 (3-4): 545--55. https://doi.org/10.1016/j.jfca.2004.02.013.

Saksena, Michelle J, Abigail M Okrent, and Tobenna D Anekwe. 2018. "Food Away from Home." Economic Information Bulletin Number 196. United States Department of Agriculture. https://doi.org/10.1093/oxfordhb/9780199569441.013.0027.

Schmidt, Julie A, Sabina Rinaldi, Pietro Ferrari, Marion Carayol, David Achaintre, Augustin Scalbert, Amanda J Cross, et al. 2015. "Metabolic Profiles of Male Meat Eaters, Fish Eaters, Vegetarians, and Vegans from the EPIC-Oxford Cohort." The American Journal of Clinical Nutrition 102 (6): 1518--26. https://doi.org/10.3945/ajcn.115.111989.

Sethu, Harish. 2016. "The Unique Challenges of Research in Farm Animal Advocacy." Humane League Labs. May 12, 2016. http://www.humaneleaguelabs.org/blog/2016-05-12-the-unique-challenges-of-research-in-farm-animal-advocacy/.

Stuessy, Gina. 2017. "Study Proposals from Our First Research Workshop on Effective Animal Advocacy." Animal Charity Evaluators. October 24, 2017. https://animalcharityevaluators.org/blog/study-proposals-from-our-first-research-workshop-on-effective-animal-advocacy/.

Thompson, Frances E., and Amy F. Subar. 2013. "Dietary Assessment Methodology." In Nutrition in the Prevention and Treatment of Disease, 5--46. Elsevier. https://doi.org/10.1016/B978-0-12-391884-0.00001-9.

Turunen, A W, S Männistö, H Kiviranta, J Marniemi, A Jula, P Tiittanen, L Suominen-Taipale, T Vartiainen, and P K Verkasalo. 2010. "Dioxins, Polychlorinated Biphenyls, Methyl Mercury and Omega-3 Polyunsaturated Fatty Acids as Biomarkers of Fish Consumption." European Journal of Clinical Nutrition 64 (3): 313--23. https://doi.org/10.1038/ejcn.2009.147.