Methodology is viewed by some to have the same type of draw as calisthenics. Yet, it must be done! In this section resides some of my work in this area. I have spent time thinking about whether laboratory behavior is a good indicator of behavior in the field, and whether, and to what extent, markets and/or market experience affect the size and extent of behavioral anomalies observed in lab experiments. In some cases, the increase in knowledge due to field experiments has overturned verdicts of yesterday. In others, the findings have reinforced the literature that has drawn inference from either lab or naturally-occurring data.
What Can We Learn From Experiments? Understanding the Threats to the Scalability of Experimental Results
Al-Ubaydli, Omar, John A. List, Dana L Suskind
American Economic Review, (P&P), forthcoming
No Abstract Available
An Economic Approach to Alleviate the Crisis of Confidence in Science: With an Application to the Public Goods Game
Butera, Luigi, John A. List
Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to independently replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer- reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.
Multiple Hypothesis Testing in Experimental Economics
List, John A., Azeem M. Shaikh and Yang Xu
NBER Working Paper No. 21875
Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives—multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogeneous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate – the probability of one false rejection – and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.
Using Field Experiments in Accounting and Finance
Floyd, Eric and John A. List
Journal of Accounting Research (2016)
The gold standard in the sciences is uncovering causal relationships. A growing literature in economics utilizes field experiments as a methodology to establish causality between variables. Taking lessons from the economics literature, this study provides an “A-to-Z” description of how to conduct field experiments in accounting and finance. We begin by providing a user’s guide into what a field experiment is, what behavioral parameters field experiments identify, and how to efficiently generate and analyze experimental data. We then provide a discussion of extant field experiments that touch on important issues in accounting and finance, and we also review areas that have ample opportunities for future field experimental explorations. We conclude that the time is ripe for field experimentation to deepen our understanding of important issues in accounting and finance.
Field Experiments in the Developed World: An Introduction
List, John A., Robert Metcalfe
Oxford Review of Economic Policy (2015) 30(4), 585-596
Field experiments represent a relatively new area in economics to understand the causal links from one variable to another. They have been used by academics to help answer interesting and policy-relevant questions in the developed world relating to educational attainment, tax avoidance, consumer finance, negative externalities, charitable giving, and labour market contracts. In this paper we bring together the key ideas behind the different variants of field experiments, how field experiments have been used to test theory, their limitations, and the new areas currently being opened up by field experiments.
Markets for Replication
Brandon, Alec and John A. List
Proceedings of the National Academy of Science, (2015), 112(50), 15267-15268.
Since 1955, The Journal of Irreproducible Results has offered a satirical view on academic research, publishing pieces such as, “A double blind efficacy trial of placebos, extra strength placebos, and generic placebos” (1). Scientists are now taking a less satirical look at the reproducibility of results, with questions emerging about the robustness of findings in fields as diverse as epidemiology, genetics, neuroscience, and the social sciences (2-4). The aftermath has seen myriad efforts to close the gap in reproducibility.
The Curious Relation Between Theory of Mind and Sharing in Preschool Age Children
Cowell, Jason, Anya Samek, John A. List and Jean Decety
PLoS One. 2015 Feb 6;10(2):e0117947. doi: 10.1371/journal.pone.0117947. eCollection 2015
Young children have long been known to act selfishly and gradually appear to become more generous across middle childhood. While this apparent change has been well documented, the underlying mechanisms supporting this remain unclear. The current study examined the role of early theory of mind and executive functioning in facilitating sharing in a large sample (N = 98) of preschoolers. Results reveal a curious relation between early false-belief understanding and sharing behavior. Contrary to many commonsense notions and predominant theories, competence in this ability is actually related to less sharing. Thus, the relation between developing theory of mind and sharing may not be as straightforward as it seems in preschool age children. It is precisely the children who can engage in theory of mind that decide to share less with others.
Do Natural Field Experiments Afford Researchers More or Less Control than Laboratory Experiments? A Simple Model
Al-Ubaydli, Omar and John A. List
NBER Working Paper No. 20877
A commonly held view is that laboratory experiments provide researchers with more “control” than natural field experiments, and that this advantage is to be balanced against the disadvantage that laboratory experiments are less generalizable. This paper presents a simple model that explores circumstances under which natural field experiments provide researchers with more control than laboratory experiments afford. This stems from the covertness of natural field experiments: laboratory experiments provide researchers with a high degree of control in the environment which participants agree to be experimental subjects. When participants systematically opt out of laboratory experiments, the researcher’s ability to manipulate certain variables is limited. In contrast, natural field experiments bypass the participation decision altogether and allow for a potentially more diverse participant pool within the market of interest. We show one particular case where such selection is invaluable: when treatment effects interact with participant characteristics.
Estimating Individual Ambiguity Aversion: A Simple Approach
Gneezy, Uri, Alex Imas and John A. List
NBER Working Paper No. 20982
We introduce a simple, easy to implement instrument for jointly eliciting risk and ambiguity attitudes. Using this instrument, we structurally estimate a two-parameter model of preferences. Our findings indicate that ambiguity aversion is significantly overstated when risk neutrality is assumed. This highlights the interplay between risk and ambiguity attitudes as well as the importance of joint estimation. In addition, over our stakes levels we find no difference in the estimated parameters when incentives are real or hypothetical, raising the possibility that a simple hypothetical question can provide insights into an individuals preferences over ambiguity in such economic environments.
How to Make Experimental Economics Research More Reproducible: Lessons from Other Disciplines and a New Proposal
Maniadis, Zacharias, Fabio Tufano, and John A. List
In: CARY A. DECK, ENRIQUE FATAS and TANYA ROSENBLAT, eds., Replication in Experimental Economics 18. Emerald. 215-230
Efforts in the spirit of this special issue aim at improving the reproducibility of experimental economics, in response to the recent discussions regarding the “research reproducibility crisis.” We put this endeavor in perspective by summarizing the main ways (to our knowledge) that have been proposed – by researchers from several disciplines – to alleviate the problem. We discuss the scope for economic theory to contribute to evaluating the proposals. We argue that a potential key impediment to replication is the expectation of negative reactions by the authors of the individual study, and suggest that incentives for having one’s work replicated should increase.
Using Field Experiments to Change the Template of How We Teach Economics
List, John A
Journal of Economic Education, (2014), 45(2), pp. 81-89
In this article, the author explains why field experiments can improve what we teach and how we teach economics. Economists no longer operate as passive observers of economic phenomena. Instead, they participate actively in the research process by collecting data from field experiments to investigate the economics of everyday life. This change can be shown to students by presenting them with evidence from field experiments. Field experiments related to factor markets, behavioral economics, and discrimination are presented to explain how this approach works across different economic content. The three questions that are highlighted are the following: (1) Why do women get paid less than men in labor markets? (2) How can we use behavioral economics to motivate teachers? (3) What seven words can end third-degree price discrimination?
One Swallow Does not Make a Summer: New Evidence on Anchoring Effects
Maniadis, Zacharias, Fabio Tufano, and John A. List
American Economic Review, (2014), 104 (1), pp. 277-290
Some researchers have argued that anchoring in economic valuations casts doubt on the assumption of consistent and stable preferences. We present new evidence that explores the strength of certain anchoring results. We then present a theoretical framework that provides insights into why we should be cautious of initial empirical findings in general. The model importantly highlights that the rate of false positives depends not only on the observed significance level, but also on statistical power, research priors, and the number of scholars exploring the question. Importantly, a few independent replications dramatically increase the chances that the original finding is true.
On the Generalizability of Experimental Results in Economics: With A Response To Camerer
Al-Ubaydli, Omar and John A. List
In Frechette, G. & Schotter, A., Methods of Modern Experimental Economics, Oxford University Press, 2013
Economists are increasingly turning to the experimental method as a means to estimate causal effects. By using randomization to identify key treatment effects, theories previously viewed as untestable are now scrutinized, efficacy of public policies are now more easily verified, and stakeholders can swiftly add empirical evidence to aid their decision-making. This study provides an overview of experimental methods in economics, with a special focus on developing an economic theory of generalizability. Given that field experiments are in their infancy, our secondary focus pertains to a discussion of the various parameters that they identify, and how they add to scientific knowledge. We conclude that until we conduct more field experiments that build a bridge between the lab and the naturally-occurring settings of interest we cannot begin to make strong conclusions empirically on the crucial question of generalizability from the lab to the field.
Field Experiments in Labor Economics
List, John A. and Imran Rasul
Chapter 2 in Handbook of Labor Economics Volume 4a, O. Ashenfelter and D. Card (editors), Elsevier, 2011, pp104-228
We overview the use of field experiments in labor economics. We showcase studies that highlight the central advantages of this methodology, which include: (i) using economic theory to design the null and alternative hypotheses; (ii) engineering exogenous variation in real world economic environments to establish causal relations and learning about the underlying mechanisms; and (iii) engaging in primary data collection and often working closely with practitioners. To highlight the potential for field experiments to inform issues in labor economics, we organize our discussion around the individual life cycle. We therefore consider field experiments related to the accumulation of human capital, the demand and supply of labor, behavior within firms, and close with a brief discussion of the nascent literature of field experiments related to household decision-making.
Why Economists Should Conduct Field Experiments and 14 Tips for Pulling One Off
John A. List
Journal of Economic Perspectives,(2011), 25(3), pp. 3-16
In this introduction to the symposium, I first offer an overview of the spectrum of experimental methods in economics, from laboratory experiments to the field experiments that are the subject of this symposium. I then offer some thoughts about the potential gains from doing economic research using field experiments and my own mental checklist of 14 steps to improve the chances of carrying out an economics field experiment successfully.
Was There Really a Hawthorne Effect at the Hawthorne Plant? An Analysis of the Original Illumination Experiments
Steven D. Levitt and John A. List
American Economic Journal: Applied Economics 3 (January 2011): 224–238
The “Hawthorne effect” draws its name from a landmark set of studies conducted at the Hawthorne plant in the 1920s. The data from the first and most influential of these studies, the “Illumination Experiment,” were never formally analyzed and were thought to have been destroyed. Our research has uncovered these data. Existing descriptions of supposedly remarkable data patterns prove to be entirely fictional. We do find more subtle manifestations of possible Hawthorne effects. We also propose a new means of testing for Hawthorne effects based on excess responsiveness to experimenter- induced variations relative to naturally occurring variation.
The Nature of Excess: Using Randomized Treatments to Investigate Price Dynamics
Al-Ubaydli, Omar, Michael K. Price and John A. List
NBER Working Paper No. 16319
This study explores empirically the price dynamics within two distinct market institutions – a double oral auction, which resembles modern asset markets, and a bilateral exchange market, which represents markets that have existed for centuries. To provide a theoretical basis to our investigation, we test and compare the excess supply model (Walras (1874, 1877, 1889, 1896)) and the excess rent model (Smith (1962, 1965)) in both market institutions. Our approach is unique in that we make use of appropriate demand and supply systems coupled with randomization of the main treatment variable to discriminate between the theories. All previous efforts, including Smith’s (1965) seminal experiments, use designs that cannot appropriately parse the models. We report several insights, perhaps most importantly, we consistently reject the Walrasian model in favor of the excess rent model, regardless of market institution. This finding has important implications both positively and normatively.
The IRB Is Key in Field Experiments
List, John A
Science, (2009), 323(5915), pp. 713-714.
As my letter suggests Institutional Review Boards (IRBs) are a necessary condition and serve an invaluable role; I wrote, ‘Local Data sharing. Shared data will only be useful if complete information about the original study design is also available. Research Ethics Committees and Institutional Review Boards in the United States serve an important role in monitoring such activi- ties.’ All of my research has IRB approval, and I suspect from W. R. Lovallo’s concerns that he and I agree on all aspects of subject approval. In my own research, I have been even more stringent than IRB require- ments—I do not deceive subjects and always ensure that they are better off due to my experiment. These conditions are certainly not a constraint in all, or even many, IRBs that I am aware of.
Introduction to field experiments in economics
List, John A.
Journal of Economic Behavior and Organization, (2009), 70(3), pp. 439-442.
The major empirical challenge for economists is going beyond correlational analysis to provide insights on causation. While economics has been served well by using precise models and econometric techniques for answering causal questions using variations in naturally occurring data, an important recent development to provide insights on causation is the expand- ing use of controlled laboratory experimentation. Under this approach, identification of causation is typically achieved via randomization. In this manner, randomization is an instrumental variable that is exogenous by definition.
Homo experimentalis Evolves
List, John A.
Science, (2008), 321(5886), pp. 207-208.
No Abstract Available
Homo economicus Evolves
Levitt, Steven D. and John A. List
Science, (2008), 319(5865), pp. 909-910.
The discipline of economics is built on the shoulders of the mythical species Homo economicus. Unlike his uncle, Homo sapiens, H. economicus is unswervingly rational, completely selfish, and can effortlessly solve even the most difficult optimization problems. This rational paradigm has served economics well, providing a coherent framework for modeling human behavior. However, a small but vocal movement in economics has sought to dethrone H. economicus, replacing him with someone who acts “more human.” This insurgent branch, commonly referred to as behavioral economics, argues that actual human behavior deviates from the rational model in predictable ways. Incorporating these features into economic models, proponents argue, should improve our ability to explain observed behavior.
So you want to run an experiment, now what? Some Simple Rules of Thumb for Optimal Experimental Design
List, J.A., S. Sadoff, M. Wagner
NBER working paper 15701 (2008)
Experimental economics represents a strong growth industry. In the past several decades the method has expanded beyond intellectual curiousity, now meriting consideration alongside the other more traditional empirical approaches used in economics. Accompanying this growth is an influx of new experimenters who are in need of straightforward direction to make their designs more powerful. This study provides several simple rules of thumb that researchers can apply to improve the efficiency of their experimental designs. We buttress these points by including empirical examples from the literature.
List, John A. and David Reiley
The New Palgrave Dictionary of Economics, Steven N. Durlauf and Lawrence E. Blume, eds., Palgrave Macmillan Publishing, (2008).
Field experiments provide a meeting ground between these two broad approaches to empirical economic science. By examining the nature of field experiments, we seek to make it a common ground between researchers.
Naturally Occurring Markets and Exogenous Laboratory Experiments: A Case Study of the Winner’s Curse
Harrison, Glenn W. and John A. List
Economic Journal, (2008), 118(528), pp. 822-843.
We examine the relevance of experimental findings from laboratory settings that abstract from the field context of the task that theory purports to explain. Using common value auction theory as our guide, we identify naturally occurring settings in which one can test the theory. Experienced agents bidding in familiar roles do not fall prey to the winner’s curse. Yet, experienced agents fall prey to the winner’s curse when bidding in an unfamiliar role. We conclude that the theory predicts field behaviour well when one is able to identify naturally occurring field counterparts to the key theoretical conditions.
Introduction to field experiments in economics with applications to the economics of charity
List, John A.
Experimental Economics, (2008), 11(3), pp. 203-212.
This special issue highlights an empirical approach that has increasingly grown in prominence in the last decade—field experiments. While field experiments can be used quite generally in economics—to test theories’ predictions, to measure key parameters, and to provide insights into the generalizability of empirical results— this special issue focuses on using field experiments to explore questions within the economics of charity. The issue contains six distinct field experimental studies that investigate various aspects associated with the economics of charitable giving. The issue also includes a fitting tribute to one of the earliest experimenters to depart from traditional lab methods, Peter Bohm, who curiously has not received deep credit or broad acclaim. Hopefully this issue will begin to rectify this oversight.
Informed Consent in Social Science
List, John A.
Science, (2008), 322(5902), p. 672.
As my perspective made clear, there are several types of field experiments. In some, subjects are made aware that they are taking part in an experiment and sign consent forms in the spirit of the guidelines of the Nuremberg code. There are, however, certain cases in which adhering to rigid ethical rules can affect the very issue that is being studied, such that it becomes quite difficult to conduct the research (1, 2). For example, if one were interested in exploring whether, and to what extent, race or gender influences the prices that buy- ers pay for used cars, it would be difficult to measure accurately the degree of discrimination among used car dealers who know that they are taking part in an experiment.
Viewpoint: On the generalizability of lab behaviour to the field
Levitt, Steven D. and John A. List
Canadian Journal of Economics, (2007), 40(2), pp. 347-370.
We can think of no question more fundamental to experimental economics than understanding whether, and under what circumstances, laboratory results generalize to naturally occurring environments. In this paper, we extend Levitt and List (2006) to the class of games in which financial payoffs and ‘doing the right thing’ are not necessarily in conflict. We argue that behaviour is crucially linked to not only the preferences of people, but also the properties of the situation. By doing so, we are able to provide a road map of the psychological and economic properties of people and situations that might interfere with generalizability of laboratory result from a broad class of games.
What do Laboratory Experiments Measuring Social Preferences Reveal About the Real World
Levitt, Steven D. and John A. List
Journal of Economic Perspectives, (2007), 21(2), pp. 153-174.
Economists have increasingly turned to the experimental model of the physical sciences as a method to understand human behavior. Peer- reviewed articles using the methodology of experimental economics were almost nonexistent until the mid-1960s and surpassed 50 annually for the first time in 1982; and by 1998, the number of experimental papers published per year exceeded 200 (Holt, 2006). Lab experiments allow the investigator to influence the set of prices, budget sets, information sets, and actions available to actors, and thus measure the impact of these factors on behavior within the context of the labora- tory. The allure of the laboratory experimental method in economics is that, in principle, it provides ceteris paribus observations of individual economic agents, which are otherwise difficult to obtain.
On the Interpretation of Giving in Dictator Games
List, John A.
Journal of Political Economy, (2007), 115(3), pp. 482-494.
The dictator game represents a workhorse within experimental economics, frequently used to test theory and to provide insights into the prevalence of social preferences. This study explores more closely the dictator game and the literature’s preferred interpretation of its meaning by collecting data from nearly 200 dictators across treatments that varied the action set and the origin of endowment. The action set variation includes choices in which the dictator can ‘take’ money from the other player. Empirical results question the received interpretation of dictator game giving: many fewer agents are willing to transfer money when the action set includes taking. Yet, a result that holds regardless of action set composition is that agents do not ubiquitously choose the most selfish outcome. The results have implications for theoretical models of social preferences, highlight that ‘institutions’ matter a great deal, and point to useful avenues for future research using simple dictator games and relevant manipulations.
Field Experiments: A Bridge between Lab and Naturally Occurring Data
List, John A.
The B.E. Journal of Economic Analysis & Policy, (2006), 6(2 – Advances), Article 8.
Laboratory experiments have been used extensively in economics in the past several decades to lend both positive and normative insights into a myriad of important economic issues. This study discusses a related approach that has increasingly grown in prominence of late-field experiments. I argue that field experiments serve as a useful bridge between data generated in the lab and empirical studies using naturally-occurring data. In discussing this relationship, I highlight that field experiments can yield important insights into economic theory and provide useful guidance to policymakers. I also draw attention to an important methodological contribution of field exper- iments: they provide an empirical account of behavioral principles that are shared across different domains. In this regard, at odds with conventional wisdom, I argue that representativeness of the environment, rather than representative of the sampled population, is the most crucial variable in determining generalizability of results for a large class of experimental laboratory games.
Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments
Gneezy, Uri and John A. List
Econometrica, (2006), 74(5), pp. 1365- 1384.
Recent discoveries in behavioral economics have led scholars to question the under-pinnings of neoclassical economics. We use insights gained from one of the most influential lines of behavioral research-gift exchange-in an attempt to maximize worker effort in two quite distinct tasks: data entry for a university library and door-to-door fundraising for a research center. In support of the received literature, our field evidence suggests that worker effort in the first few hours on the job is considerably higher in the ‘gift’ treatment than in the ‘nongift’ treatment. After the initial few hours, however, no difference in outcomes is observed, and overall the gift treatment yielded inferior aggregate outcomes for the employer: with the same budget we would have logged more data for our library and raised more money for our research center by using the market-clearing wager at her than by trying to induce greater effort with a gift of higher wages.
Experimental approaches to public economics: guest editors’ introduction
Andreoni, James, and John A. List
Journal of Public Economics, (2005), 89(8), pp. 1355-1359.
Standard applications of utility theory assume that utility depends solely on outcomes and not on causes. This study uses a field experiment conducted in the Netherlands to determine if alternative causes of an environmental problem affect willingness to pay to ameliorate it. We find evidence supporting the hypothesis that people are willing to pay significantly more to correct problems caused by humans than by nature (the ‘outrage effect’), but find no support for the hypothesis that ‘moral responsibility’ matters. We also find support for the hypothesis that stated willingness to pay values obtained via ‘cheap talk’ and ‘consequential’ treatments are lower than without inclusion of these protocols.
Young, Selfish, and Male: Field Evidence of Social Preferences
List, John A.,
Economic Journal, (2004), 114(492), pp. 121-149.
This study examines social preferences in three distinct field environments. In the first field setting, I allow consumers of all age and education levels to participate in one-shot and multiple-shot public goods games in a well-functioning marketplace. The second field study, an actual university capital campaign, gathers data from mail solicitations sent to 2,000 Central Florida residents. In the third field experiment, I examine data from an uncontrolled environment, a television game show, which closely resembles the classic prisoner’s dilemma game. Several insights emerge; perhaps the most provocative is that age and social preferences appear linked.
The Hidden Costs and Returns of Incentives – Trust and Trustworthiness among CEOs
Fehr, Ernst and List, John A.
Journal of the European Economic Association, (2004), 2(5), pp. 743-771.
We examine experimentally how Chief Executive Officers (CEOs) respond to incentives and how they provide incentives in situations requiring trust and trustworthiness. As a control we compare the behavior of CEOs with the behavior of students. We find that CEOs are considerably more trusting and exhibit more trustworthiness than students thus reaching substantially higher efficiency levels than students. Moreover we find that, for CEOs as well as for students, incentives based on explicit threats to penalize shirking backfire by inducing less trustworthy behavior, giving rise to hidden costs of incentives. However, the availability of penalizing incentives also creates hidden returns: if a principal expresses trust by voluntarily refraining from implementing the punishment threat, the agent exhibits significantly more trustworthiness than if the punishment threat is not available. Thus trust seems to reinforce trustworthy behavior. Overall, trustworthiness is highest if the threat to punish is available but not used, while it is lowest if the threat to punish is used. Paradoxically, however, most CEOs and students use the threat, CEOs use it less.
Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards
List, John A.
American Economic Review, (2001), 91(5), pp. 1498-1507.
The goal of environmental policy is to protect the well-being of humans and the ecosystems vital to human existence. Because benefit-cost analyses are now required at the federal level, and increasingly at the state level, much attention has been paid to the development of practical, credible approaches for estimating the benefits and costs of environmental programs. Although policy makers have a good handle on measuring the explicit costs associated with increased environmental protection, at present several disparate approaches are utilized to measure economic values of environmental goods and services. Arguably the most contentious of these approaches is the Contingent Valuation Method (CVM), which allows the researcher to measure the total value of the commodity in question (see Peter A. Diamond and Jerry A. Hausman’s  critical review). Chief amongst these concerns is whether hypothetical bias is inherent in CVM responses.