The ANNALS of the American Academy of Political and Social Science

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here to watch the video

Click here to sign up for SAGE Journal Email Alerts today!

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Glazerman, S.
Right arrow Articles by Myers, D.
Right arrow Search for Related Content
Social Bookmarking
 Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
The ANNALS of the American Academy of Political and Social Science, Vol. 589, No. 1, 63-93 (2003)
DOI: 10.1177/0002716203254879

Nonexperimental Versus Experimental Estimates of Earnings Impacts

Steven Glazerman

Mathematica Policy Research

Dan M. Levy

Mathematica Policy Research, U.S. Centers for Disease Control and Prevention

David Myers

Mathematica Policy Research

To assess nonexperimental (NX) evaluation methods in the context of welfare, job training, and employment services programs, the authors reexamined the results of twelve case studies intended to replicate impact estimates from an experimental evaluation by using NX methods. They found that the NX methods sometimes came close to replicating experimentally derived results but often produced estimates that differed by policy-relevant margins, which the authors interpret as estimates of bias. Although the authors identified several study design factors associated with smaller discrepancies, no combination of factors would consistently eliminate discrepancies. Even with a large number of impact estimates, the positive and negative bias estimates did not always cancel each other out. Thus, it was difficult to identify an aggregation strategy that consistently removed bias while answering a focused question about earnings impacts of a program. They conclude that although the empirical evidence from this literature can be used in the context of training and welfare programs to improve NX research designs, it cannot on its own justify the use of such designs.

Key Words: nonexperimental • quasi-experimental • design replication


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?


This article has been cited by other articles:


Home page
Eval RevHome page
D. A. Freedman and R. A. Berk
Weighting Regressions by Propensity Scores
Eval Rev, August 1, 2008; 32(4): 392 - 409.
[Abstract] [PDF]


Home page
EDUCATIONAL EVALUATION AND POLICY ANALYSISHome page
K. A. Frank, G. Sykes, D. Anagnostopoulos, M. Cannata, L. Chard, A. Krause, and R. McCrory
Does NBPTS Certification Affect the Number of Colleagues a Teacher Helps With Instructional Matters?
Educational Evaluation and Policy Analysis, March 1, 2008; 30(1): 3 - 30.
[Abstract] [Full Text] [PDF]


Home page
POLIT ANALHome page
D. E. Ho, K. Imai, G. King, and E. A. Stuart
Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
Political Analysis, July 1, 2007; 15(3): 199 - 236.
[Abstract] [Full Text] [PDF]


Home page
POLIT ANALHome page
K. Arceneaux, A. S. Gerber, and D. P. Green
Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment
Political Analysis, January 1, 2006; 14(1): 37 - 62.
[Abstract] [Full Text] [PDF]


Home page
Eval RevHome page
J. K. Luellen, W. R. Shadish, and M. H. Clark
Propensity Scores: An Introduction and Experimental Test
Eval Rev, December 1, 2005; 29(6): 530 - 558.
[Abstract] [PDF]


Home page
The ANNALS of the American Academy of Political and Social ScienceHome page
L. W. Sherman and H. Strang
Experimental Ethnography: The Marriage of Qualitative and Quantitative Research
The ANNALS of the American Academy of Political and Social Science, September 1, 2004; 595(1): 204 - 222.
[Abstract] [PDF]


Home page
American Behavioral ScientistHome page
L. W. Sherman and H. Strang
Verdicts or Inventions?: Interpreting Results from Randomized Controlled Experiments in Criminology
American Behavioral Scientist, January 1, 2004; 47(5): 575 - 607.
[Abstract] [PDF]