Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010 to 2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, IV estimates are often highly uncertain, and the commonly used t-test for two-stage-least-squares (2SLS) estimates frequently underestimate the uncertainties. Third, in most replicated studies, 2SLS estimates are significantly larger in magnitude than ordinary-least-squares estimates, and their absolute ratio is inversely related to the strength of the instrument in observational studies—a pattern not observed in experimental ones—suggesting potential violations of unconfoundedness or the exclusion restriction in the former. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.

Small businesses employ more than half of the entire workforce, account for more than sixty percent of new jobs created in the United States, and are responsible for about fifty percent of private domestic gross product. It is noteworthy, however, that small business owners in credit markets, in particular minority owners, have difficulty in securing sources of capital for their business operation. The literature on credit market discrimination shows consistent results that can be interpreted as evidence that minority owners are discriminated against compared to their counterparts (i.e., White owners) in obtaining loans, which may be caused by lenders’ discrimination, although such behavior is prohibited under current fair-lending laws. This paper uses pooled cross-sectional data from the Survey of Small Business Finances (1993, 1998, and 2003) and a bivariate probit model based on James J. Heckman’s approach to deal with sample selection bias for those choosing to apply for loans. Those who didn’t apply for loans have been ignored in analyses of credit markets for small business owners. This paper adds to the small business lending market literature by 1) combining cross sectional data from the Survey of Small Business Finances (SSBF) for 1993, 1998, and 2003 to get more precise estimates and test statistics with more power; 2) conducting regression analyses with different model specifications to show the robustness of the empirical results; and 3) dealing directly with problems of sample selection based on Heckman’s approach with particular attention to the assumptions required to justify the identification of the effect (i.e., exclusion restrictions).

The analysis confirms previous results, suggesting that minority owners are discriminated against in credit markets. These conclusions are supported in a variety of model specifications.