Previous studies have suggested that maternal active smoking can increase the risk of birth defects, but evidence on second-hand tobacco smoke (SHS) is limited. We aimed to assess the association between maternal exposure to SHS and birth defects in a Chinese population. The data were based on a large-scale cross-sectional survey conducted in Shaanxi Province, China. Considering the characteristics of survey design and the potential impact of confounding factors, we adopted propensity score matching (PSM) to match the SHS exposure group and the non-exposure group to attain a balance of the confounders between the two groups. Subsequently, conditional logistic regression was employed to estimate the effect of SHS exposure on birth defects. Furthermore, sensitivity analyses were conducted to verify the key findings. After nearest neighbor matching of PSM with a ratio of 2 and a caliper width of 0.03, there were 6,205 and 12,410 participants in the exposure and control group, respectively. Pregnant women exposed to SHS were estimated to be 58% more likely to have infants with overall birth defects (OR = 1.58, 95% CI: 1.30–1.91) and 75% more likely to have infants with circulatory system defects (OR = 1.75, 95% CI: 1.26–2.44). We also observed that the risk effect of overall birth defects had an increasing trend as the frequency of exposure increased. Additionally, sensitivity analyses suggested that our results had good robustness. These results indicate that maternal exposure to SHS likely increases the risk of overall birth defects, especially circulatory system defects, in Chinese offspring.