Irritability and anxiety are two common clinical phenotypes that involve high-arousal negative affect states (anger and fear), and that frequently co-occur. Elucidating how these two forms of emotion dysregulation relate to perturbed neurodevelopment may benefit from alternate phenotyping strategies. One such strategy applies a bifactor latent variable approach that can parse shared versus unique mechanisms of these two phenotypes. Here, we aim to replicate and extend this approach and examine associations with neural structure in a large transdiagnostic sample of youth (N = 331; M = 13.57, SD = 2.69 years old; 45.92% male). FreeSurfer was used to extract cortical thickness, cortical surface area, and subcortical volume. The current findings replicated the bifactor model and demonstrate measurement invariance as a function of youth age and sex. There were no associations of youth's factor scores with cortical thickness, surface area, or subcortical volume. However, we found strong convergent and divergent validity between parent-reported irritability and anxiety factors with clinician-rated symptoms and impairment. A general negative affectivity factor was robustly associated with overall functional impairment across symptom domains. Together, these results support the utility of the bifactor model as an alternative phenotyping strategy for irritability and anxiety, which may aid in the development of targeted treatments.