We present Keck/HIRES spectra of six metal-poor stars in two of the ultra-faint dwarf galaxies orbiting the Milky Way, Ursa Major II and Coma Berenices, and a Magellan/MIKE spectrum of a star in the classical dwarf spheroidal galaxy (dSph) Sculptor. Our data include the first high-resolution spectroscopic observations of extremely metal-poor stars ([Fe/H] < −3.0) not belonging to the Milky Way (MW) stellar halo field population. We obtain abundance measurements and upper limits for up to 26 elements between carbon and europium. The stars span a range of −3.8 < [Fe/H] < −2.3, with the ultra-faints having large spreads in Fe. A comparison with MW halo stars of similar metallicity reveals substantial agreement between the abundance patterns of the ultra-faint dwarf galaxies and Sculptor and the MW halo for the light, α and iron-peak elements (C to Zn). This agreement contrasts with the results of earlier studies of more metal-rich stars (−2.5 ≲[Fe/H]≲ −1.0) in more luminous dwarfs, which found significant abundance discrepancies with respect to the MW halo data. The abundances of neutron-capture elements (Sr to Eu) in all three galaxies are extremely low, consistent with the most metal-poor halo stars, but not with the typical halo abundance pattern at [Fe/H]≳ −3.0. Our results are broadly consistent with a galaxy formation model which predicts that massive dwarf galaxies are the source of the metal-rich component ([Fe/H]≳ −2.5) of the MW inner halo, but we propose that dwarf galaxies similar to the dSphs are the primary contributors to the metal-poor end of the metallicity distribution of the MW outer halo.