Fuchsian groups with a modular embedding have the richest arithmetic properties among non-arithmetic Fuchsian groups. But they are very rare, all known examples being related either to triangle groups or to Teichmüller curves. In Part I of this paper we study the arithmetic properties of the modular embedding and develop from scratch a theory of twisted modular forms for Fuchsian groups with a modular embedding, proving dimension formulas, coefficient growth estimates and differential equations. In Part II we provide a modular proof for an Apéry-like integrality statement for solutions of Picard–Fuchs equations. We illustrate the theory on a worked example, giving explicit Fourier expansions of twisted modular forms and the equation of a Teichmüller curve in a Hilbert modular surface. In Part III we show that genus two Teichmüller curves are cut out in Hilbert modular surfaces by a product of theta derivatives. We rederive most of the known properties of those Teichmüller curves from this viewpoint, without using the theory of flat surfaces. As a consequence we give the modular embeddings for all genus two Teichmüller curves and prove that the Fourier developments of their twisted modular forms are algebraic up to one transcendental scaling constant. Moreover, we prove that Bainbridge’s compactification of Hilbert modular surfaces is toroidal. The strategy to compactify can be expressed using continued fractions and resembles Hirzebruch’s in form, but every detail is different.