On a complex curve, we establish a correspondence between integrable connections with irregular singularities, and Higgs bundles such that the Higgs field is meromorphic with poles of any order. Moduli spaces of these objects are obtained with fixed generic polar parts at each singularity, which amounts to fixing a coadjoint orbit of the group $GL_r(\mathbb{C}[z]/z^n)$. We prove that they carry complete hyper-Kähler metrics.