In this paper we investigate the relationship between the existence of parallel semi-Riemannian metrics of a connection and the reducibility of the associated holonomy group. The question as to whether the holonomy group necessarily reduces in the presence of a specified number of independent parallel semi-Riemannian metrics is completely determined by the the signature of the metrics and the dimension d of the manifold, when d ≠ 4. In particular, the existence of two independent, parallel semi-Riemannian metrics, one of which having signature (p,q) with p ≠ q, implies the holonomy group is reducible. The (p,p) cases, however, may allow for more than one parallel metric and yet an irreducible holonomy group: for n = 2m, m ≥ 3, there exist connections on Rn with irreducible infinitesimal holonomy and which have two independent, parallel metrics of signature (m,m). The case of four-dimensional manifolds, however, depends on the topology of the manifold in question: the presence of three parallel metrics always implies reducibility but reducibility in the case of two metrics of signature (2,2) is guaranteed only for simply connected manifolds. The main theorem in the paper is the construction of a topologically non-trivial four-dimensional manifold with a connection that admits two independent metrics of signature (2,2) and yet has irreducible holonomy. We provide a complete solution to the general problem.