It is well known that X-ray luminosity (Lx) originating from high mass X-ray binaries (HMXBs) is tightly correlated with the host galaxy’s star formation rate (SFR). We explore this connection using a sample representative of the star-formation activity in the local Universe (Star-Formation Reference Survey; SFRS) along with a comprehensive set of star-formation (radio, FIR, 24μm, 8 μm, Hα, UV, SED fitting) and stellar mass (K-band, 3.6 μm, SED fitting) indicators, and Chandra observations. We investigate the Lx–SFR and Lx– stellar mass (M*) scaling relations down to sub-galactic scales of ∼lkpc2. This way we extend these relations to extremely low SFR (∼10−6M⊙.yr−1) and M* (∼104M⊙). We also quantify their scatter and their dependence on the age of the local stellar populations as inferred from the different age sensitive SFR indicators. These results are particularly important for setting the benchmark for the formation of X-ray binaries in vigorous, but low SFR objects such as galaxies in the early Universe.