In the European Union, at least 1% of farms are inspected every year and sanctions are applied to those that do not comply with the legislation on animal welfare. These on-farm inspections can result in measures to correct welfare problems detected. They can also highlight major risks that will require a focus of efforts and help prevent further non-compliances. Here, we analysed the reports from inspections of French cattle farms between 2010 and 2013 to check whether inspection stimulates improvement and to propose ways to improve how animal welfare legislation is implemented through the cross-compliance system. French inspectors use 32 items to assess overall compliance of farms inspected. We found that compliance improves on farms that are re-inspected but not in other farms (8% of severely non-compliant farms). Nine items do not influence the overall assessment whereas eight have a huge impact. The importance attributed to items varies from the first to the second visit to a farm. The major risks are absence of farm records, lack of basic care (practices or enclosures likely to harm animals, insufficient feeding) and inadequate skills (no veterinarian consulted, insufficient qualified staff). To improve compliance with EU animal welfare legislation and the efficiency of the inspection system, we suggest organising consultation between inspectors, ministry central services and welfare experts to: (i) refine the checklist and harmonise interpretations of item compliance; (ii) make sure all farmers are aware of the legislative requirements and the major risks of non-compliance; and (iii) define plans for a step-wise improvement of non-compliant farms.