This paper investigates the implications and consequences of choosing special gauges or gauge-invariant or non-gauge-invariant approximations in the action integral for the variational formulation of theories involving electromagnetic fields. After some interesting special gauges are considered, it is shown that non-gauge-invariant approximations always lead to inconsistent Euler–Lagrange equations. As a concrete example, Maxwell–Vlasov theory is investigated. The special non-gauge-invariant case considered, which is sometimes used in the literature, is obtained by replacing the contribution of the electric field to the Maxwell part of the Lagrangian density by the contribution of the gradient of the scalar potential alone. The detailed investigation concerns the local energy conservation law, and it is shown that the law thus derived is incorrect since it contains non-physical, spurious terms. An improvement of this situation can be obtained by the introduction of a Lagrange multiplier in the non-gauge-invariant theories in order to avoid inconsistencies in the local charge conservation law. The results are also valid for drift-kinetic and gyrokinetic theories, and for other theories, e.g. two-fluid theories.