The linear and nonlinear response tensors describe the response of a plasma to an electromagnetic field disturbance. A novel construction of the linear and quadratic response tensors for an unmagnetized collisionless plasma is presented here using a convenient 4-tensor notation. The construction uses the equations of charge continuity and gauge invariance, together with symmetry properties and dimensional arguments, to constrain the coefficients of an arbitrary linear combination of tensors. A new interpretation of the response tensors follows as expressions uniquely determined by fundamental physical constraints, and not simply as coefficients in a perturbation expansion of induced plasma response.