Using the Fiedler–Polyak–Viro Gauß diagram formulae we study the Vassiliev invariants of degree 2 and 3 on almost positive knots. As a consequence we show that the number of almost positive knots of a given genus or unknotting number grows polynomially in the crossing number, and also recover and extend, inter alia to their untwisted Whitehead doubles, previous results on the polynomials and signatures of such knots. In particular, we prove that there are no achiral almost positive knots and classify all almost positive diagrams of the unknot. We give an application to contact geometry (Legendrian knots) and property P.