We present an analysis of the parsec-scale jet structure of the quasar 4C+21.35 with a resolution of 0.1 milliarcseconds based on 63 epochs of Very Long Baseline Array observations at 43 GHz from 2007 June to 2014 May along with the Fermi LAT γ-ray light curve and multi-frequency optical photometric and polarimetric data. We find that the innermost jet of the quasar consists of a very compact core of size ~0.03 mas, as well as feature A1 located 0.16 ± 0.03 mas from the core. The distance of A1 remains fairly stable, but its position angle with respect to the core changes from -10 to +10 deg. We detect 4 superluminal knots in the inner jet with apparent speeds ranging from 10c to 20c. The first two components appeared in the jet during the high γ-ray state of the quasar from mid-2010 to early 2011, while the fourth knot appears to be connected with the γ-ray active state in late 2013 - early 2014. The first knot can be associated with the dramatic VHE flare in 2010 June and possesses an extreme Doppler factor ~60. We find that maxima in the γ-ray light curve coincide with epochs of interaction between the moving knots and the core and feature A1. This suggests that the core and A1 are recollimation shocks where γ-ray flares occur. The Chandra 0.5-6 keV image reveals the existence of X-ray emission in the kiloparsec scale jet of the quasar that can be explained via inverse Compton scattering off the cosmic microwave background by relativistic electrons if no deceleration occurs between the parsec- and kiloparsec-scale jets.