The stability of continuously stratified vortices with large displacement of isopycnal surfaces on the f-plane is examined both analytically and numerically. Using an appropriate asymptotic set of equations, we demonstrated that sufficiently large vortices (i.e. those with small values of the Rossby number) are unstable. Remarkably, the growth rate of the unstable disturbance is a function of the spatial coordinates. At the same time, the corresponding boundary-value problem for normal modes has no smooth square-integrable solutions, which would normally be regarded as stability.

We conclude that (potentially) stable vortices can be found only among ageostrophic vortices. Since this assumption cannot be verified analytically due to complexity of the primitive equations, we verify it numerically for the particular case of two-layer stratification.