The paper presents a theory of nonlinear evolution and secondary instabilities in Marangoni (surface-tension-driven) convection in a two-layer liquid–gas system with a deformable interface, heated from below. The theory takes into account the motion and convective heat transfer both in the liquid and in the gas layers. A system of nonlinear evolution equations is derived that describes a general case of slow long-scale evolution of a short-scale hexagonal Marangoni convection pattern near the onset of convection, coupled with a long-scale deformational Marangoni instability. Two cases are considered: (i) when interfacial deformations are negligible; and (ii) when they lead to a specific secondary instability of the hexagonal convection.

In case (i), the extent of the subcritical region of the hexagonal Marangoni convection, the type of the hexagonal convection cells, selection of convection patterns – hexagons, rolls and squares – and transitions between them are studied, and the effect of convection in the gas phase is also investigated. Theoretical predictions are compared with experimental observations.

In case (ii), the interaction between the short-scale hexagonal convection and the long-scale deformational instability, when both modes of Marangoni convection are excited, is studied. It is shown that the short-scale convection suppresses the deformational instability. The latter can appear as a secondary long-scale instability of the short-scale hexagonal convection pattern. This secondary instability is shown to be either monotonic or oscillatory, the latter leading to the excitation of deformational waves, propagating along the short-scale hexagonal convection pattern and modulating its amplitude.