Despite significant atomic-scale heterogeneity, bulk metallic glasses well below their glass transition temperature exhibit a surprisingly robust elastic regime and a sharp elastic-to-plastic transition with a yield stress that depends approximately linearly on temperature. The present work attempts to understand these features within the framework of thermally activated plasticity. The presented statistical thermal activation model, in which the number of available structural transformations scales exponentially with system size, results in two distinct temperature regimes of deformation. At temperatures close to the glass transition temperature thermally activated Newtonian plastic flow emerges, whilst at lower temperatures the deformation properties fundamentally change due to the eventual kinetic freezing of the available structural transformations. In this regime, a linear temperature dependence emerges for the stress which characterises the elastic to plastic transition. For both regimes the transition to macroscopic plastic flow corresponds to a transition from a barrier energy dominated to a barrier entropy dominated statistics. The work concludes by discussing the possible influence that kinetic freezing might have on the low temperature heterogeneous and high temperature homogeneous plasticity of bulk metallic glasses.