A fundamental question in attention theory concerns the earliest processing stages that can be modulated by selective attention. A series of experiments is reported in which very early attention effects are found under specific conditions in the frequency-following potential (FFP), a brain stem response to low-frequency tone stimuli. In two experiments, stimuli of two different modalities were applied, and attention directed to one of the modalities. In two further experiments, only auditory stimuli were presented. In the first of these last two experiments, a dichotic paradigm with sustained attention to one ear was used, in the second a monotic paired-stimuli paradigm was used, in which the first stimulus served as reference for the second one. Only in the last experiment significant attention effects were found in the latency, but not in the amplitude of the FFP. The results show that a very early attention effect on the latency of the FFP can be demonstrated, but only under highly specific conditions. The size and preconditions of the attention effect suggest that it reflects subtle intramodal tuning mechanisms in the cochlea or in the lower brain stem.