Recent experimental work suggests that the concept of contextual fields should be generalized to allow the modulation of local information extraction by both external and internal context. The external context relates to the coherent information of the stimulus; the internal context refers to the parts of this information which are relevant for behavior. This dual interaction, present at every level of the hierarchy, requires a fundamental unit of processing more complex than a single neuron appears today. We argue that the cortical column supplies the required mechanisms for a separate top-down and bottom-up processing and allows for interaction without destructive cross-talk. This flexibility might explain why the same basic architecture of a cortical column is used in the whole isocortex.