The dorsolateral prefrontal cortex in human and non-human primates functions as the highest-order executor for the perception-action cycle. According to this view, when perceptual stimuli from the environment are novel or complex, the dorsolateral prefrontal cortex serves to set consciously a goal-directed scheme which broadly determines an action repertory to meet the particular demand from the environment. In this respect, the dorsolateral prefrontal cortex is a short-term activation device with the properties of a ‘cognitive switch’, because it couples a particular set of perceptual stimuli to a particular set of actions. Here, I suggest that, in order for the organism to react systematically to the environment, neural traces for the switch function must be stored in the brain. Thus, the highest-order, perception-action interface function of the dorsolateral prefrontal cortex per se depends on permanently stored neural traces in the dorsolateral prefrontal cortex and related structures. Such a memory system may be located functionally between two of the well-documented memory systems in the brain: the declarative memory system and the procedural memory system. Finally, based on available neurophysiological data, the possible mechanisms underlying the formation of cognitive switch traces are proposed.