It is widely accepted that human color vision is based on two types of cone-opponent mechanism, one differencing L and M cone types (loosely termed “red–green”), and the other differencing S with the L and M cones (loosely termed “blue–yellow”). The traditional view of the early processing of human color vision suggests that each of these cone-opponent mechanisms respond in a bipolar fashion to signal two opponent colors (red vs. green, blue vs. yellow). An alternative possibility is that each cone-opponent response, as well as the luminance response, is rectified, so producing separable signals for each pole (red, green, blue, yellow, light, and dark). In this study, we use psychophysical noise masking to determine whether the rectified model applies to detection by the postreceptoral mechanisms. We measured the contrast-detection thresholds of six test stimuli (red, green, blue, yellow, light, and dark), corresponding to the two poles of each of the three postreceptoral mechanisms. For each test, we determined whether noise presented to the cross pole had the same masking effect as noise presented to the same pole (e.g. comparing masking of luminance increments by luminance decrement noise (cross pole) and luminance increment noise (same pole)). To avoid stimulus cancellation, the test and mask were presented asynchronously in a “sandwich” arrangement (mask-test-mask). For the six test stimuli, we observed that noise masks presented to the cross pole did not raise the detection thresholds of the test, whereas noise presented to the same pole produced a substantial masking. This result suggests that each color signal (red, green, blue, and yellow) and luminance signal (light and dark) is subserved by a separable mechanism. We suggest that the cone-opponent and luminance mechanisms have similar physiological bases, since a functional separation of the processing of cone increments and cone decrements could underlie both the separation of the luminance system into ON and OFF pathways as well as the splitting of the cone-opponent mechanisms into separable color poles.