Color in natural images adds a perceptual dimension which may facilitate object identification. Here we investigate whether color in natural images leads to changes in visual memory performance in the primate and study the neural basis of this effect in extrastriate area V4 and dorsolateral prefrontal cortex (PF). We used a procedure based on Fourier analysis to create colored and achromatic natural images as well as control patterns that carried no task specific information. We employed a delayed matching to sample paradigm: a sample stimulus (250ms) was presented followed by a probe stimulus (1s) after a delay period (1500ms). A lever press was required if the sample stimulus matched the probe. Monkeys’ recognition performance was near ceiling for both chromatic and achromatic natural images, and at chance for both kinds of control patterns. We have single neuron responses from a total of 84 neurons in V4 and 62 neurons in PF recorded from two monkeys. In V4 the majority of visually responsive units showed greater selectivity for chromatic than achromatic natural images in both monkeys (paired t-test