Categorization of faces is fundamental for social interactions of primates. To understand its neural basis, we investigate how human and monkey face categories are represented in the monkey brain. As stimuli, we use realistic three dimensional morphed faces that spanned the continuum between humans and monkeys. We record extracellular brain (temporal lobe) signals while monkeys fixate at those faces. To analyze the population of recorded data, we use machine learning algorithms and information theory. Besides, we perform parallel experiments with human subjects using the same face stimuli. In the monkey brain we found evidence for a specialized neural mechanism for the processing of monkey faces. In human observers, we also find a “mirror effect” indicating a specialized mechanism for the processing of human faces. Our results suggest a possible mechanism used by the primate brain to assign more “brain resources” to behavioral relevant stimuli.