Please send correspondence to: Thomas A. Busey

Department of Psychology

Indiana University

Bloomington, IN 47405

email: busey@indiana.edu

Models of face recognition and classification often adopt a framework in which faces are represented as points in a multidimensional space. This psychological 'face space' organizes the faces according to similarity and makes predictions for representational theories of faces. A variety of image processing techniques have been used to create novel stimuli in this space that represent the average of a population or make a face appear more distinctive. The current research examines the relation the between the stimuli created by these image processing techniques and the underlying psychological representation as measured by multidimensional scaling procedures. Morphing procedures were used to create 16 faces that are embedded in a set of 84 other faces. Similarity ratings were collected between all possible pairs of faces and the data was analyzed using MDS procedures. Dimensions that emerged from the MDS solution included age, race, adiposity and facial hair. In the MDS space, the morphs appear more typical than the parents as predicted by the geometric model. A number of biases were examined, including the tendency of the morph to be less typical than predicted, with may be attributed to the effects of density near the center of face space. In addition, age and facial adiposity biases were also found. The results support the use of the face-space framework for models of face recognition, although image processing techniques that are designed to create novel stimuli in this space may introduce systematic biases.

Recent models of face recognition have focused on the coding and representation of faces using a framework that represents the similarity relations between faces. In this 'face-space' model, a face is encoded as a point in an n-dimensional space along dimensions that differentiate faces, such as age, race or eye width. The distance between any two points is analogous to the similarity between the two faces, and the most typical face will appear in the center of space. The face-space model, while taking its roots from the categorization literature (e.g. Medin & Shaffer, 1978; Nosofsky, 1986), was first proposed by Valentine (1991a,b) and adopted by others to address a number of representational questions in face recognition, including typicality and distinctiveness effects (Valentine & Endo, 1992; Rhodes, Carey, Byatt & Proffitt, in press) and cross-racial identification biases (Chiroro & Valentine, 1995; Levin, 1996).

The face-space model has many desirable characteristics, and has received strong empirical support in the categorization literature. However, testing between models based on the geometric framework often requires precise control of the stimuli. In the categorization literature, representational issues such as the existence of a psychological prototype have required the creation of a physical prototype stimulus from the training exemplars. The creation of novel prototypes is relatively easy when the stimuli consist of random polygons (e.g. Homa, Goldhardt, Burrel-Homa & Smith, 1993), but faces are much more difficult to manipulate.

Undaunted by the challenge of manipulating faces, face perception researchers have created a number of image-processing techniques that provide a physical representation that is assumed to map onto an underlying psychological representation. For example, caricaturing is used to systematically distort an image away from an average face, and it is assumed that this physical manipulation moves the stimulus away from the centroid of the population in psychological space, making the caricature appear more distinctive (e.g. Mauro & Kubovy, 1992; Rhodes, Brennan & Carey, 1987). In addition to caricaturing, proposed techniques include averaging faces after minimal feature alignment (Langlois & Roggman, 1990), morphing by first aligning important features and then warping the digital images onto a common set of features (Benson & Perrett, 1991), and the use of eigenfaces which seek commonalties across the pixels representing digital images of faces (e.g. Bichsel & Pentland, 1994; O'Toole & Thompson, 1993).

These image processing techniques are only useful if the physical representation provided by the image transformation produces a stimulus that maps onto psychological space in a systematic manner. For example, an average of two faces might be created by morphing the two faces, or by locating the two faces in 'eigenspace' and reconstructing a point that represents the average of the two locations in eigenspace. Both techniques produce an average face within their representations, but neither may correspond to the average of the locations of the parent faces in psychological space. Determining a close correspondence between a physical and a psychological representation is important for research such as the facial attractiveness literature where the physical averaging of faces is assumed to correspond to the psychological centroid (e.g. Langlois & Roggman, 1990) or a prototype experiment with faces in which the physical prototype created by combining faces is assumed to correspond to a psychological prototype created by unknown psychological combination mechanisms (e.g. Solso & McCarthy, 1981; Benson & Perrett, 1993; Reinitz, Lammers & Cochran, 1992).

The psychological 'face space' can be measured by asking subjects to make pairwise similarity ratings between all pairs of faces in an experiment and submitting the results to a multi-dimensional scaling (MDS) procedure (e.g. Rolls & Tovee, 1995; Henss, 1994; Hojo, 1988; Sergent, 1984). Despite the fact that these techniques quantify the face space, virtually none of the tests of the face-space hypothesis that have used image-manipulation techniques have measured the face space in order to determine whether the physical techniques produce reasonable results in the psychological space.

The goal of this article is to determine whether one type of manipulation, morphing, creates a physical stimulus with predictable properties in the psychological space. In the morphing procedure, key points are placed on important features on each face. During the warping stage, the key points are used to distort the features of the two faces towards each other, and pixels that fall between the key points are interpolated using linear triangulation (Benson & Perrett, 1993). Once the features have been brought into correspondence, the images are blended together using a pixel-by-pixel averaging procedure. The morph is an average of two faces, and therefore should fall on the midpoint of the vector connecting the two parent faces in face space. Deviations from this point can be used to address questions of typicality, density and the perception of age.

The stimuli were 100 pictures of bald men, 16 of which were morphs (Kayser, 1985). The pictures were all taken under similar lighting with neutral expressions. Faces with facial hair do not morph well, and these were not chosen to be parents. Of the 52 non-parent faces, 21 had facial hair and there were 10 African-American faces. Five of the African-American faces had facial hair. None of the 32 parent faces had facial hair, and there were 3 African-American faces used as parents. This created 16 morphs, 3 of which were blends of a white and an African-American face.

To create the morphs, control points were placed on the salient features of each parent face and 50% averages were created using the Morph™ software package (Gryphon Software). At least 150 control points were placed on each parent, and control points were added to remove obvious artifacts in the resulting morph. Pilot data was collected in which faces were pre-sorted by an observer to determine the rough similarity between the faces. These values were used to group the parent faces into 8 pairs in which the parents were similar to each other, and 8 pairs in which the parents were dissimilar. This manipulation simply ensures that a range of inter-parent similarities is used, since the final MDS solution can be used to determine the exact similarity between the parents.

The faces were digitized and displayed on a 21" Macintosh grayscale monitor. Data was collected by a PowerMac computer using 5 numeric keypads that provided identifiable responses from each keypad. Up to 5 participants completed the experiment at the same time. The faces were simultaneously shown side by side, in random order and position. A face subtends 5.6° by 4.0° degrees from the viewing distance of 180 cm.

Prior to collecting similarity ratings, participants participated in a recognition memory experiment, the results of which are discussed elsewhere (Busey & Tunnicliff, submitted). Subjects saw either a parent or a morph (but not both) in this pre-experiment, controlling the effects of familiarity for the similarity judgments but provided the range of faces.

To obtain a reasonable MDS solution, a minimum of 6 ratings were obtained on each of the 4950 different pairs of faces. Participants made similarity ratings on 20 practice and 177 real face pairs. Similarity ratings were made on a scale from 1 (most similar) to 9 (least similar). To correct for individual differences in the use of the rating scale, individual observer ratings were converted to z-scores prior to computing mean similarity values for all pairs of faces.

Each pair of faces was rated by at least 6 participants, although some pairs were rated by up to 15 participants. The mean number of raters for each face was 8.1. Note that while this is a relatively low number for each pair of faces, the location of a particular face in MDS space is constrained by its measured similarity to all other faces, which tends to average out noise in the individual pairs.

Participants were queried at the end of the experiment if they had noticed anything unusual about the faces, but none identified the use of morphs.

The similarity ratings between all possible pairs of faces were submitted to the ALSCAL scaling procedure. The resulting solution consists of a six dimensional solution with an r2 value of 0.785, indicating that the geometric solution was accounting for a large portion of the variability in the similarity ratings. One test of the adequacy of the face-space model is whether the recovered dimensions are interpretable. In the present case, all six dimensions were easily interpretable, as shown in Table 1. Inspection of the faces plotted along the 6 dimensions revealed that the particular view of MDS space was slightly rotated from what was clearly the interpretable dimensions, which require a rotation procedure that changes how the MDS space is viewed. External ratings were collected on the first 4 dimensions from 8 subjects, which were used to rotate the space such that the interpreted dimension align with the major axis. This rotation does not change the distances or typicality between faces, and the analyses described below don't change if the unrotated space is used instead. Note that the MDS dimensions emerge from the scaling algorithm according to the dimensions along with faces vary and reveal the dimensions that subjects use when comparing faces, and Table 1 demonstrates that age, race and adiposity seem to be primary dimensions along which we organize faces.

One test of whether there is a close correspondence between the physical and psychological representations of faces is whether the morphs lie near their expected locations in MDS space. Define the Euclidean distance in the MDS space between the two parents to be . Likewise, define to be the distance between the morph and its predicted location (M'). If the morph lies in between the two parents in face space, it will lie inside a circle centered at the predicted morph location, with radius. In six dimensions this is a hypersphere centered at M' with radius . Thus I expect

Table 2 shows the distances between the morph and the predicted morph locations () along with the distances between the parents (). These values are substantial, and for 6 triplets the morph actually lies outside the hyper sphere. The ratio of these distances to the radii of the hyper sphere is also quite large: on average the ratio of to is .77, which implies that the morphs are closer to the edge of the hyper-sphere than they are to the center.

Possible explanations for these deviations include effects due to the relation of the morph to dense regions of the space and physical artifacts introduced during blending and warping stages.

A number of researchers have assumed that average faces will be more typical than parent faces, and this is predicted by the geometric model, as illustrated by the logic of the left panel of Figure 1. Two morphs are constructed in this example, and in both cases the morph lies closer to the center of the space than the parents, although Morph 1 will be much more typical than either parent. Typicality here is defined as the sum of the Euclidean distances between a face and all other faces, such that if a face is near the center of space it will have a small typicality value.

Eleven of the 16 morphs were closer to the center of MDS space than both parents and an unpaired t-test comparing the morph typicality values to the parent typicality values is significant at the 0.05 criterion (t(46)= 5.03; p<0.05). Typicality ratings were obtained from 16 additional subjects who rated the similar morphs, the similar parents and the dissimilar parents on how typical the face appeared, operationalized as how easy it would be to spot the face at a train station. The typicality ratings correlated with the summed distance measure across all 40 faces (r=0.61; p<0.05) and the similar morphs were rated as more typical than the similar parents (t(7)=7.68; p<0.05). The demonstration that the morphs are more typical than the parents is important for prototype recognition experiments (Solso & McCarthy, 1981; Reinitz et al, 1992), since this demonstrates that the morph is similar to many other faces. This raises the possibility that exemplar-based models might account for the prototype effects seen in recognition experiments on the basis of similarity alone (Nosofsky, 1986).

One implication of this increase in the typicality of the morphs relative to the parents is that the morphs are likely to be located in a denser region of space than the parents, and this may influence how the morph is perceived. Krumhansl (1978) has demonstrated that psychological space is warped by the density of the stimuli in a particular region. In her distance-density model, two points in a dense sub region will appear less similar than two points of equal physical separation located in a sparse sub region. The density of the space may help the observer gain experience with stimuli in that region and distinguish between smaller deviations than in other regions. This situation can affect the morphs, since the closer the morph lies to the center of space, the more it will be affected by the local density. These inhomogeneities due to density are important, because a similar mechanism has been used to explain the cross-race identification effects in face recognition (Chiroro & Valentine, 1995). Under this model, other-race faces lie in a sparse region and as a result appear more similar, making cross-racial identification difficult.

Predictions for the morphs are straightforward, and are shown in the right panel of Figure 1. The density of the center of the face space will enhance the perceived differences between the morph and the members of the dense region, decreasing the similarity between the morph and the faces in the dense region. From the perspective of the geometric model, this amounts to pushing the morph away from the center of space, making it less typical than predicted. This effect will be less pronounced for morphs that lie farther from the dense region of space. To test this prediction, I compute the typicality values for the predicted and actually morph locations, and compare these deviations against the predicted morph typicality values. The data support the prediction, as shown in Figure 2. Morphs that are predicted to lie near the center of the space have a tendency to be shifted away from the center, while morphs that lie in more sparse sub-regions are unshifted or shifted towards the center of MDS space. Although these results do not directly address the cross-racial identification phenomenon, the results provide converging evidence that such a density-based mechanism is at work in face recognition and therefore could be used to explain cross-racial identification effects.

In addition to the typicality and density effects described above, the morphing operation may affect the apparent age of the morph by smoothing out wrinkles. The effects of morphing on age can be seen in Figure 3, which plots the morphs in MDS space along dimension 1 (age) and dimension 3 (adiposity). The lines connect each morph to its parent faces, and the morphs are systematically shifted towards the younger end of this scale. All 16 morphs were located closer to the younger parent, and on average the morphs are younger than the average age of the parents (t(15) = -10.4; p<0.05).

Independent age ratings for the morphs and parents were obtained from 14 Indiana University undergraduate students, which are plotted in the left panel of Figure 4. The computed and rated age values are highly correlated (r=.95). The data show two effects. First, the morphs are on average rated younger than the parent faces (t(15)= -4.5; p<0.05), which is consistent with the ordering of the morphs and parents along the age dimension in the MDS solution. Second, the slope of the regression line is significantly less than 1.0 (0.835, t(14)=2.36; p<0.05).

In the morphing operation, the two parent faces are first warped towards each other and then blended together. Either of these two steps may have reduced the apparent age of the morphs. First, it seems reasonable to assume that the blending operation reduced the appearance of wrinkles in the morph, thus reducing its apparent age. Since older faces have more wrinkles, the blending artifact might have more effect on older parent faces, which may account for the slope of .835 in the ratings data. Alternatively, the warping operation that occurs during morphing may have made the faces appear more typical, and therefore have an age that is closer to the mean population age. This would tend to decrease the slope of the regression line between mean parent age and morph age, since it would make young faces older and old faces younger.

The possibility that the warping operation contributes to the age bias was evaluated by warping 42 of the original 100 faces to a common set of feature locations. The 42 faces included 13 of the parent faces and none of the morphs. Control points were placed on the 42 faces, which were then averaged to produce a set of control points that represents the average locations of the features. A new set of faces was created by warping the features of each of the 42 faces to the average locations. The effects of the warping and morphing procedures can be seen in Figure 5.

Age ratings on these unwarped and warped faces were obtained from 10 Indiana University undergraduates, and the results are shown in the right panel of Figure 4. Unlike the morph data, the warped faces if anything appear older than the unwarped faces (t = 1.44, n.s.). However, as with the previous data, the slope was significantly less than 1.0 (0.85; t(40)=3.70; p<0.05), demonstrating that the warping procedure makes old faces look younger and young faces look older. This is significant since it demonstrates that the positions of global features are used as cues to age. A related result for caricatures was reported by Burt & Perret (1995), who caricatured faces by manipulating both the texture and the feature locations, and found that both manipulations independently aged the face.

Inspection of the vertical dimensions in Figure 3 suggests that morphs constructed from thin parents may appear pudgy. Such a bias may be introduced if the gross structures of the face such as cheekbones or dimples are smoothed out by the morphing procedure. To test this possibility, twenty-one Indiana University students make ratings on the apparent adiposity of each of the faces. These ratings correlate with the pudginess dimension with an r of .89, indicating that this dimension was interpreted correctly. The obtained morph ratings were compared with values produced by averaging the ratings of the two parents. Overall, the morphs did not appear fatter than their parents. However, inspection of the predicted morph adiposity values suggests a natural grouping that includes the thinnest 7 parent pairs. For this sub-group, the morphs were indeed fatter than predicted by the parent ratings (t(6) = 2.917; p<0.05). This demonstrates that the morphing operation can affect the apparent adiposity of morphed faces, as long as the parents are relatively thin.

There exists the possibility that the age and adiposity biases in some way contribute to or are caused by the density effects seen in Figure 2. If so, morphs that appear younger or fatter than expected should be most affected by the density bias seen in Figure 2. The deviations between the morph and the mean of the parents in the age and adiposity ratings were correlated against the density biases, which produces a correlation of .423 with age and .245 with adiposity, neither of which is greater than zero (critical r with 14 df = 0.497).

The present results address a number of issues regarding the correspondence between the physical representation provided by the morphing operation and the psychological representation of faces as embodied by the face-space model. In general, the results might be viewed as good news for face-space models and potentially troubling news for image manipulation techniques. First, the MDS procedures applied to similarity ratings produced six readily-interpretable dimensions that correlate well with external ratings. This demonstrates that age appears to be the primary dimension along which we organize faces, with race, adiposity and facial hair also contributing. The morphs appear more typical than the parents as predicted by the geometric model. In addition, density effects that have been used by a number of researchers to explain the cross-racial identification effect were found to influence the perception of the morphs, systematically shifting them away from the center of face space.

The current analyses reveal that morphs lie rather far from the predicted locations, which in part can be attributed to effects on the appearance of age and adiposity, in addition to the density effects described above. This may prove problematic for any research that uses image processing techniques to derive a face that represents the centroid of a population. For example, Langlois & Roggman (1990) asked whether an average face is attractive. They created composite faces by aligning the eyes of 32 faces and performing a pixel-by-pixel average. The age biases seen in the morphs suggest that the composite faces would be seen as younger than the original faces, and in a subsequent article, Langlois, Roggman & Musselman, (1994) demonstrate that while the blended faces do indeed appear younger, age and attractiveness ratings do not correlate. Thus they argued that age cannot account for the attractiveness findings. However, Jones (1995) argues that the appearance of youthfulness is linked to attractiveness through fertility or genetic integrity. These non-visual considerations are likely to be viewer-specific. For example, white subjects who demonstrate racist attitudes tend to rate black faces as less attractive than white faces (Fazio, Jackson, Dunton & Williams, 1995). In addition, subjects may reference different standards when making judgments (Biernat & Kobrynowicz, 1997). This may lead to situations where a person is judged attractive for his age rather than against an absolute standard. This would tend to de-correlate age and attractiveness. To deal with this situation, the geometric model may have to be enhanced with selective attention or domain specific knowledge. Levin (1996) suggests such an approach with cross-race identification effects; although we might use a single face-space representation, we may apply different rules when recognizing or categorizing faces in this space. An alternative would be to rely on different sub-spaces in different situations, in which case rules dictating the appropriate subspace must be found (Krumhansl, 1978).

Despite these cautions, the multidimensional approach remains a solid framework for theories concerning face recognition. In addition, morphed faces should not be abandoned as techniques for producing novel stimuli in face-space, despite the potential biases. If the precise similarity relations between the morphs and the parents are important for testing models, similarity ratings should be gathered and used to construct the MDS face space from which quantitative models can be derived.

I gratefully acknowledge assistance from Robert Nosofsky with the rotation and scaling algorithms used to rotate the MDS space. I also gratefully acknowledge Jennifer Tunnicliff's creation of the morph stimuli and collecting the typicality ratings.