Cultural background modulates how we look at other persons’ gaze

Introduction

A brief look at another person’s face would tell you a lot about who they are (e.g., identity, age, gender, ethnicity, health and attractiveness). Faces are also the “window to the soul”, because facial expressions signal their emotional states, and gaze direction would tell you what they see and what they know. Having such significance in human social communication, it is not surprising that faces attract attention from very early in infancy. Even newborns preferentially orient to faces (Farroni et al., 2005^; Johnson, Dziurawiec, Ellis, & Morton, 1991), especially those with direct gaze (Farroni, Csibra, Simion, & Johnson, 2002). Eye-tracking studies have demonstrated that infants start to show adult-like face-scanning behaviour, such as preferential fixations on the eyes and mouth (Yarbus, 1967), from as early as 6 weeks after birth (Hunnius & Geuze, 2004). Atypical patterns of face scanning behaviour can be found in neurodevelopmental disorders, such as autism spectrum disorders (ASD), whereby individuals show profound difficulties in social interaction and communication (Senju & Johnson, 2009). Although the mechanisms underlying atypical face-scanning behaviour in ASD is still unclear, it highlights the potential relationship between face-scanning behaviour and the development of social skills.

An important question about the development of face gaze is the role of postnatal environment. Several major theories of social skills development emphasize the role of input from their parents (or caregiver) as well as those from other members of the society, which are essential for the infant brain to learn the social world and become an “expert” (Gauthier & Nelson, 2001^; Pascalis et al., 2005). For example, Sugita (2008) reared infant monkeys with no exposure to faces, and found that general preference to faces develops without exposure to faces, but fine discrimination of faces do not develop. The results highlight the role of innate capacity to detect face-like shapes, and the role of postnatal learning in shaping the capacity to recognize individual faces. However, such a control of postnatal environment is impossible in human studies. So, how can we study the effect of postnatal environment on the development of face gaze in humans? One of the most promising ways is a cross-cultural comparison, because different cultural norms would systematically modulate how the people in each culture would learn to process and interact with others in face-to-face communication.

Two independent lines of research contrasted face gaze between Western European/North American culture and Eastern Asian culture, and found clear differences in the face gaze between the two cultures. First, a series of studies (McCarthy, Lee, Itakura, & Muir, 2006^, 2008) reported that Canadian participants maintain longer eye contact with an interviewer than Japanese participants when they answer cognitively demanding questions. In these studies, the gaze direction of the participants were analysed from the video recording. These studies clearly show the differential face gaze in realistic face-to-face interaction, which is consistent with the cultural norms that gaze avoidance is perceived as insincere in Western culture, but the same behaviour does not have such a negative value in Eastern culture; it can even signal respect in some contexts (Argyle, Henderson, Bond, Iizuka, & Contarello, 1986). However, the video recording does not have a sufficient spatial resolution to examine which part of the face (eyes, nose or mouth) the participants look at. Second, another series of studies recorded eye movements of Western European (British) and Eastern Asian (mainly Chinese) participants as they processed static images of faces, and found that Western European participants showed triangular fixation on both eyes and mouth, but Eastern Asian participants showed more fixation on the centre of the face (Blais, Jack, Scheepers, Fiset, & Caldara, 2008^; Kelly et al., 2011). It was also suggested that reduced fixation on the mouth could partly explain cross-cultural difference in facial expression processing (Jack, Blais, Scheepers, Schyns, & Caldara, 2009). These studies clearly show the subtle differences in face fixations between participants with different cultural backgrounds, but it is not clear whether it is specific to the context they analyse facial information from static images, or a more general pattern of fixations in a more realistic context where they face dynamic sequences of facial actions.

The current study aimed to bridge these gaps in knowledge by investigating how cultural background (British or Japanese) affects the face gaze when they observe dynamic face stimuli. We also examined whether the gaze direction of face stimuli (looking toward or away from the observer) would affect cross-cultural differences in face gaze. We predicted that British participants would maintain longer and sustained eye contact and make a triangular fixation (that is, more fixation on the mouth than Japanese participants), whereas Japanese participants would show shorter and flexible eye fixations and more central fixations. We also predicted that the response to direct and averted gaze would be modulated by the cultural background because of the different cultural norms on the use of eye contact, but no further specific predictions have been made due to the exploratory nature of the manipulation.

Methods

Nineteen British adults (ten females and nine males, mean age 27.98 years) and 22 Japanese adults (11 females and 11 males, mean age 27.75 years) participated in the study. (Four participants were not included because of excessive eye tracker data loss, under 70% samples.) British adults were recruited in central London, and Japanese adults were recruited in central Tokyo. All the participants have normal or corrected-to-normal acuity.

Four computer-generated faces (one Caucasian female, one Caucasian male, one Eastern Asian female, one Eastern Asian male) were selected from the library of Poser 7 (Smith Micro Software, Aliso Viejo, CA), and were used to create 7-second animations with the same software. All the animations started with a face presented upright, facing 30º to the left or to the right and gazing forward, followed by a mouth movement (1 second after the start) and an eye movement (2 seconds after the start). Mouth movements were either smile (Figure 1a, 1b) or mouth opening (Figure 1c, 1d). Eye movements were either direct gaze (Figure 1a, 1c) or averted gaze (Figure 1b, 1d), which involved rotating both eyes laterally by 25º either towards the centre (direct gaze) or away from it (averted gaze). The amount of rotation for direct gaze was selected based on the rating of 10 naïve observers, who rated the perception of “directedness” of the gaze (Todorovic, 2006). The same amount of rotation in the opposite direction was used for averted gaze. In total, 32 animations were generated (4 faces, 2 mouth movements, 2 gaze directions and 2 face orientations). The faces extended 18.4 × 13.0 cm on the screen.

Open multimedia modal

Two Tobii T120 eye-trackers (Tobii, Stockholm, Sweden), equipped with an integrated 17-inch display, were used to present stimuli and record eye-movement in London and in Tokyo. Tobii Studio software was used to control stimulus presentation and to analyse the gaze data.

Recordings were conducted in a quiet and soundproofed room within each research institute. Participants were instructed to watch the movies of the faces. The same experimenter (AS) conducted the recording in both the UK and Japan, to maintain strictly similar experimental conditions such as instructions. A 9-point calibration was conducted using Tobii Studio software before the recording. Recording consisted of two blocks, and each of 32 animations was presented twice (once in each block), in a randomized order. An experimenter also sat in the same testing room, out of sight of the participant, and monitored the recording with Tobii studio software. Viewing distance was approximately 60–65 cm from the display.

The gaze data were initially processed with Tobii studio software to calculate the total visit time. Then, we calculated the fixation duration for each stimulus, for the following areas of interest (AOIs); front eye, back eye, bridge, centre and mouth. Note that faces are tilted either to the right or to the left, one of the eyes is always closer to the observer (i.e., Front Eye) than the other eye (i.e., Back Eye), the latter which is off to the side (Figure 2). The AOIs were selected based on the findings of relevant literature in cross-cultural face scanning studies (Blais et al., 2008^; Jack et al., 2009^; Kelly et al., 2011). These fixation duration data were extracted from the Tobii Studio for statistical analyses.

Open multimedia modal

The gaze data for different head orientations and the blocks were averaged together. The visiting duration for each AOI were then divided with the total visiting duration of the whole face, to calculate the relative visiting duration. The relative visiting duration was analysed with mixed-design ANOVAs to test the effects of cultural background (British or Japanese) and the sex (male or female) of the participants, as well as the ethnicity (Caucasian or Eastern Asian), gender (male or female), gaze direction (direct or averted), mouth movement (smile or mouth opening) and the AOI (front eye, back eye, bridge, centre and mouth) of the stimuli. An initial ANOVA was conducted on the whole 7-second data, which were then followed up by the analyses of seven 1-second bins of the data. For the significant interactions, post-hoc analyses were conducted on each contrast with Wilcoxon sign rank tests with the Bonferroni correction for multiple testing, to provide robust statistics.

Results

As predicted, the interaction between the cultural background, gaze direction and the AOI was significant, F(4, 148) = 4.684, p < .01, η_p² = 0.11, demonstrating that British and Japanese participants fixated differently to the face, depending on whether the face was with direct or averted gaze. The effect was modulated by the sex, F(4, 148) = 4.142, p < .01, η_p² = 0.10 (see also the supplementary material), but not with other factors such as the mouth movement, the gender or the ethnicity of the face stimuli. These interactions remained significant when we excluded British participants of non-Caucasian ethnic origin (three females), and when we excluded British participants who had stayed two years in East Asian countries (two males, one in China and the other in Thailand) and East Asian participants who had stayed 6 months in the USA (two males).

Follow-up analyses revealed that Japanese participants fixated longer on the back eye than did British participants, in both direct gaze and averted gaze conditions. British participants, by contrast, fixated longer on the mouth than Japanese participants, in both direct gaze and averted gaze conditions. British participants also showed longer fixation on the centre, which was only significant in the averted gaze condition (Figure 3).

Open multimedia modal

The two groups showed similarities and differences in the response to different gaze directions. Both groups showed longer fixations on the front eye in the direct gaze condition than in averted gaze condition. However, only Japanese participants showed differential fixations on the back eye, with longer fixation in averted gaze condition (Figure 3). No other contrasts reached significance, including any contrast in the bridge area.

Further analyses were conducted on the seven 1-second bins of the data, to explore the time-course of the differential fixations on the four AOIs showing the gaze and cultural background interactions; front eye, back eye, mouth and centre.

Front eye

No effects reached significance for the first, second and third bins. In the fourth bin, the point right after the gaze shift of the stimuli, both groups showed longer fixation in direct gaze condition than in averted gaze condition. Interestingly, this effect was exaggerated in Japanese participants, who showed even longer fixation than British participants in response to direct gaze and even shorter fixation than British participants in the averted gaze condition (Figure 4a). The same trend remained in the fifth bin, in which only Japanese participants looked longer in the direct gaze condition than in the averted gaze condition. No effect reached significance from the sixth bin.

Open multimedia modal

Discussion

The current study is the first to investigate how the cultural background of the observer affects the face gaze when they observe dynamic faces looking toward or away from the observer. The results clearly demonstrate the cultural difference between British and Japanese participants in the way they look at different parts of the faces, and how they respond to different gaze direction. First, British participants fixated more on the mouth than did Japanese participants, replicating previous studies using static images of faces (Blais et al., 2008^; Jack et al., 2009^; Kelly et al., 2011). Second, both groups of participants fixated equally long on the eye closer to the observer, but Japanese fixated longer on the other eye than British participants. It is consistent with the finding that Eastern Asian participants fixate longer on the eyes (Jack et al., 2009) but it is a novel finding because to our knowledge, this study is the first to use head-turned faces. On the other hand, we did not replicate the longer fixation on the central parts of the faces in Eastern Asian participants, which is reported in previous studies (Blais et al., 2008^; Kelly et al., 2011). By contrast, we even found a longer fixation in British participants on the centre in averted gaze condition (Figure 4d), which happened later than the increased mouth fixation (Figure 4e) and is possibly explained by the residual effects of fixations on the mouth. It might suggest that the increased central fixation on the face in the Eastern Asian population is specific to the task which requires perceptual analyses of the faces, such as the recognition and categorization tasks used in the previous studies (Blais et al., 2008^; Kelly et al., 2011), and does not happen in passive viewing. Another possibility is that Eastern Asians look more to the eyes when the face is expressive, but fixate centrally when they observe faces with neutral expression. Moreover, Japanese participants did not show shorter eye fixation than British participants, which is consistent with some studies (e.g., Jack et al., 2009) but not with others (McCarthy et al., 2006^, 2008). Further studies will be required to test whether the shorter face gaze could be observed in Eastern Asian participants in more naturalistic settings.

Interestingly, these two groups responded differently to the gaze shift of the stimuli. Second-by-second analyses revealed that both groups of participants fixated longer on the front eye immediately (that is, around 1 second) after they saw the gaze shift toward them, but such a change was more exaggerated and lasted longer with Japanese participants than British participants. Moreover, the initial cultural difference in the fixation on the back eye was overridden by the effect of gaze shift, in which Japanese participants fixated longer on the back eye only when they saw an averted gaze. By contrast, the fixation on the back eye was not affected by the gaze direction in British participants. The results showed that Japanese participants shifted their own fixation to the corresponding direction of the observed gaze shift (i.e., to the front, medial eye in response to direct gaze and to the back, lateral eye in response to averted gaze), as if they “followed” the direction of the face gaze. British participants did not show such a change of fixation following observed gaze shift. It might suggest that the eye fixation of British participants reflects the cultural expectation to maintain eye contact, but the eye fixation of Japanese participants reflects the cultural norm to conform to others’ behaviour. Note that it is not a general difference in the sensitivity to facial motion, because mouth movements did not exaggerate or diminish the cultural differences in mouth fixation. The effect was more prominent in males than in females (see the Supplementary material), suggesting that the effect of cultural norm on face and gaze processing manifests more strongly in males. Further studies will be required to see how the differences in gender-related cultural norm interact with face-scanning behaviour.

The current study clearly demonstrates the cultural differences in face gaze in adults, but it cannot tell us how it develops. For example, Kelly et al. (2011) demonstrated that 70% of British-born Chinese adults show a face fixation pattern similar to Eastern culture, whereas 30% of them show a Western pattern of face fixation. This study suggests that cultural diversity in face fixation is more strongly affected by early familial environment, but could also be affected by societal environment (e.g., peers) in some individuals. Further studies will be necessary to study the time-course of the emergence of cultural diversity in face gaze early in the development. Moreover, as in previous adult studies (Blais et al., 2008^; Jack et al., 2009^; Kelly et al., 2011), we did not replicate the other race effect on face scanning (that is, the significant interaction between the ethnicity of the participants and the ethnicity of the stimuli), which contrasts with previous infant research (Liu et al., 2011^; Wheeler et al., 2011). Future developmental studies will be essential to assess the role of face familiarity on face scanning throughout the course of development. We also need to examine how the gender difference found in the current study (that is, larger cross-cultural differences in male than in female participants) develops, by testing younger populations.

To summarize, the current study revealed that the cultural background of the participant affects how they look at another person’s eyes and mouth, and how they modulate eye fixation in response to the gaze direction of others. These differences are consistent with the culturally-relevant strategy of perceptual analyses, as well as the cultural norms on the use of eye contact in face-to-face communication. These results highlight the new frontier of the research about how cultural norms can affect behavioural, cognitive and neural development, which would provide a great opportunity to study the effect of postnatal environment on human behavioural and cognitive development.