Saturday 30 October 2010

Talking bodies and listening eyes

Languages are not just the things that we have been trained to dissect while sitting in general or specialised classrooms, where we are told to recast sentences in the past tense or identify phonetic properties of allophones (sorry, I just had to include this wink to fellow linguists here). If they were, how come we make faces and gesture emphatically even when we are talking on the phone? It seems that there’s more to languages than meets the ear.

Some languages have no sounds, and therefore cannot be spoken. This is because sound, the medium of many languages around the world, makes sense only if you hear it. Some of us cannot hear, and so use languages that make sense when you see them. Saying that some languages use audible movements of the mouth to produce meaningful exchanges, whereas other languages use visible movements of the hands for the same purpose, makes it look (or sound) like we’re talking about two completely distinct kinds of language. We’re not. As I hope is becoming clear throughout my posts here, things about languages are not as all-or-none as we sometimes like to believe: it’s rather a matter of clines.

Human means of expression are, to use a fancy word, multimodal. This “multi” word, very welcome in this blog, means that we draw on several modes: spoken modes use mostly the mouth, and sign modes mostly the hands. But all of us use both hand and mouth movements in human-to-human interaction – and sometimes human-to-other too: I cannot be the only one shaking fists and uttering profanity at the vagaries of my internet connection, for example. Mouth and hand gestures each offer unique expressive possibilities, which we combine in order to produce more meaning than what a single mode can achieve. In this sense, we are all multilinguals, and we all mix our languages.

Like in any exchange, sometimes there may be glitches, or what some of us might interpret as such: a hand gesture may reinforce, but also contradict a mouth gesture, and vice versa. We may do this intentionally, for example for sarcastic purposes. Or we may fail to notice that we are giving out ambiguous, or even unintelligible signals. If you are a lip-reader, or would like to see what it’s like to be one, you can try one experiment. Experiments are of course artificial, and often probe for extreme effects, but this one may give you a feeling of how visual and auditory cues can interfere with each other. This experimental paradigm became known as the McGurk Effect

There is a common misconception that sign languages are spoken languages “written” in signs, as it were. This reminds of the misconception that spoken languages are simple reproductions of one another, mentioned in a previous post. One reason that might explain it is that a number of sign languages are, or contain, fingerspelling, where hand gestures correspond to printable symbols. Spelling is of course a visual representation of spoken languages too. Printed forms of language are extremely interesting, by the way, because they have managed to take over spoken ones as tokens of so-called good linguistic usage. I will have quite a few things so say about this in a future post. My point here is that sign languages are not the same as hand spellings.

Sign languages are as sophisticated means of communication as spoken ones. If they weren’t, they couldn’t serve their users. All of our languages are acquired in the same way: babies babble, with their hands if they’re acquiring sign languages, with their mouths if acquiring spoken ones. All languages show geographical, historical and individual variation. We can be multilingual in all of them, sign, spoken, or both. Sign and spoken languages are mutually unintelligible, obviously, but so are spoken languages and sign languages among themselves. It may come as a surprise, for example, that British Sign Language (BSL) and American Sign Language (ASL) are not different variants of the same language, like spoken American English and British English: they’re different languages altogether. In addition, fingerspelling can vary, for the same spoken language: BSL fingerspelling and ASL fingerspelling are different languages too.

There is a second misconception about gestural language. Gestures that go with spoken languages are often seen as just flourish: you add them because you belong to a funny culture – those who “add” gestures have equally definite opinions about those who don’t, of course. Take Latinos, for example, by which word I mean anyone sharing a Latin background. They have a reputation for not being able to keep their hands still when they’re talking, so the old joke goes that the way to shut them up is to tie their arms behind their backs. My own Latino roots are often betrayed by my gestural exuberance (I come from Portugal, in case I forgot to mention this), and so I see it as my duty to put the record straight on this one, publicly: I do use my hands a lot when I speak, but not whenever I speak. I’ve lived in several European, African and Asian countries, with two consequences: one, I’ve noticed that different peoples use different visual cues when they talk; and two, I’ve learned to adapt. So when I use my hands, I use them not because I’m Portuguese, but because I’m being Portuguese, which is an entirely different thing. In case this ability to be being different things reminds you of a trait commonly attributed to multilinguals, a “split personality”, I/we hereby pledge to say more about it in a future post.

Gestural language, and body language in general, are not ornamental. They are a necessary part of intelligible exchanges, and they have their own grammar, in that they pattern regularly. We can hear smiles in a voice and we can see passion in a face. If, that is, we’ve learned to associate passion with that particular expression, and perhaps on that particular face, just like we’ve learned to associate the word passion with its meaning. Meanings don’t come out of the blue (or out of dictionaries): we shape them, according to our cultural conventions. Some of us are professionally trained to gain awareness of cultural habits of this kind, and to interpret them in order to assess our overall state of health, including linguistic health. We’ll see how, next time.

© MCF 2010

Next post: The fight for a fair deal. Wednesday 3rd November 2010.


  1. First of all I want to say that I really enjoy your blog - multilingualism is for the time being one of my favourite subjects within linguistics, and I really like reading a language blog written by someone who knows what she's talking about.

    Second, I would like to complement you on this particular entry. I've always been very fascinated by sign language and have even tried to learn Norwegian sign language once, unfortunately without success.

    A few days ago, I was on the bus with a class of elementary school children. I though it was strange because they were very quiet, not like school children at all, but then I noticed: they were all signing. I asked their teacher and she toled me that 75% of the students had cochlear implants and could both speak and hear - but because of the 25% of the students without implants they used only sign language most of the time.

    When I think of these children I think of how lucky they are to be bilingual in both spoken and signed Norwegian, and later they might learn to speak and sign another foreign language too. I just hope they realize how lucky they are to master both.

    And I feel sorry for those who do not see that sign language is just as interesting to study as spoken language.

    I'm looking forward to your next post!

  2. Thanks for the link to the video on the McGurk Effect, Madalena. It’s fun to try out. I must say, though, that in the segment which presented both audio and visual cues, I interpreted the syllable as "ba" instead of "da", the latter of which the video says is the interpretation of most, and that is evidence of the audio-visual illusion. And my initial interpretation seems to have a confounding, residual effect on the next segment without the audio and just the visual, which I also interpreted as “ba” when it should be “ga”. Perhaps one’s interpretation in such cases would depend on whether the part of the mind processing audio cues is more or less activated than the part of the mind processing visual cues at a particular point in time, with the basis of the illusion -- my own illusion in this case being the misinterpretation of visual “ga” as “ba” -- being a ‘carry-over’ of audio to visual cue interpretation or vice versa, and not necessarily a ‘fusion’ of audio and visual cue interpretations?

    I suppose the question on whether we are looking at a ‘carry-over’ or ‘fusion’ is a question we might also ask of “mixing” (borrowing the use of this term from your earlier post, Madalena) between spoken languages? Sometimes, (impromptu) “mixes” by multilinguals are a more integrated ‘fusion’ of two or more languages, e.g. applying the suffixation features of a language (or its variety :)) to a word from another language (I can think of some such ‘innovations’ among local Singaporeans, but going into them would make this lengthy, so I’ll leave this at that for now). At other times, “mixes” by multilinguals are simply a borrowing/‘carry-over’ of words, as they are, from one language to another. Further thoughts?


  3. This is wonderful food for thought you give here, Ingeborg Sophie. To me, it shows the ability to adapt, that we all have, and the will to do so, that we might sometimes want to exercise a little more often. Being able to choose how to communicate with other people can never be a disadvantage.
    I also fully agree with you that we, sound or sign users, have quite a lot to learn from one another. I hope to learn from reader comments like yours too, so I’m very glad you’re finding this blog of use!

  4. My previous post went missing! Anyway, I wanted to say that a friend of mine told me about this owner of a roti prata shop who loses count of the number of pratas each customer orders and usually, the customers have to report the number of each type of prata they had had for him to bill them. I was also told that he gets confused if you talk to him without shaking your head. True enough, when I spoke to him in my regular manner, he couldn't figure out what I had ordered. When I started shaking my head and flinging my arms about like him, he got it immediately. I also find that I tend to receive better service from Indians if I try to speak like them!

  5. Jessie: lovely example of “In Rome, be a Roman”. I believe we all feel a little flattered when we see someone else replicate our traits. Maybe that is what helps communication.


Note: only a member of this blog may post a comment.


Related Posts Plugin for WordPress, Blogger...