We spend a lot of our time and energy assessing, ourselves and others. We look and re-look in the mirror for the perfect fit to the image we wish to project. We slot objects, people and attitudes into the mental rating boxes that guide our opinions, and we sometimes do this at first sight. Less benign emotions than love can take root in this way too.
Assessment experts, among them teachers, medical doctors and other clinicians, rely on professional rating boxes. Whether lay or specialist, our expectations appear to carry a surprisingly significant weight in how we gather information. Surprisingly, because they’re often based on nothing more substantial than hearsay; and significant, because they mean that we already know the answers to the questions that we nevertheless keep asking: we need not look any further.
I can give one example. I was once taking my regular walk in a nature reserve, in Singapore, where I live, and I heard a group of tourists heatedly debating whether they would be lucky enough to see a crocodile. One of them had read it somewhere that crocodiles are a common sight in Southeast Asian cities. A glimpse of an unmistakably reptile tail, which suddenly scrambled away from us, made his day: “There it is! I just saw one!!” What this visitor actually saw was a (large) monitor lizard. I took a picture of his (her?) friend, or perhaps of her-/himself, on another occasion.
 |
| A (rare) specimen of Crocodillus Singaporeanus. Photo: MCF |
But he (the visitor, not the lizard) will swear he never, ever, laid eyes on a monitor lizard, because he “knew” that crocodiles might be around, not lizards. He will tell all his friends that he saw a crocodile in Singapore, who will tell all their friends, who will then all “know” that there are crocodiles in Singapore. Their source is the indisputable affidavit of an eyewitness.
On a rather more serious level, the way in which expectations affect observations was studied in a school, in the late 1960’s. The findings became known as the Rosenthal Effect. Like the children in this school, multilinguals get labelled before observation, and are therefore expected to be the special case, as mentioned earlier. One common expectation, for example, is that being multilingual is a self-explanatory “diagnosis”. Expectations, incidentally, are also what lies behind national and other stereotypes. If you “know” that an X will provide evidence of W properties, you’ll both see W and disregard all evidence of non-W. The same effect can explain, for example, successful therapies with placebos. And superstition.
Multilinguals, their caregivers, school teachers and speech-language therapists are all in the same boat. Awareness of multilingual behaviour is not required for teaching or clinical certification, even of multilinguals, and language assessment tools are standardised for monolingual users. All too often, teachers and clinicians thus have no reliable tools to guide their assessments. Adapting or translating tests works only partially and often badly, for the reasons we saw before: each language is unique. If dismissing adaptation and translation as a handy solution sounds far-fetched, have a look at what is involved in the mammoth task of making speech-language assessment usable, in this ASHA Directory (American Speech-Language-Hearing Association).
To top it all, there are speech-language assessment instruments for only a tiny fraction of the world’s languages, which of course raises problems for the assessment of monolingual children too. For multilinguals, yet another issue arises, paradoxically, where assessment instruments do exist for one of their languages. This is usually the mainstream language, which may not coincide with the “main” language of a multilingual. On the (monolingual-based) assumption that testing one language is enough to assess the overall linguistic ability of a multilingual, two things can happen. First, that underachievement in that one language may lead to suspicion of language delay. Children may be referred to further therapy, when what they would need is a language tutor. Second, that monolingual assessment naturally misses features of typical mixing, whose regular patterns in fact aid in the diagnosis of language disorder among multilingual children.
Other typical characteristics of multilinguals may go undetected, or pass as atypical. We don’t just mix languages, but also gestures, posture, attitude. A speech-language assessment, despite its name, crucially involves observation of body language, including body and eye contact. The absence (or presence) of standard signals of this kind may be culture-bound. For example, falling silent with a lowered gaze may be interpreted in some cultures as the absence of a response, whereas in others it is the presence of a mark of respect.
The problem is not so much that children (and the rest of us) are not usually socialised in speech-language clinics. The problem is that clinics and clinical tools belong to specific cultures too. What, many of us may wonder, can be a more striking image of childhood well-being than cuddling a teddy-bear? Many of us may indeed wonder, but not all. Perhaps not even most of us. Some of us may wonder instead why someone would want a child to engage in close bodily contact with a miniature representation of a beast of prey.
Cuddly toys are part of the extensive array of clinical aids routinely used to elicit linguistic and other behaviour from children. One common test involves having several toys made available to a child, and then ask the child to grab, in turn, the ones named by the clinician. Children may fail this apparently straightforward test of vocabulary and object recognition, not because of a language problem, but because their culture forbids them to touch objects that do not belong to them. The child wouldn’t have either the sophisticated social and linguistic awareness that is needed to explain, preferably politely, this “non-compliant” behaviour to an adult. Perhaps the child’s culture doesn’t allow children to address adults, or question their demands. Or perhaps the culture does allow this, but not in the language that the child happens to share with the clinician. If you’re interested in research, reviews and reports on these issues, have a look in my recent collection, Multilingual Norms.
School teachers and school kids don’t have it much easier, whether in monolingual or multilingual settings. The language tests that they have to set and sit may also misrepresent their own uses. Here’s one example, taken from a real-life paper in a real-life English school subject:
And here’s one suggestion for you: a) First, answer according to your own use of English, without thinking much – the more one thinks about language uses, especially when they come robed in correctness, the worse it gets, I find; b) Then ask friends, relatives, schoolchildren, teachers of English, to answer too, in the same way; c) Then compare your results; d) Then come back to us on this blog with what you found out.
I won’t tell you (yet) what the test creator took as the right answer. I’ll do it in a comment to this post a few days after publishing it. Your little survey will provide you with a database of actual uses of English, for this particular example, that we can also discuss here. On a suitably large scale and with suitable statistical treatment, this is how norms of usage are established.
To round up my thoughts for today: the neurologist Oliver Sacks wrote, in his 1986 book The man who mistook his wife for a hat, that “Our tests, our approaches [...] our ‘evaluations’, are ridiculously inadequate. They only show us deficits, they do not show us powers.” Granted, he was talking about severely disordered patients, but my point is that we all come out as more or less severely disordered if there are no means of assessing our “normality”. Sacks’ plea concerned a woman with severe mental decomposition, who nevertheless found ways to integrate her world with nature and music, for example. Engaging with nature and music was not part of standard neurological assessment tools.
This is why I mean the word fight in the title of this post quite literally. We may lack the tools that do justice to whoever we’re assessing, but we must at least realise that we do lack them. If we insist on fitting everyone’s feet into Cinderella slippers (another cultural stereotype, I know), which were fashioned for Cinderella’s feet, there is no way we can provide ourselves with the means to offer a fair deal in assessment: we’ll keep finding Ugly Sisters. Or crocodiles.
I have one last thing to say, a confession to make. You may have noticed that I had some difficulty trimming all that I wanted to say about assessment, particularly child assessment, into a post of reasonable dimensions. I had tremendous difficulty. This is an issue that lies very, very close to my heart, but I also know that no one has time to read blogs that just go on scrolling down and down forever. I will come back to these matters. Standardisation, not of tests but of the languages that are used in tests, is in line next but, until then, I hope you will let me know of your own concerns about assessment of multilinguals, here or privately. We need to learn from one another. I also meant quite literally what I said in my welcome message: this blog is for you.
© MCF 2010
= Answer to test question =
ReplyDeleteThe correct answer to the test question was Yes, I have. Perhaps I should say the “correct” answer, or “the correct answer according to this test creator”.
If you don’t think this is the correct answer, you’d have failed this question, and perhaps the test paper because of it.
Madalena