Gesturing and Linguistic Meaning
I spent the previous two days in an intensive workshop on the psycholinguistics of gesture. My head hurts. But if I don’t get his sorted out, I will lose the impact of the several info dense presentations. The presenters were from the France, Sweden and South Africa, talk about a multilingual treatment. Interestingly, the audience was all doctoral students here at UWC. None of the faculty in lingu, or edu attended. Don’t know what that means. I had heard that the pre workshop presentation on Monday, made by one of the participants, was perceived by resident faculty as trickery in empirical methods, but that is hearsay. I was blown away by the content and the style of the presenters. So, on to content.
The following information is taken from talks by Marianne Gullberg, Univ of Lund. It is often assumed that when L2 speakers gesture while they speak, it is to fill in for missing L2 words, syntax, etc. According to what I learned in the workshop, this bit of commonly shared “wisdom” is untrue. Gesture is not a filler for language gaps, neither syntax, nor lexemes. Rather, gesture is produced simultaneously with the language output (both L1 and L2), and therefore can facilitate learning if used in conjunction with the language and/or content production. This may be especially true if the teacher is adept at using a conventional set of gestures that the students come to recognize. They may even be specific to a given content area (say, math).
When talk stops, gesture stops. This means that gesture is not, cannot, be a filler. However, when an L2 speaker is struggling, he may use a non-content gesture (hand raised, extended, twirling at wrist) to “keep the floor” so that turn in speaking is not lost while he problem solves internally. But this is characteristically done without eye contact, without engagement with the interlocutor. In contrast, when gesture is used to find a particular word, a word that addressee presumably knows, eye contact is intense, interlocutor may participate in a representational gesture with the speaker as way of verifying the word derived. In addition, conditional or qualifying gestures may be used when the derived, previously missing, word is only an approximation. The conditional gesture demonstrates that the L2 speaker is aware that the word is not an exact match, but will suffice.
Gestures may be “pre-loaded” (such as a lifting and drawing back of an arm at the elbow, “cocked arm”) but will be deployed in alignment (syncopation) with the target word it is intended to elaborate. So gestures are timed with particular targets (words, or phrases). To me, this means that the generation of targeted language bit and the gesture are generated simultaneously. Therefore, gestures are part of the structure of the language system. Only certain things can be regarded as gestures. The physical movements associated with an intended action are not gestures. For example, the physical in picking up a toy are not gestures. Self-regulating behaviors, such as scratching noses, hair, or other self-soothing pats are not gestures. (They may however be meaningful from a psychological perspective, but not as gestures).
Gesture may be conceptualized on a continuum between conventionalized and non-conventionalized. Conventionalized gestures may even stand alone, are perhaps “quotable,” are culture and language specific, and may represent idiomatic expressions. Less conventional gestures are created in flux. They co-occur with the language that they are intended to elaborate, and have no set, independent meaning.
Neill categorizes gesture as iconic, metaphoric, and deictic. Iconic gestures look like what they signal, such as “just a pinch” that is accompanied by index and thumb pinching together. Metaphoric, or more abstract gestures indicate a precision but not an exact pinch, to use the previous example. Diectic gestures are indexical, for example “this, not that” while pointing at the objects. But these three categories are not mutually exclusive (as the double use of “a pinch” indicates). Rather, the categories “scale out” the degree of representativeness of the gesture, or how much the gesture used matched what they are intended to mean (more later on gesturers’ intended meaning). McNeill also points out that some gestures are more temporal, marking the prosody of the spoken language.
Gestures also carry inferential referential meaning for particular spoken words. Here think of a fisherman’s arm and hand gestures when he says “It was [this] big.” Gestures can also function as complete speech acts, even without spoken counterparts. If a “speaker” holds his hand phone to his ear with raised eyebrows and eye contact and a tilt of his head, there is no doubt that he is asking his interlocutor to call him. Gestures may also fill specific syntactic slots. “ Will you ______ the ball?” where the blank is filled by a throwing action, even while the word throw is known, and at hand for the speaker and interlocutor.
One well-worn hypothesis is that when L2 speakers experience difficulty, they recruit gesture to fill the gap created by their “deficit.” This is logical, but not true according to the experimental psycholinguistic data. When the same L2 speaker acquired a certain amount of competence, this gap theory would predict that the speaker would now stop using gesture. This doesn’t happen. In fact, proficient native speakers of any language should not need gesture. But they do use gesture.
So, the research question: Is gesture a linguistic compensation device?
Such a device would need to be;
A rich semiotic resource (in order to be able to generate the variety of gestures needed)
Have expressive power (as the previous examples demonstrated)
Have a tight link between the language and the device
Have relevance to the addressee (must be recognizable)
In fact gesture has all these affordances, so it seems reasonable that it would/could be used as a linguistic compensatory device. But it just isn’t so. Here are two more practical questions:
1. Do gestures replace speech? Apparently not, as there are only gestures when there is speech. Silence means no gestures. There are two instances of gesture during silence: McNeill mentions a “conduit gesture (1985: 354). Kendon (2004) mentions pragmatic gestures. In this latter instance, during silence, there can be a gesture by the struggling speaker, but not to substitute for missing words, but to hold the floor, so that no other speakers step on the talk space currently held by the struggling speaker.
2. Can gestures be used to solve, assist language problems. Yes, but different language problems elicit different types of gestures than simple word supply. Lexical problems (not knowing the word in the L2) are accompanied by gestures that signal engagement with the interlocutor through intense eye contact and a representational use of gesture that is at times co-constructed and subsequently co-performed for verification of the derived word.
The following information is taken from a talk by Ramonna Kunene, Post Doc UCT, PhD from Univ. of Grenoble. Fluent in Zulu, French. Her dissertation compared gestures that accompany accompany narrative recalls of 2:45 minute videos. Participants were speakers of Zulu and French. Transcribed the verbal and gestural products with software ELAN. Data were analyzed for length and type of clause for spoken language; function of gesture and its relationship to the content on the cartoon episode in video. Gesture annotations included presence of gesture (was there a gesture) established in no audio condition. Then subsequent analysis (with audio) for function, relationship to speech, time placement in the audio sequence, gesture form (mouth, hand). Differences were sought between the dependent variables (on gesture) and independent variables of language, age and gender. Significant differences were found for type of clause, type of gesture for age. No significant differences were found for the independent variable of language.
Overall interpretation of the findings point out different approaches to the task. French speaking participants respond to the retell with a tendency to explain, interpret, comment. Zulz speakers tend to approach the retell with narrativizing. Kunene invoked a Sapir Whorf idea of Zulu culture and language being one of oral tradition that highlight narrative. Whereas French is conditioned by centuries of preference for print, and use of pragmatic language.
Incidental findings reported (no doubt related to findings, but I missed the link, so I have the following recorded as somewhat independent findings):
Gesture develops with age.
Zulu speakers tend to use more representational gestures.
Gestures per clause increase with age.
In Zulu, boys make greater use of physical space with larger gestures, and greater number of strokes per gesture.
In use of representational gestures, French speakers plateau at 9-10.
In use of non-representational gestures, French speakers do not plateau.
In Zulu, representational gestures may disambiguate large number of non-indicative shortened forms in an agglutinative language (think pronoun reference).
These findings reminded me of students’ predispositions to summarization tasks in school settings. One of the findings in Brown and Days’ 1980’s research in the development of summarizing abilities is that some participants had difficulty with the task and instead retold in order the events, the steps. It seems that if an oral culture is linguistically grounded in narrativizing experience (like Zulu), those speakers come to the task of summarizing with different language abilities than the French. In the study, the French are characterized as talking ABOUT the film, in abstracted, non-representational way (was the word partege?). So if parts of the US population, identifiable groups, are more prepared to give narrative accounts, what would that mean about observable “ability” differences in summary as a outcome measure? I do remember that the younger the participant in the Brown & Day,, the more likely they were to give a temporal narrative instead of abstracted summaries. This reminds me of the comprehension recall where younger students give the and then, and then report. These were/are seen as “deficit” in that research on comprehension. What if these are linguistically driven communication patterns that can be associated with cultural linguistic worlds?
A final incident in this presentation merits some attention: Manne, post doc from Sweden, asked about the researchers’ prompts that immediately preceded the recall production. Dr. Kunene used several versions: “What did you see?” and “What happened?” Manne’s point was that the questio that precedes the outcome performance has an impact on what they say. Asking “see” suggests (subtly) that the researches wants a description of the physical space,etc. Whereas, asking to “tell about” may elicit a temporal narrative, or a topic/comment description (depending on the backgrounds). The point is that the way you frame the questio impacts the kind of data you get. So frame in the way you want the data to come in, and make a protocol, and stick to it with all participants. I also learned this new term, questio, as a superordinate term for researcher prompt that elicits a response from participants. He will send me a ref that plays this out more thoroughly.
The following is from another talk by Marianne Gullberg. This one gutted me. I was so exhausted from following the exciting intricacies of this beautiful study, that I needed a nap, but was too excited to stop talking bout through lunch (fortunately with Marianne).
What do gestures reveal about what language learners actually mean?
Up until recently, gestures were thought to indicate what L2 speakers didn’t know or couldn’t do. To set up the paper presentation, Gullberg differentiated children’s and adult’s acquisition of gestures as they relate to oral language use.
For Kids, the most common theory of language acquisition is the innate (nativism) hypothesis. This is the generative approach we have grown up with. That language is a human competence that we are born with, and we only need to hear language to have the competence activated. This is Noam Chomsky’s approach generative transformational grammar.
A second approach that is used here by Gullberg is that language is learned gradually through input (emergentists) hypothesis. It is based on what learners hear, and the frequency of input as well as the semantic complexity of the input. The observed uniformity in language development, despite the documented paucity and diversity in the input provided in different homes, is explained by language consistencies, such as [things] are more frequent topics than [events] and [relations]. Semantically general words used more frequently, but semantically specific words with lower frequency may have more impact because of their specificity.
Time out: while I was listening to frequency vs. salience, I was reminded of Eleanor Rosch and Carolyn Mervis’s work with basic level nouns. This is a compelling theory about the organization of semantic memory, and how it maps or doesn’t map the “real world.” R and M suggest that there is level of representation where words work most efficiently to cover the intended meaning, as it is related to the physical world, and that is called the basic level. In biologic taxonomies, it is a regular use of the category such as dog that is basic. Pet is too general to be efficient to create the concept of “dogness” and Chihuahua is too specific. Pet would be considered the superordinate, Chihuahua would be subordinate, and dog is basic. Other examples of Super, basic, sub: furniture, couch, Chippendale couch; fruit, banana, Caribbean banana; flower, rose, climbing rose. Notice that the subordinate often has a modifier. Rosch cites this as additional evidence that the conditionality of the sub position requires a modifier to establish it, where basic is the very level where modifiers are no longer needed (of course, same with levels above basic). Rosch makes the case that this is representative of the “real world” and how things are “actually distributed.” This assertion deserves some debate, but the effects of a basic level of linguistic and cognitive representation appear in data. So what does this effect mean? And how is it related to semantically general and semantically specific language as stimulus for language acqusition? Back to Gullberg:
What about adults acquisition of a second language? Here the adult has a language against which the newer language is situated. So, adults are concerned with
New forms for old meanings
Consider looking up an unknown Dutch word in a dictionary. You now have a
new word (form) for a meaning you already have in L1, right?
Well, not exactly:
Consider the English word on. In Dutch, a bowl on the table is represented in Dutch by the word op. But, a picture on the wall is represented in Dutch as aan.
Therefore, English speakers learning Dutch must experience a re-organization towards the target language, partitioning the earlier more generalized on into two types of on. This is a different task than that if an initial acquisition of a first language.
The separation of what is categorized as boot and shoe differs between English and Spanish. That is bota (boot)and zapato (shoe) somehow conceptualize the differences differently. When I checked this with Richard, my Spanish L1, and English L2 partner, he didn’t see the difference and thought the bota/zapto and boot/shoe relationship was conceptually isomorphic. But, nonetheless… gullberg’s point is that just because an L2 speaker acquires and uses the correct form, they may still be confused as to the correct (precise) meaning differences between the two languages. Gesture can be used to explicate this confusion. What can a look at gestures as a signal of what L2 learners might be intending? Gestures are a vehicle of language specific meaning integrated systems. Gestures are coordinated to deliver the same meaning at the same time as the oral language. Gestures happen at the same time with the information that the speaker considers the most important. Meaning should be revealed in the aligned gestures. But there are differences in cross linguistic uses of gestures, both in form and timing. And these differences in gesture are related to linguistic differences between languages.
What can gestures tell us about how learners form meaning and about the development of meaning. Consider placement verbs – to put something somewhere. Pinker (1989) suggests that put (the conceptual organizing around put). Goldberg has replicated the study here around the world in 29 unrelated languages in their representations of “put.” In Dutch there is leggen and zetten (put down generally, and to place specifically in an upright position). In German there are legen and stellen. These more specified partitions of “put” signify different conceptual frameworks for the generalized “put,” different semantic models for how to understand putting.
The question that guided the study was: Do placement gestures differ in crosslinguistics and reflect language specific verb meaning?
Hyp 1: Language neutral: Dutch=French=German (=action)
Hyp 2: Language is specific: Dutch, French, German not equal:
Form Timing
Dutch
Movement movement (on the) verb
Object and object
Path/handshape
French
Movement movement (on the) verb
Smooth motion path, no handshake
German
Movement (obj) movement to goal (on the) locative
To a goal path to a goal (“to the floor”)
The table above suggests that there should be differences in the tempo and form of gesture, differentiated by language. Movement refers to the action across the interaction space. path is the actual direction, the movement itself. The object refers to what is to be put. So if it is a bowl, as it was in one of the videos, some speakers (those languages that reference the object as part of put) have gestures that cup the two hands side by side to make a 'bowl"). Locatives are the place where the intended put ends up. Such phrases as on the floor, on the table, in the living room are all locatives. Emphasis in the locatives occurs on the preposition.
Task: First person watches a video with a standardized short video segments of putting an object somewhere. First person who watched tells a second person (who didn’t watch) so second person can draw what was in the video. This is a solid data collection scenario as the teller has a real reason for retelling. The researcher was not interested in the drawing, but in the gestures the reteller deployed while doing the oral retelling. There are participants who are a.) German speaking and learning French as L2; and Dutch speaking and learning French as L2. (Recall the language differences in emphasis for the concept put).
Results: Dutch speakers who are attempting a French put (mettre) have a gesture pattern that preserves the shape of the object and preserves the movement. This signals that the Dutch speaker is preserving the Dutch segmentation of leggen/zetten in the gestures that accompany the generalized French mettre. German speakers who are also attempting a French put (mettre) have a gesture pattern that preserves the timing of the gesture on the locative, not on the verb (as do French speakers). The outcome suggest that L2 speakers of French, with undifferentiated put, may be producing the correct form of put (mettre) but are actually meaning a more specific meaning for put. These differences are language specific.
No comments:
Post a Comment