[Elsnet-list] Speech Communication: Special Issue on Gesture and Speech in Interaction, Call for Papers

Zofia Malisz zofia.malisz at uni-bielefeld.de
Tue Feb 7 12:54:20 CET 2012


Special Issue on Gesture and Speech in Interaction
 	

Following recent progress in multimodal systems and an increasing  awareness of the important role of manual and facial gestures in  communication, we invite submissions for a special issue of Speech Communication entitled “Gesture and Speech in Interaction“.
 
 	With this issue we would like to promote interdisciplinary perspectives  that facilitate an efficient understanding of the complexity of  multimodal communication. We would like to invite linguists,  phoneticians, speech technologists, computer and cognitive scientists as  well as researchers from all other fields who share an interest in  speech and gesture interfaces. The focus of the special issue lies on  speech-accompanying gestures interacting temporally and functionally  with the speech stream. The type of gestures addressed will be primarily  manual and/or head gestures while not excluding additional analyses  such as eyebrow movement and facial expressions. However, studies on  articulatory gestures and sign language will not be considered because  the former is an integral part of the speech stream and the latter is a  separate linguistic system. Also, papers on facial expression of emotion  are not encouraged unless they have a direct (pragma-)prosodic rather  than psychological focus.
 
 	Background
 
 	During the last years at conferences such as INTERSPEECH, the speech  science community has witnessed a steadily growing interest in the topic  of multimodality. Clear evidence of the growing need for understanding  the speech-manual gesture interaction has been proved at recent  workshops on the topic such as GESPIN conferences in Poznan (2009) and  Bielefeld (2011) which have attracted an international,  interdisciplinary audience.
 
 	A similar scope was adopted by the Gesture Workshops (GW) with main  focus on technical aspects of manual gesture modeling in human-machine  interaction. Both GESPIN and GW have evolved in addition to the AVSP  workshops (concentrating largely on technical aspects of multimodal  facial communication) and ISGS meetings, most widely attended by gesture  researchers, that include all classic gesture topics with specific  themes changing at each meeting. However it is the two interdisciplinary  GESPIN meetings that have explicitely encouraged a deeper understanding  of the topics below that we would like submissions to this issue to  address.
 
 	A growing number of gesture researchers have recently gained great  insights about the role of manual gesture in communication. This  invaluable experience has been gained using mainly psycholinguistic and  discourse analytic methods but has so far rarely dealt with the  gesture-speech interface in a strictly formal manner, e.g.: applicable  in technical systems.
 
 	On the other hand, the wider speech science community (concentrated  around e.g. ISCA affiliated journals such as Speech Communication and  ISCA conferences) with its command of formal tools and applications has  already begun to address the challenges of multimodal communication. It  is ready to concentrate specifically on the analysis and modeling of the  interaction between speech and head/manual gesture. Both communities  are ultimately interested in a joint account of the interplay of the two  modalities in a wide context of human communication. With the proposed  special issue the two communities will receive an opportunity to combine  their expertise.
 
 	Topics
 
 	We would like to see the following topics to be be addressed in the issue:
 
 	1. Mechanisms of temporal coordination of gesture and speech  production. How is gesture form and type constrained to coordinate with  speech?
  -  		locally: what are the anchor points in the speech stream  ("affiliates") that the gesture is coordinated with, are they lexical,  conceptual or prosodic?
 -  		globally: given that gesture is holistic and synthetic and speech is  linear and segmented, how are different gesture forms (smooth, punctual,  mixed) that arise in succession coordinated with continuous speech?

 
 	2. The functional interplay of gesture and speech in communication; the semantics and pragmatics of gesture-speech interaction
  -  		how do deictic and beat gestures express and modify communicative intention and dialogue structure?
 -  		how do iconic and metaphoric gestures express additional meaning and complement semantic content together with speech?

 
 	3. An integrative perceptual account of speech and gesture
  -  		the effects of gesture-speech timing mismatches on the perception of the message.

 
 	4. The interaction of manual and head gestures with perception and production of prosody in a simultaneous speech stream
  -  		how is prosodic emphasis modified by head movement and manual gesture?
 -  		relations between prominence and "intensity" of gesturing.

 
 	5. Application and modeling of the above in technical systems
  -  		dialogue systems involving head and manual gesture generation and recognition,
 -  		artificial agents and robots as implementations of speech-hand-head timing models,
 -  		in general, papers describing technical systems that are able to  process both speech and head/manual gesture (and other modalities) will  be encouraged.

 
 	As regards the methodological angle that will be promoted, theory  building, empirical investigations and gesture-speech modeling will  define the general approach to the problems. At the same time, we would  like to put special emphasis on technical applications (as in point 5  above) as well as first steps towards tools and annotation standards,  i.e., tools for integrative analyses of speech and gesture corpora and  systems for joint gesture and speech annotation.
 
 	Important Dates
 
 	Submission deadline: 31 May 2012
 	Notification of acceptance: September 2012
 	Final manuscript due: January 2013
 	Tentative publication date: Summer 2013
 
 	Editors
 
 	Prof. Dr. Petra Wagner (Bielefeld University, Germany)
 	Dr Zofia Malisz (Bielefeld University, Germany)
 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.elsnet.org/pipermail/elsnet-list/attachments/20120207/ebc25092/attachment-0001.htm 


More information about the Elsnet-list mailing list