Reaching beneath the tip of the iceberg: A guide to the Freiburg Multimodal Interaction Corpus
Most corpora tacitly subscribe to a speech-only view filtering out anything that is not a ‘word’ and transcribing the spoken language merely orthographically despite the fact that the “speech-only view on language is fundamentally incomplete” (Kok 2017, 2) due to the deep intertwining of the verbal, vocal, and kinesic modalities (Levinson and Holler 2014). This article introduces the Freiburg Multimodal Interaction Corpus (FreMIC), a multimodal and interactional corpus of unscripted conversation in English currently under construction. At the time of writing, FreMIC comprises (i) c. 29 h of video-recordings transcribed and annotated in detail and (ii) automatically (and manually) generated multimodal data. All conversations are transcribed in ELAN both orthographically and using Jeffersonian conventions to render verbal content and interactionally relevant details of sequencing (e.g. overlap, latching), temporal aspects (pauses, acceleration/deceleration), phonological aspects (e.g. intensity, pitch, stretching, truncation, voice quality), and laughter. Moreover, the orthographic transcriptions are exhaustively PoS-tagged using the CLAWS web tagger (Garside and Smith 1997 ). ELAN-based transcriptions also provide exhaustive annotations of re-enactments (also referred to as (free) direct speech, constructed dialogue, etc.) as well as silent gestures (meaningful gestures that occur without accompanying speech). The multimodal data are derived from psychophysiological measurements and eye tracking. The psychophysiological measurements include, inter alia, electrodermal activity or GSR, which is indicative of emotional arousal (e.g. Peräkylä et al. 2015 ). Eye tracking produces data of two kinds: gaze direction and pupil size. In FreMIC, gazes are automatically recorded using the area-of-interest technology. Gaze direction is interactionally key, for example, in turn-taking (e.g. Auer 2021) and re-enactments (e.g. Pfeiffer and Weiss 2022 ), while changes in pupil size provide a window onto cognitive intensity (e.g. Barthel and Sauppe 2019). To demonstrate what opportunities FreMIC’s (combination of) transcriptions, annotations, and multimodal data open up for research in Interactional (Corpus) Linguistics, this article reports on interim results derived from work-in-progress.
Eye Tracking Glasses
Software
Research on pathfinding cognition in complex underground public space utilizing eye movement and EEG synchronous detection
The complex underground public space brings challenge on pathfinding cognition. This research took Luoxiong Road Station in Wuhan as experimental site, using eye movement and electroencephalogram (EEG) synchronous detection, through pathway analysis and space-time fault method, exploring relationship between pathfinding cognition and complex underground public space. It was found that pathfinding habits and pathfinding decision-making were mainly affected by personal factors and environmental factors, respectively. Pathfinders influenced by personal factors, had behaviors such as left-walking or right-walking tendency, and the ratio of brain wave β/α was low (impression ratio ≤1). Pathfinders influenced by environmental factors, interested in the other side situation, vertical traffic and so on, and the ratio of brain wave β/α was high (impression ratio >1), with the frontal and parietal occipital lobe regions of brain being more active. These conclusions provided theoretical and data supplements for underground rail transit environment optimizing design in future.
Eye Tracking Glasses
Simulator
Social and affective neuroscience of everyday human interaction: From theory to methodology
This book seeks to address central aspects for the scientific understanding of social and affective neuroscience as a whole. The book contains four parts:(I) Affective Neuroscience;(II) Social Neuroscience and Moral Emotions;(III) Clinical Neuroscience; and (IV) Methods Used in Social and Affective Neuroscience. The first part, Affective Neuroscience, presents the current state of affective neuroscience research. The term ‘affective’relates to moods and emotions and their processing, which plays a crucial role in human social interactions. We are constantly presented with our own emotions and moods and those of others. In social interactions, perceived affect processing and the processing of one’s own affect constitute ongoing necessities. Affective states guide our attention as well as motivation and thus have an effect on social interactions. The chapters in this part investigate psychological, neural and molecular aspects of affective neuroscience. The Social Neuroscience and Moral Emotions part covers phenomena present in society. Social and moral emotions guide our behaviour towards others, but the magnitude with which individuals experience these emotions varies greatly. The chapters in this part present neurobiological and behavioural processes in relevance to social interactions covering topics such mirror neurons and sex differences in social cognition as well as the development of morality and trust in the realm of social interaction. The Clinical Neuroscience part focuses on disorders/conditions that affect social cognition. As much as neuroscience can be used to explain everyday phenomena in social interactions, neuroscience can explain disorders/conditions of clinical relevance. The investigation of brains of healthy individuals compared to those with clinical diagnoses provides invaluable information on the disorders and the associated symptoms. With some conditions affecting social functioning, atypical brain processes can explain abnormalities in regard to social skills. Neuroscience can further explain emotion regulation and deficiency thereof. The chapters in this part stretch from clinical neuroscience in childhood and adolescence to adulthood The last part covers Methods Used in Social and Affective Neuroscience. Experts present state-of-the-art methods to investigate social and affective neuroscience with the typical currently widely applied equipment. That is, brain imaging (MRI), NIRS), electrophysiology (EEG, facial EMG), brain stimulation (TMS AND tDCS) and eye-tracking. The chapters in this part familiarise the reader with the listed methods by providing the necessary basic information but also to deepen the understanding and usability of these methods in social and affective neuroscience for more experienced readers.
Eye Tracking Glasses
Software
Study on the gaze characteristics of urban roads in cold areas under congestion
Abstract: In order to analyze the gaze characteristics of urban road drivers in cold area congestion, the German Dikablis eye tracker and its supportingD-Lab software were used to carry out real vehicle tests on urban roads in Changchun City, and the influence of cold area congestion road characteristics on driver gaze characteristics was quantified by statistical analysis. The results show that drivers mainly obtain traffic information through gaze when driving, and the fixation points are mostly concentrated in road conditions and road traffic flow, especially in the state of ice and snow thawing pavement. The one-way variance result of driver fixation duration was less than 0.05, which was significantly different when driving on urban roads in cold areas under congestion.
Eye Tracking Glasses
Software
Texting while driving: a literature review on driving simulator studies
Road safety is increasingly threatened by distracted driving. Studies have shown that there is a significantly increased risk for a driver of being involved in a car crash due to visual distractions (not watching the road), manual distractions (hands are off the wheel for other non-driving activities), and cognitive and acoustic distractions (the driver is not focused on the driving task). Driving simulators (DSs) are powerful tools for identifying drivers’ responses to different distracting factors in a safe manner. This paper aims to systematically review simulator-based studies to investigate what types of distractions are introduced when using the phone for texting while driving (TWD), what hardware and measures are used to analyze distraction, and what the impact of using mobile devices to read and write messages while driving is on driving performance. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) guidelines. A total of 7151 studies were identified in the database search, of which 67 were included in the review, and they were analyzed in order to respond to four research questions. The main findings revealed that TWD distraction has negative effects on driving performance, affecting drivers’ divided attention and concentration, which can lead to potentially life-threatening traffic events. We also provide several recommendations for driving simulators that can ensure high reliability and validity for experiments. This review can serve as a basis for regulators and interested parties to propose restrictions related to using mobile phones in a vehicle and improve road safety.
The Effect of Attention Cues Training on Saccadic Eye Movement and Learning Open Motor Skills in Novice: Quasi-Experimental Study
Introduction: The cues of the focus of attention, specifically the external focus of attention affect motor performance and improve motor learning. Researches in this field has mainly investigated closed motor skills in novices. Thus generalization of the findings to unpredictable tasks that require response adaptation to external stimuli (Open Skills) needs further investigation. The purpose of this study was to investigate the effect of practicing attention cues on the eye saccade movement and learning the skill of receiving and sending volleyball service as an open skill in novices.
Materials and Methods: The statistical sample of the current research was 20 novice female students with an age range of 10-12 years old, who were selected through convenient sampling and randomly organized into two groups (10 people) of technical and attention cues training. After the pre-test, the participants took part in 9 training sessions. The first and second retention tests were taken 48 hours and 4 weeks later, respective. Data in the performance variable were analysed using 3x2 mixed analysis of variance and in the saccade variable with the Mann-Whitney U test.
Results: The performance of the attention cues training group significantly increased compared to the technical training group from the pre-test to the first retention and the second retention (P = 0.007). The results of the Mann-Whitney U test in the saccade variable was not significant different between the groups at the first retention (P = 0.105) and the second retention (P = 0.089) phase.
Conclusion: Apparently, the training of attention cues is a suitable method for teaching open skills to novices in sports.
Keywords: Attention cues; Saccade; Motor skills
Citation: Sharafiyan F, Shahbazi M, Tahmasebi-Boroujeni S, Moaamer-Ghalehkhalili Y. The Effect of Attention Cues Training on Saccadic Eye Movement and Learning Open Motor Skills in Novice: Quasi-Experimental Study. J Res Rehabil Sci 2023; 19.
Eye Tracking Glasses
Software
The impact of sound in people’s behaviour in outdoor settings: A study using virtual reality and eye-tracking
This paper presents an analysis of space perception and how visual cues, such as landmarks and sound, are perceived and impact people's behaviour while exploring a given outdoor space. The primary goal of the research is to investigate how auditory sensations and visual stimuli influence people's behaviour in outdoor built environments. Our technique compares people's perception of the built environment in different conditions: the real world and a replicated virtual world. As a case study, a university campus was used, and four experimental conditions were designed. The study followed a between-subjects design, and the data collection included gaze data acquired from an eye-tracking device as well as self-reports. The study concludes that sound influences human behaviour in such settings. More specifically conclusions are that: i) human behaviour in virtual replications of the real space, including both visual and sound stimuli, is tendentially more similar to human behaviour in the real world than in simulations omitting sound; and ii) there is a difference in human behaviour when people explore the same virtually replicated outdoor space, by varying the presence of sound. This study is particularly useful for researchers working on the comparison between human behaviour in virtual and real environments, related to visual and sound stimuli.
Eye Tracking Glasses
Software
Transitions from presence, belonging to engaged participation in an inclusive classroom: an eye-tracking study
While many studies about special education needs (SEN) have discussed inclusive efforts in educational settings, more proximal research examining ways to support students with SEN in-process to achieve is needed. This study discussed proximal data, collected from eye trackers and lesson videos, showing how students with SEN transit between the state of presence, belonging, and engaged participation in an inclusive classroom. Using an eye tracking case study of an inclusive science classroom, with a focus on how science teachers support students with dyslexia to learn science, we unpacked how the teacher supported students through such a transformative experience. We characterised the transitions and examined how these transitions happen and found that the transitions were non-unidirectional and could be prematurely truncated due to a change in the teacher’s eye gaze. Some students might not move to the state of engaged participation during a lesson. These transitions were facilitated by the teaching practices. Implications for teachers of inclusive classrooms and research in inclusive classrooms were discussed.
Eye Tracking Glasses
Software
Transportation Research Interdisciplinary Perspectives
Introduction: Drivers turning at urban intersections pose a high risk to Vulnerable Road Users (VRUs), such as cyclists and pedestrians. In vehicle collisions with VRUs, driver attention misallocation is considered a leading contributor. While previous naturalistic studies have examined driver gaze behaviors at intersections, findings are limited to general gaze directions obtained through video analysis, meaning specific areas to which drivers attend cannot be determined. Method: We present a secondary analysis of an on-road instrumented vehicle dataset collected in 2019 which offers eye-tracking and video data from 26 experienced drivers (13 cyclists and 13 non-cyclists). Three coders jointly examined eye-tracking footage from four right-signalized turns (n = 96) to quantify drivers’ glance distributions to various areas of interest, including those most relevant to VRU safety when drivers turn. Individual temporal glance patterns and general attention allocation trends are presented and described. Results: (1) Relevant pedestrians were the top objects of glance irrespective of signal status, and (2) at red light turns, driver attention was heavily skewed toward leftward traffic. Conclusions: This analysis provides a detailed report of driver glance distributions toward scene-specific areas (as opposed to general directions) at urban intersections and discusses how these patterns may influence VRU safety. Practical applications: This study provides important information regarding the human factors challenges of vehicle-VRU collisions and their prevention.
Eye Tracking Glasses
Simulator
