Inducing visual attention through audiovisual stimuli: Can synchronous sound be a salient event?
We present an experimental research aiming to explore how spatial attention may be biased through auditory stimuli. In particular, we investigate how synchronous sound and image may affect attention and increase the saliency of the audiovisual event. We have designed and implemented an experimental study where subjects, wearing an eye-tracking system, were examined regarding their gaze toward the audiovisual stimuli being displayed. The audiovisual stimuli were specifically tailored for this experiment, consisting of videos contrasting in terms of Synch Points (i.e., moments where a visual event is associated with a visible trigger movement, synchronous with its correspondent sound). While consistency across audiovisual sensory modalities revealed to be an attention-drawing feature, when combined with synchrony, it clearly emphasized the biasing, triggering orienting, that is, focal attention towards the particular scene that contains the Synch Point. Consequently, results revealed synchrony to be a saliency factor, contributing to the strengthening of the focal attention. In today's increasingly complex multimedia landscape, the interaction between auditory and visual stimuli plays a pivotal role in shaping our perception and directing our attention. Within the context of the research on multisensory attention, this study endeavors to explore the intricate dynamics of attentional allocation concerning audiovisual stimuli, specifically focusing on the impact of synchronized auditory and visual cues on capturing and directing attention.
Pistol: Pupil invisible supportive tool in the wild
This paper is an in the wild evaluation of the eye tracking tool Pistol. Pistol supports Pupil Invisible projects and other eye trackers (Dikablis, Emke GmbH, Look, Pupil, and many more) in offline mode. For all eye tracking recordings, Pistol is able to compute pupil and iris ellipse, eye aperture, eye lids, 3D eye ball, optical vector estimation, and eye movement types. For Pupil Invisible projects, it also supports the detection of a special marker and allows the 2D and 3D gaze estimation mapped on the scene camera. In this extended journal paper for Pistol we focus on challenging in the wild situations, such as reflections on glasses, eye tracking during sport, changing lighting conditions, and this for two eye trackers. The tool Pistol can be downloaded here Link
Quantitative Analysis of Physiological and Psychological Impacts of Visual and Auditory Elements in Wuyishan National Park Using Eye-Tracking
Amidst rapid societal changes and increasing urbanization, human connectivity with nature has declined, exacerbating public health concerns. This study assesses the efficacy of Shinrin-yoku, or ‘forest bathing’, in Wuyishan National Park as a simple and effective method to counteract the adverse health effects of contemporary lifestyles. Employing repeated-measures analysis of variance, forty-one participants were observed over three days across eight distinct forest settings. Techniques included eye-tracking for visual attention and soundscape perception assessments via questionnaires. Physiological responses were gauged through heart rate variability and skin conductance, while psychological evaluations utilized the Profile of Mood States (POMS) and Positive and Negative Affect Schedule (PANAS). Findings revealed that (1) natural soundscapes—especially birdsong, flowing water, wind, and bamboo raft sounds—and visual elements, such as distant mountains, streams, trees, Danxia landforms, tea gardens, and bamboo views, play pivotal roles in regulating heart rate variability, reducing arousal, and enhancing stress adaptation. Additionally, cultural landscapes, such as classical music and ancient structures, bolster parasympathetic activity. (2) Natural and cultural auditory stimuli, including flowing water and classical music, coupled with visual features, such as Danxia landforms, streams, distant mountains, lawns, and guide signs, effectively induce positive mood states, regulate mood disturbances, and enhance psychological well-being across diverse forest settings. These findings underscore the significant health benefits of immersive natural experiences and advocate for integrating forest-based wellness programs into public health strategies, offering compelling evidence for enriching life quality through nature engagement.
Recognition of marine navigators’ workload based on eye movement features using bridge simulation
The human factor accounts for 80% of marine traffic accidents, either directly or indirectly. Workload significantly impacts the reliability of marine navigators’ behavior. Employing eye movement measurement as an objective assessment tool can effectively evaluate seafarers’ workload. This study conducts bridge simulation experiments to analyze eye movement features and investigate marine navigators’ workload levels across different tasks. By categorizing marine navigators’ workload into high and low levels based on specific navigation duties, the study employs an independent sample T-test to select blink frequency, fixation time percentage, and pupil area standard deviation as key eye movement features for workload recognition. A marine navigators’ workload recognition model is developed utilizing the support vector machine (SVM) algorithm, which is subsequently trained and evaluated with experimental data. Results demonstrate a high recognition rate of 90.91% for marine navigators’ workload, indicating the SVM method’s accuracy in workload assessment based on eye movement features. These findings lay the foundation for the advancement of intelligent monitoring systems for marine navigators’ workload and the implementation of human–computer interaction active early warning technology and equipment.
The effects of errorless psychomotor training in the
This study investigated the training effect of errorless psychomotor training, a motor training method with perceptual, attentional, and psychological manipulation, in a balance-related, lower limb reaching task (Y balance reaching task) on dynamic balance by young adults. Thirty-nine participants (Mean age = 27.03 years, SD = 2.64 years) were trained with different psychomotor training methods in the Y balance reaching task. Results illustrate that errorless psychomotor training significantly improved the participants’ dynamic balance and proprioceptive abilities. Additionally, gaze fixation duration on target during reaching decreased after errorless psychomotor training, suggesting that errorless psychomotor training could decrease visual information demand and be concurrently compensated by up-weighting on proprioception. This multisensory reweighting and cross-modal attention could contribute to the improvement of dynamic balance ability in sports.
The Role of Visuomotor Behaviours in Understanding the Functionality of Upper Limb Prostheses
Advanced upper limb prostheses aim to restore coordinated hand and arm function. However, this objective can be difficult to quantify as coordinated movements require an intact visuomotor system. Eye tracking has recently been applied to study the visuomotor behaviours of upper limb prosthesis users by enabling the calculation of eye movement metrics. This scoping review aims to characterize the visuomotor behaviours of upper limb prosthesis users as described by eye tracking metrics, to summarize the eye tracking metrics used to describe prosthetic behaviour, and to identify gaps in the literature and potential areas for future research. A review of the literature was performed to identify articles that reported eye tracking metrics to evaluate the visual behaviours of individuals using an upper limb prosthesis. Data on the level of amputation, type of prosthetic device, type of eye tracker, primary eye metrics, secondary outcome metrics, experimental task, aims, and key findings were extracted. Seventeen studies were included in this scoping review. A consistently reported finding is that prosthesis users have a characteristic visuomotor behaviour that differs from that of individuals with intact arm function. Visual attention has been reported to be directed more towards the hand and less towards the target during object manipulation tasks. A gaze switching strategy and delay to disengage gaze from the current target has also been reported. Differences in the type of prosthetic device and experimental task have revealed some distinct gaze behaviours. Control factors have been shown to be related to gaze behaviour, while sensory feedback and training interventions have been demonstrated to reduce the visual attention associated with prosthesis use. Eye tracking metrics have also been used to assess the cognitive load and sense of agency of prosthesis users. Overall, there is evidence that eye tracking is an effective tool to quantitatively assess the visuomotor behaviour of prosthesis users and the recorded eye metrics are sensitive to change in response to various factors. Additional studies are needed to validate the eye metrics used to assess cognitive load and sense of agency in upper limb prosthesis users.
User-Centered Development of Interaction Concepts for a Comfortable and Safe Use of Travel Time in the Context of Fragmented Automated Drives
Automated driving will fundamentally change the way car travel time is utilized. The possibility to engage in a variety of non-driving related activities (NDRAs) is perceived as a major user benefit. However, technical and regulatory restrictions will limit the availability of automated driving functions for the next decades. This will result in fragmented trips consisting of manual, assisted, and automated driving segments. The present thesis seeks to support users of such vehicles in using their drive time as comfortably and safely as possible. Different interaction concepts were developed to support users prior to a drive and during automated and assisted driving. The development followed the user-centered design process. An introductory literature research reveals that users want to be involved in route planning when contrasted with fragmented drives. User requirements for a trip planning tool were derived in a workshop. Based on user needs like travel profiles, prioritization of NDRAs, and an easy-to-interpret visualization of route alternatives, a first prototype was designed, iteratively evaluated, and improved. Finally, a functional smartphone app was developed and evaluated in a usability study. Besides good usability and a high level of intention to use the app, users were able to plan a drive in a short amount of time. Furthermore, the user requirements were highly fulfilled. In addition to the pre-drive phase, the automated driving phase was investigated. During automated driving, NDRAs will be performed on different devices and objects. To prepare users for interruptions caused by transitions to manual or assisted driving, peripherally visible concepts were developed to communicate the remaining time in automated driving in a device-independent and non-interruptive manner. Therefore, a light-emitting diode (LED) stripe was mounted at the bottom of the windshield. In a set of two driving simulator studies, the developed concepts were evaluated positively. However, remaining time estimates will be subject to uncertainty, e.g., due to changing traffic or weather conditions, updated infrastructure information, or unforeseen route changes. Thus, a concept to communicate time budget uncertainties was developed in a small-scale user study. However, the concept using the well-known mobile phone connection icon to convey time budget prediction confidence was rated too complicated in another driving simulator study. Finally, the assisted driving phase was investigated. Users are known to quickly disengage from the monitoring task and rather engage in NDRAs, especially when using a reliable assisted driving function. Thus, a concept to counteract this effect by displaying short motivating pop-up messages was designed in cooperation with usability experts. In a driving simulator study, it was shown to have positive effects on monitoring behavior but did not improve drivers’ reactions to a silent system malfunction. In summary, concepts that can support users in their use of drive time were developed and found to increase comfort and safety in the context of fragmented automated drives. For further implementation, the actual availability and prediction capability of automated driving systems need to be considered.
Using eye tracking to support professional learning in vision-intensive professions: a case of aviation pilots
In an authentic flight simulator, the instructor is traditionally located behind the learner and is thus unable to observe the pilot’s visual attention (i.e. gaze behavior). The focus of this article is visual attention in relation to pilots’ professional learning in an Airbus A320 Full Flight Simulator. For this purpose, we measured and analyzed pilots’ visual scanning behavior during flight simulation-based training. Eye-tracking data were collected from the participants (N = 15 pilots in training) to objectively and non-intrusively study their visual attention behavior. First, we derived and compared the visual scanning patterns. The descriptive statistics revealed the pilots’ visual scanning paths and whether they followed the expected flight protocol. Second, we developed a procedure to automate the analysis. Specifically, a Hidden Markov model (HMM) was used to automatically capture the actual phases of pilots’ visual scanning. The advantage of this technique is that it is not bound to manual assessment based on graphs or descriptive data. In addition, different scanning patterns can be revealed in authentic learning situations where gaze behavior is not known in advance. Our results illustrate that HMM can provide a complementary approach to descriptive statistics. Implications for future research are discussed, including how artificial intelligence in education could benefit from the HMM approach.
