Driving Simulation

Informing Drivers of ADAS Capability: Effects of Functionality vs. Limitation-Focused Training on Takeover in Silent Failure Scenarios

Year: 2025

Authors: S Yan, J Zhang, Z Wang, D He , Thrust of Robotics and Autonomous Systems, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China, Thrust of Intelligent Transportation, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China, HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen, China

As advanced driver assistance systems evolved to incorporate more complex features, enhancing drivers’ understanding of the ADAS capability becomes crucial for ensuring safe and effective human-automation collaboration in critical system failure situations. However, although academia has emphasized the importance of informing drivers about the limitations of ADAS, most ADAS information conveyed in the sales channel remains basic or is mainly positive, highlighting the functions rather than the limitations of the ADAS. Thus, it is essential to comprehend how such positive information influences users’ perception of ADAS and whether additional limitation information enhances or hinders drivers’ understanding of ADAS. Further, most previous studies validated the effects of training in system failures with takeover requests (TORs), which are different from real-world scenarios, where silent failures without TORs dominate. Thus, this study investigates the effectiveness of various training strategies in conveying ADAS capabilities (including functionality-focused, limitation-focused, and combined training) to drivers. In a driving simulator experiment with 32 participants, we evaluated how drivers behave in silent failure scenarios after receiving different training programs. The results show that functionality-focused training induces hesitation in critical scenarios, while limitation-focused training encourages vigilance but triggers over-reactions during takeover events. In contrast, combined training significantly enhances situational awareness and takeover performance without increasing mental workload, underscoring the importance of a balanced educational approach. These findings highlight the role of driver training, even in silent failures, emphasizing the need for comprehensive training that integrates both system functionalities and limitations and provide insights for optimizing human-automation collaboration in safety-critical scenarios.

1 version available:

Investigating effects of temperature and CO2 on driver drowsiness in the context of conditional automated driving

With the introduction of conditional automated driving, drivers are freed from continuous control but must remain alert for takeover requests. This study examines how in-cabin temperature (22.5 °C, 25 °C and 27.5 °C) and CO2 (4200 and 1200 ppm) influence driver drowsiness and physiological responses under conditional automated driving. A driving simulator experiment involving 60 participants was conducted, collecting subjective ratings, eye-tracking and physiological data. Results showed that cooler temperatures were associated with lower drowsiness levels compared to neutral temperatures. However, physiological responses may mainly reflect thermoregulation when temperature varies, obscuring drowsiness-related changes. Further, although CO2 concentration did not significantly affect subjective drowsiness, higher CO2 levels attenuated cardiovascular and autonomic activity, suggesting CO2 effects on physiological responses can emerge before conscious awareness. These findings suggest that climate control systems in automated vehicles should balance comfort, efficiency and driver alertness, while physiology-based driver monitoring systems should incorporate environmental data to detect drowsiness earlier.

1 version available:

Modulating driver visual behavior in highway tunnel groups through rhythm-based visual cues

Drivers navigating highway tunnel groups face complex and repetitive lighting environments, which increase cognitive load and compromise safety. This study aimed to evaluate the effectiveness of rhythm-based visual cues in improving perceptual orientation and regulating driver visual behavior in tunnel group scenarios. Methods To determine the most suitable rhythm pattern for visual guidance, a rhythm adaptability selection process was first conducted using a fuzzy evaluation method. A simulated driving experiment was then performed in UC-win/Road, exposing participants to varying lighting conditions with and without rhythm-based visual cues. Driver eye-movement metrics, including fixation duration, saccade amplitude, and saccade velocity, were used as behavioral indicators to assess visual load and attention regulation. Results Extended tunnel driving was associated with increased fixation durations, a dominance of small-angle saccades (0°–10°), and slower saccade velocities—patterns indicative of visual fatigue and decreased attentional control. The introduction of rhythm-based visual cues significantly improved visual behavior by reducing fixation durations, lowering the frequency of small-angle saccades, and enhancing saccade dynamics. These improvements were more prominent under low-light conditions, indicating better visual adaptability and engagement. Conclusions Rhythm-based visual cues provide an effective and energy-efficient alternative to increased tunnel illumination by directly targeting visual behavior. The findings support the application of perceptually informed environmental design strategies that utilize rhythm sensitivity to improve driver safety in highway tunnel groups.

1 version available:

Overview of optical radiation safety requirements for eye tracking systems

Eye-tracking systems have gained significant attention in various applications, including human-computer interaction, and direct eye projection applications such as near-to-eye displays and augmented/virtual/mixed reality. These systems use optical sensors to monitor eye movements and gaze direction in order to adapt and optimize the displayed image. The working principles of these sensors are based on the emission and detection of optical radiation of light emitting diodes or lasers. Therefore, the optical radiation safety needs to be considered during development of such eye-tracking systems to enable safe use. The aim of the paper is to provide an overview of existing standards and technical specifications related to eye tracking and to highlight potential gaps. As a result, it contributes to the responsible and safe deployment of eye-tracking technology across various applications.

2 versions available

Personalized Course Recommendations Leveraging Machine and Transfer Learning Toward Improved Student Outcomes

University advising at matriculation must operate under strict information constraints, typically without any post-enrolment interaction history.We present a unified, leakage-free pipeline for predicting early dropout risk and generating cold-start programme recommendations from pre-enrolment signals alone, with an optional early-warning variant incorporating first-term academic aggregates. The approach instantiates lightweight multimodal architectures: tabular RNNs, DistilBERT encoders for compact profile sentences, and a cross-attention fusion module evaluated end-to-end on a public benchmark (UCI id 697; n = 3630 students across 17 programmes). For dropout, fusing text with numerics yields the strongest thresholded performance (Hybrid RNN–DistilBERT: f1-score ≈ 0.9161, MCC ≈ 0.7750, and simple ensembling modestly improves threshold-free discrimination (Area Under Receiver Operating Characteristic Curve (AUROC) up to ≈0.9488). A text-only branch markedly underperforms, indicating that numeric demographics and early curricular aggregates carry the dominant signal at this horizon. For programme recommendation, pre-enrolment demographics alone support actionable rankings (Demographic Multi-Layer Perceptron (MLP): Normalized Discounted Cumulative Gain @ 10 (NDCG@10) ≈ 0.5793, Top-10 ≈ 0.9380, exceeding a popularity prior by 25–27 percentage points in NDCG@10); adding text offers marginal gains in hit rate but not in NDCG on this cohort. Methodologically, we enforce leakage guards, deterministic preprocessing, stratified splits, and comprehensive metrics, enabling reproducibility on non-proprietary data. Practically, the pipeline supports orientation-time triage (high-recall early-warning) and shortlist generation for programme selection. The results position matriculation-time advising as a joint prediction–recommendation problem solvable with carefully engineered pre-enrolment views and lightweight multimodal models, without reliance on historical interactions.

1 version available:

Physdrive: A multimodal remote physiological measurement dataset for in-vehicle driver monitoring

Robust and unobtrusive in-vehicle physiological monitoring is crucial for ensuring driving safety and user experience. While remote physiological measurement (RPM) offers a promising non-invasive solution, its translation to real-world driving scenarios is critically constrained by the scarcity of comprehensive datasets. Existing resources are often limited in scale, modality diversity, the breadth of biometric annotations, and the range of captured conditions, thereby omitting inherent real-world challenges in driving. Here, we present PhysDrive, the first large-scale multimodal dataset for contactless in-vehicle physiological sensing with dedicated consideration on various modality settings and driving factors. PhysDrive collects data from 48 drivers, including synchronized RGB, near-infrared camera, and raw mmWave radar data, accompanied with six synchronized ground truths (ECG, BVP, Respiration, HR, RR, and SpO2). It covers a wide spectrum of naturalistic driving conditions, including driver motions, dynamic natural light, vehicle types, and road conditions. We extensively evaluate both signal-processing and deep-learning methods on PhysDrive, establishing a comprehensive benchmark across all modalities, and release full open-source code with compatibility for mainstream public toolboxes. We envision PhysDrive will serve as a foundational resource and accelerate research on multimodal driver monitoring and smart-cockpit systems.

1 version available:

Road Signs Perception: Eye Tracking Case Study in Real Road Traffic

This study investigates driver visual perception of road traffic signs under real road conditions. Using mobile eye tracking technology, we analyzed glance behavior toward various traffic signs and advertisements along urban and highway routes during daytime and nighttime conditions. Results showed significant differences in glance duration and frequency based on sign type, environmental conditions, and the presence of advertisements. Drivers primarily focused on speed limit and directional signs, while advertisements attracted longer glance durations despite their lower frequency of detection. Nighttime conditions generally led to increased glance durations and higher frequencies for most traffic sign types. These findings highlight the importance of optimizing road signage design and placement to improve driver attention and road safety, especially in environments with high visual clutter. Limitations include the exclusion of peripheral vision effects and potential biases introduced by experimental settings.

1 version available:

The aesthetic nature of Chinese-inscribed poetry and painting texts—evidence from eye movements

Previous studies on traditional Chinese inscribed poems and paintings have primarily been subjective qualitative analyses conducted by researchers. In this study, eye-tracking techniques and equipment were utilized to objectively analyze hotspot maps, Area of Interest (AOI) first gaze duration, AOI gaze duration, and sight-switching frequency of viewers when observing inscribed poems and paintings. This research presents these analyses impartially in the form of quantitative data. It reports the eye-tracking data and aesthetic tendencies of three groups with varying art education backgrounds when viewing inscribed poetic paintings. The experiments demonstrated statistically significant differences in aesthetic appreciation, revealing that the inscribed poems and paintings significantly influence the aesthetics of individuals with different art education backgrounds. In the group with the original inscribed poems and paintings, significant differences were observed in the eye-tracking data across the three groups with varying art education backgrounds. Participants with professional art backgrounds focused more on the interaction between the poetry text and the painting. In contrast, in the text + audio narration group, the eye-tracking data of all three groups overlapped significantly, suggesting that understanding the meaning of the poetic text enhanced aesthetic appreciation, particularly for those without professional art backgrounds. Our findings provide insights into the aesthetic appreciation of traditional Chinese paintings, audience perceptions and understandings of artworks, and art museum display practices.

2 versions available

The different synesthesia effects with soundless dynamic lighting and musical dynamic lighting

Dynamic lighting has been proved to have substantial physical, psychological and sociological effects on humans. The degree of the response depends on the intensity, the light color, the flickering frequency and the duration of lighting. The study examined the physiological response to soundless lighting and musical dynamic lighting. Therefore, the physiological signals were recorded to investigate the resonance between the exposure of no sound dynamic lighting with different color (red, green, blue, 3000K, 6000K) and musical dynamic lighting. In the experiment, the heart rate, respiratory rate and electrodermal activity are recorded under dynamic lighting conditions. The results show that the musical dynamic lighting environment has a stronger impact on emotional perception than no sound dynamic lighting. Music plays a main effect role in the sound-light fusion environment. The physiological response is most sensitive under the fusion of dynamic lighting and music. This research will help us better understand synesthesia and provide theoretical support for the future practical application in meta-universe and such virtual simulation area.

1 version available:

Toward Adaptive and User-Centered Intelligent Vehicles: AI Models with Granular Classifications for Risk Detection, Cognitive Workload, and User Preferences

As artificial intelligence (AI) increasingly integrates into our transportation systems, intelligent vehicles have emerged as research topics. Many advancements aim to enhance both the safety and comfort of drivers and the reliability of intelligent vehicles. The main focus of my research is addressing and responding to the varying states and needs of drivers, which is essential for improving driver-vehicle interactions through user-centered design. To contribute to this evolving field, this thesis explores the use of physiological signals and eye-tracking data to decode user states, perceptions, and intentions. While existing studies mostly rely on binary classification models, these approaches are limited in capturing the full spectrum of user states and needs. Addressing this gap, my research focuses on developing AI-driven models with more granular classifications for cognitive workload, risk severity levels, and user preferences for self-driving behaviours. This thesis is structured into three core domains: collision risk detection, cognitive workload estimation, and perception of user preferences for self-driving behaviours. By integrating AI techniques with multi-modal physiological data, my studies develop ML (Machine Learning) models for the domains introduced above and achieve high performance of the ML models. Feature analytical techniques are employed to enhance model interpretability for a better understanding of features and to improve the model performance. These findings pave the way for a new paradigm of intelligent vehicles that are not only more adaptive but also more aligned with user needs and preferences. This research lays the groundwork for the future development of user-centered intelligent companion systems in vehicles, where adaptive, perceptive, and interactive vehicles can better meet the complex demands of their users.

1 version available:

Publication Hub Archive

Informing Drivers of ADAS Capability: Effects of Functionality vs. Limitation-Focused Training on Takeover in Silent Failure Scenarios

Investigating effects of temperature and CO2 on driver drowsiness in the context of conditional automated driving

Modulating driver visual behavior in highway tunnel groups through rhythm-based visual cues

Overview of optical radiation safety requirements for eye tracking systems

Personalized Course Recommendations Leveraging Machine and Transfer Learning Toward Improved Student Outcomes

Physdrive: A multimodal remote physiological measurement dataset for in-vehicle driver monitoring

Road Signs Perception: Eye Tracking Case Study in Real Road Traffic

The aesthetic nature of Chinese-inscribed poetry and painting texts—evidence from eye movements

The different synesthesia effects with soundless dynamic lighting and musical dynamic lighting

Toward Adaptive and User-Centered Intelligent Vehicles: AI Models with Granular Classifications for Risk Detection, Cognitive Workload, and User Preferences