Towards generalizable drowsiness monitoring with physiological sensors: A preliminary study
Accurately detecting drowsiness is vital to driving safety. Among different measures, physiological-signal-based drowsiness monitoring can be more privacy-preserving than a camera-based approach. However, conflicts exist regarding how physiological metrics are associated with different drowsiness labels across datasets, which might reduce the generalizability of data-driven models trained with multiple datasets. Thus, we analyzed key features from electrocardiograms (ECG), electrodermal activity (EDA), and respiratory (RESP) signals across four datasets, where different drowsiness inducers (such as fatigue and low arousal) and assessment methods (subjective vs. objective) were used. Binary logistic regression models were built to identify the physiological metrics that are associated with drowsiness. Findings indicate that distinct drowsiness inducers can lead to different physiological responses, and objective assessments were more sensitive than subjective ones in detecting drowsiness. Further, decreased heart rate stability, respiratory amplitude, and tonic EDA are robustly associated with increased drowsiness. These results enhance the understanding of drowsiness detection and can inform future generalizable monitoring designs.
Training Development and Learning Facilitation for Lower-Level Automated Driving Systems Across Age Groups
Autonomous driving holds the potential to transform the transportation industry, offering significant improvements in safety, efficiency, and convenience. However, traditional model-based planning approaches struggle to address the complexities and uncertainties of real-world driving environments. This thesis employs deep reinforcement learning (DRL) to achieve safe and efficient autonomous driving using realistic simulation settings and evaluation based on rational criteria. The proposed framework integrates five key factors—driving safety, driving efficiency, training efficiency, unselfishness, and interpretability (DDTUI) to ensure reliable and optimal decision-making across various driving scenarios. The research addresses two primary applications: highway driving and autonomous racing. In highway driving, the DRL-based framework demonstrates superior performance compared to popular baseline algorithms, improving safety and efficiency. In autonomous racing, an extreme case of autonomous driving, the framework is adapted to manage high velocities and safe control, achieving fewer collisions, faster lap times, and reduced training time in comparison to benchmark algorithms. This thesis contributes to the field by advancing RL-based planning techniques and establishing a design methodology for integrating key factors in autonomous driving. The results of this study provide evidences of the development of safer, more efficient, and interpretable autonomous driving systems. Finally, key achievements are summarized, limitations are discussed, and future research directions are proposed.
Understanding Visual Attention to Button Design Utilizing Eye-Tracking: An Experimental Investigation
As graphical user interfaces continue to become more complex; it is becoming increasingly important for user interface (UI) and user experience (UX) designers to understand how design elements influence user attention. This study investigates the impact of button shape on user perception, focusing on shape preferences, attention distribution, and perceived pleasantness. To isolate the effect of shape, buttons with five different corner radii (completely angular to completely curved) were presented without contextual influences in a pairwise comparison. The research combined eye-tracking technology with digital questionnaires to collect both objective and subjective data. The results obtained revealed a preference for buttons with moderate corner radii, while buttons with completely angular corners received the least attention and were the least favored. Notably, discrepancies emerged between subjective preferences and objective attention rankings, particularly for wireframe buttons. This research demonstrates the effectiveness of eye-tracking in UI/UX design studies and provides valuable insights into the relationship between attention and preference for abstract design elements. The findings offer fundamental theory for creating more intuitive and effective graphical user interfaces, while also highlighting the limitation and importance of examining design elements within relevant contexts in future studies.
Using eye tracking to study the takeover process in conditionally automated driving and piloting systems
In a conditionally automated environment, human operators are often required to resume manual control when the autonomous system reaches its operational limits — a process referred to as takeover. This takeover process can be challenging for human operators, as they must quickly perceive and comprehend critical system information and successfully resume manual control within a limited amount of time. Following a period of autonomous control, human operators’ Situation Awareness (SA) may be compromised, thus potentially impairing their takeover performance. Consequently, investigating potential approaches to enhance the safety and efficiency of the takeover process is essential. Human eyes are vital in an individual’s information gathering, and eye tracking techniques have been extensively applied in the takeover studies in previous research works. The current study aims at enhancing the takeover procedure by utilizing operators’ eye tracking data. The data analysis methods include machine learning techniques and the statistical approach, which will be applied to driving and piloting domains, respectively. Simulation experiments were conducted in two domains: a level-3 semi-autonomous vehicle in the driving domain and an autopilot-assisted aircraft landing scenario in the piloting domain. In both domains, operators ’eye tracking data and simulator-derived operational data were recorded during the experiments. The eye tracking data went through two categories of feature extractions: eye movement features linked predominantly to fixation and saccades, and Area-of-Interest (AOI) features associated with which AOI the gaze was located. Eye tracking features were analyzed using both traditional statistical techniques and machine learning models. Key eye tracking features included fixation-based metrics and AOI features, such as dwelling time, entry count, and gaze entropy. Operators’ SA and takeover performance were measured by a series of domain specific metrics, including Situation Awareness Global Assessment Technique (SAGAT) score, Hazard Perception Time (HPT), Takeover Time (TOT) and Resulting acceleration. Three research topics were discussed in the current thesis and each topic included one driving study and one piloting study. In topic 1, significant differences in eye movement patterns were found between operators with higher versus lower SA, as well as between those with better and worse takeover performance. Besides the notable differences in various Area-of-Interests (AOIs) across three pre-defined Time windows (TWs), in the driving domain, drivers with a better SA and better takeover performance showed inconsistent eye movement patterns after the Takeover Request (TOR) and before they perceived hazards. In the piloting domain, pilots with shorter TOT showed more distributed and complex eye movement pattern before the malfunction alert and after resuming control. During the intervening period, their eye movements were more focused and predictable, indicating vifast identification of necessary controls with minimal visual search. In topic 2, significant differences in eye movement patterns were observed between younger and older drivers, as well as between learner and expert pilots. As for driving domain, older drivers exhibited more extensive visual scanning, indicating difficulty in effectively prioritizing information sources under time pressure. In piloting domain, expert pilots not only allocate more attention to critical instrument areas but also dynamically adjust their scanning behavior based on the current tasks. In topic 3, machine learning models trained on eye tracking features successfully performed binary classification for both SA-related and takeover performance-related metrics. Model performance was evaluated using standard classification metrics, including accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC). Finally, comparisons were made across Topics 1 and 2, as well as between the driving and piloting domains. The results suggest that better operators can flexibly adapt their gaze strategies to meet task demands, shifting between broad visual scanning and focused searching when appropriate. This shift in patterns underscores the importance of accounting for the specific Time window (TW) when interpreting operators’ eye movements. Overall, this thesis advances the understanding of different eye movement patterns during the takeover process by exploring a range of eye tracking features. The findings support the development of operator training programs and the design of customized interfaces to enhance the safety and efficiency of takeover performance.
Analysis and regulation of driving behavior in the entrance zone of freeway tunnels: Implementation of visual guidance systems in China
In China, visual guidance systems are commonly used in tunnels to optimize the visual reference system. However, studies focusing specifically on visual guidance systems in the tunnel entrance zone are limited. Hence, a driving simulation test is performed in this study to quantitatively evaluate the effectiveness of (i) visual guidance devices at different vertical positions (pavement and roadside) and (ii) a multilayer visual guidance system for regulating driving behavior in the tunnel entrance zone. Furthermore, the characteristics of driving behavior and their effects on traffic safety in the tunnel entrance zone are examined. Data such as the vehicle position, area of interest (AOI), throttle position, steering wheel angle, and lane center offset are obtained using a driving simulation platform and an eye-tracking device. As indicators, the first fixation position (FP), starting deceleration position (DP), average throttle position (TPav), number of deceleration stages (N|DS), gradual change degree of the vehicle trajectory (G|VT), and average steering wheel angle (SWAav) are derived. The regulatory effect of visual guidance devices on driving performance is investigated. First, high-position roadside visual guidance devices effectively reduce decision urgency and significantly enhance deceleration and lane-keeping performance. Specifically, the advanced deceleration performance (AD), smooth deceleration performance (SD), trajectory gradualness (TG), and trajectory stability (TS) in the tunnel entrance zone improve by 63%, 225%, 269%, and 244%, respectively. Additionally, the roadside low-position visual guidance devices primarily target the trajectory gradualness (TG), thus resulting in improvements by 80% and 448% in the TG and TS, respectively. Meanwhile, the pavement visual guidance devices focus solely on enhancing the TS and demonstrates a relatively lower improvement rate of 99%. Finally, the synergistic effect of these visual guidance devices facilitates the multilayer visual guidance system in enhancing the deceleration and lane-keeping performance. This aids drivers in early detection and deceleration at the tunnel entrance zone, reduces the urgency of deceleration decisions, promotes smoother deceleration, and improves the gradualness and stability of trajectories.
Biosignals Monitoring for Driver Drowsiness Detection using Deep Neural Networks
Drowsy driving poses a significant risk to road safety, necessitating the development of reliable drowsiness detection systems. In particular, the advancement of Artificial Intelligence based neuroadaptive systems is imperative to effectively mitigate this risk. Towards reaching this goal, the present research focuses on investigating the efficacy of physiological indicators, including heart rate variability (HRV), percentage of eyelid closure over the pupil over time (PERCLOS), blink rate, blink percentage, and electrodermal activity (EDA) signals, in predicting driver drowsiness. The study was conducted with a cohort of 30 participants in controlled simulated driving scenarios, with half driving in a non-monotonous environment and the other half in a monotonous environment. Three deep learning algorithms were employed: sequential neural network (SNN) for HRV, 1D-convolutional neural network (1D-CNN) for EDA, and convolutional recurrent neural network (CRNN) for eye tracking. The HRV-Based Model and EDA-Based Model exhibited strong performance in drowsiness classification, with the HRV model achieving precision, recall, and F1-score of 98.28%, 98%, and 98%, respectively, and the EDA model achieving 96.32%, 96%, and 96% for the same metrics. The confusion matrix further illustrates the model's performance and highlights high accuracy in both HRV and EDA models, affirming their efficiency in detecting driver drowsiness. However, the Eye-Based Model faced difficulties in identifying drowsiness instances, potentially attributable to dataset imbalances and underrepresentation of specific fatigue states. Despite the challenges, this work significantly contributes to ongoing efforts to improve road safety by laying the foundation for effective real-time neuro-adaptive systems for drowsiness detection and mitigation.
Designing an Experimental Platform to Assess Ergonomic Factors and Distraction Index in Law Enforcement Vehicles during Mission-Based Routes
Mission-based routes for various occupations play a crucial role in occupational driver safety, with accident causes varying according to specific mission requirements. This study focuses on the development of a system to address driver distraction among law enforcement officers by optimizing the Driver–Vehicle Interface (DVI). Poorly designed DVIs in law enforcement vehicles, often fitted with aftermarket police equipment, can lead to perceptual-motor problems such as obstructed vision, difficulty reaching controls, and operational errors, resulting in driver distraction. To mitigate these issues, we developed a driving simulation platform specifically for law enforcement vehicles. The development process involved the selection and placement of sensors to monitor driver behavior and interaction with equipment. Key criteria for sensor selection included accuracy, reliability, and the ability to integrate seamlessly with existing vehicle systems. Sensor positions were strategically located based on previous ergonomic studies and digital human modeling to ensure comprehensive monitoring without obstructing the driver’s field of view or access to controls. Our system incorporates sensors positioned on the dashboard, steering wheel, and critical control interfaces, providing real-time data on driver interactions with the vehicle equipment. A supervised machine learning-based prediction model was devised to evaluate the driver’s level of distraction. The configured placement and integration of sensors should be further studied to ensure the updated DVI reduces driver distraction and supports safer mission-based driving operations.
Dynamic driving risk in highway tunnel groups based on pupillary oscillations
This study aims to understand the dynamic changes in driving risks in highway tunnel groups. Real-world driving experiments were conducted, collecting pupil area data to measure pupil size oscillations using the Percentage of Pupil Area Variable (PPAV) metric. The analysis focused on investigating relative pupil size fluctuations to explore trends in driving risk fluctuations within tunnel groups. The objective was to identify accident-prone areas and key factors influencing driving risks, providing insights for safety improvements. The findings revealed an overall “whipping effect” phenomenon in driving risk changes within tunnel groups. Differences were observed between interior tunnel areas and open sections, including adjacent, approach, and departure zones. Higher driving risks were associated with locations closer to the tail end of the tunnel group and shorter exit departure sections. Targeted safety improvement designs should consider fluctuation patterns in different directions, with attention to tunnels at the tail end. In open sections, increased travel distance and lengths of upstream and downstream tunnels raised driving risks, while longer open zones improved driving risks. Driving direction and sequence had minimal impact on risks. By integrating driver vision, tunnel characteristics, and the environment, this study identified high-risk areas and critical factors, providing guidance for monitoring and improving driving risks in tunnel groups. The findings have practical implications for the operation and safety management of tunnel groups.
Evaluation of driver’s situation awareness in freeway exit using backpropagation neural network
Based on combining the relevant studies on situation awareness (SA), this paper integrated multiple indicators, including eye movement, electroencephalogram (EEG), and driving behavior, to evaluate SA. SA is typically divided into three stages: perception, understanding, and prediction. This paper used eye movement indicators to represent perception, EEG indicators to represent understanding, and driving behavior indicators to represent prediction. After identifying indicators for evaluating SA, a driving simulation experiment was designed to collect data on the indicators. 41 subjects were recruited to participate in the investigation, and the experimenter collected data from each subject in a total of 9 groups. After removing 4 groups of invalid data, 365 groups of valid data were finally obtained. The grey correlation analysis was used to optimize the SA indicators, and 10 SA evaluation indicators were finally determined. There were the average fixation duration, the nearest neighbor index, pupil area, the percentage power spectral density values of the 3 rhythmic waves (θ, α, β), rhythmic wave energy combination parameters (α/θ), mean speed, SD of speed and acceleration. Taking the optimized 10 indicators as input and the SA scores as output, a backpropagation neural network model with a topological structure of 10-8-1 was constructed. 75% of the data were randomly selected for model training, and the final network training’s mean square error was 0.0025. Using the remaining 25% of data for verification, the average absolute error and average relative error of the predicted results are 0.248 and 0.046, respectively. This showed that the model was effective, and it was feasible to evaluate the SA by using the data of eye movement, EEG and driving behavior parameters.
Exploring the occupational fatigue risk of short-haul truck drivers: effects of sleep pattern, driving task, and time-on-task on driving behavior and eye-motion metrics
Driver fatigue is the leading cause of truck-related accidents. The most significant occupational fatigue factors among short-haul truck drivers are sleep patterns, the round-trip driving task, and the time-on-task. However, the underlying mechanisms of these influential factors remain unclear. This study aims to explore the interactive effects of sleep patterns, driving task, and time-on-task on driving behavior and eye-motion metrics among short-haul truck drivers. We obtained test data from eleven professional short-haul truck drivers, with each driver participating in a three-day test under the conditions of two driving tasks and three different sleep patterns. We applied three-way repeated-measure ANOVA and non-parametric tests to analyze the data. The results reveal that: (1) violation of sleep-related legal requirements, insufficient sleep, and unreasonable time-on-task can have negative effects on short-haul truck drivers' vigilance and driving performance; (2) both driving task and sleep pattern contribute to driver fatigue, and the interaction of time-on-task and sleep pattern exacerbates driver fatigue more than the effects of any single factor alone; and (3) short-haul truck drivers who are sleep deprived exhibit short periods of controlled compensatory behavior during the outbound task, and sleepiness is more prevalent during the inbound task compared to the outbound task due to the monotony and low workload of the driving process. These findings provide theoretical and practical guidance for transportation industry managers to strengthen company-wide fatigue-related regulations, ensure adequate sleep for drivers via regulations, and optimize work schedules to improve safety outcomes of short-haul truck drivers.
