The different synesthesia effects with soundless dynamic lighting and musical dynamic lighting
Dynamic lighting has been proved to have substantial physical, psychological and sociological effects on humans. The degree of the response depends on the intensity, the light color, the flickering frequency and the duration of lighting. The study examined the physiological response to soundless lighting and musical dynamic lighting. Therefore, the physiological signals were recorded to investigate the resonance between the exposure of no sound dynamic lighting with different color (red, green, blue, 3000K, 6000K) and musical dynamic lighting. In the experiment, the heart rate, respiratory rate and electrodermal activity are recorded under dynamic lighting conditions. The results show that the musical dynamic lighting environment has a stronger impact on emotional perception than no sound dynamic lighting. Music plays a main effect role in the sound-light fusion environment. The physiological response is most sensitive under the fusion of dynamic lighting and music. This research will help us better understand synesthesia and provide theoretical support for the future practical application in meta-universe and such virtual simulation area.
Toward Adaptive and User-Centered Intelligent Vehicles: AI Models with Granular Classifications for Risk Detection, Cognitive Workload, and User Preferences
As artificial intelligence (AI) increasingly integrates into our transportation systems, intelligent vehicles have emerged as research topics. Many advancements aim to enhance both the safety and comfort of drivers and the reliability of intelligent vehicles. The main focus of my research is addressing and responding to the varying states and needs of drivers, which is essential for improving driver-vehicle interactions through user-centered design. To contribute to this evolving field, this thesis explores the use of physiological signals and eye-tracking data to decode user states, perceptions, and intentions. While existing studies mostly rely on binary classification models, these approaches are limited in capturing the full spectrum of user states and needs. Addressing this gap, my research focuses on developing AI-driven models with more granular classifications for cognitive workload, risk severity levels, and user preferences for self-driving behaviours. This thesis is structured into three core domains: collision risk detection, cognitive workload estimation, and perception of user preferences for self-driving behaviours. By integrating AI techniques with multi-modal physiological data, my studies develop ML (Machine Learning) models for the domains introduced above and achieve high performance of the ML models. Feature analytical techniques are employed to enhance model interpretability for a better understanding of features and to improve the model performance. These findings pave the way for a new paradigm of intelligent vehicles that are not only more adaptive but also more aligned with user needs and preferences. This research lays the groundwork for the future development of user-centered intelligent companion systems in vehicles, where adaptive, perceptive, and interactive vehicles can better meet the complex demands of their users.
Towards generalizable drowsiness monitoring with physiological sensors: A preliminary study
Accurately detecting drowsiness is vital to driving safety. Among different measures, physiological-signal-based drowsiness monitoring can be more privacy-preserving than a camera-based approach. However, conflicts exist regarding how physiological metrics are associated with different drowsiness labels across datasets, which might reduce the generalizability of data-driven models trained with multiple datasets. Thus, we analyzed key features from electrocardiograms (ECG), electrodermal activity (EDA), and respiratory (RESP) signals across four datasets, where different drowsiness inducers (such as fatigue and low arousal) and assessment methods (subjective vs. objective) were used. Binary logistic regression models were built to identify the physiological metrics that are associated with drowsiness. Findings indicate that distinct drowsiness inducers can lead to different physiological responses, and objective assessments were more sensitive than subjective ones in detecting drowsiness. Further, decreased heart rate stability, respiratory amplitude, and tonic EDA are robustly associated with increased drowsiness. These results enhance the understanding of drowsiness detection and can inform future generalizable monitoring designs.
Training Development and Learning Facilitation for Lower-Level Automated Driving Systems Across Age Groups
Autonomous driving holds the potential to transform the transportation industry, offering significant improvements in safety, efficiency, and convenience. However, traditional model-based planning approaches struggle to address the complexities and uncertainties of real-world driving environments. This thesis employs deep reinforcement learning (DRL) to achieve safe and efficient autonomous driving using realistic simulation settings and evaluation based on rational criteria. The proposed framework integrates five key factors—driving safety, driving efficiency, training efficiency, unselfishness, and interpretability (DDTUI) to ensure reliable and optimal decision-making across various driving scenarios. The research addresses two primary applications: highway driving and autonomous racing. In highway driving, the DRL-based framework demonstrates superior performance compared to popular baseline algorithms, improving safety and efficiency. In autonomous racing, an extreme case of autonomous driving, the framework is adapted to manage high velocities and safe control, achieving fewer collisions, faster lap times, and reduced training time in comparison to benchmark algorithms. This thesis contributes to the field by advancing RL-based planning techniques and establishing a design methodology for integrating key factors in autonomous driving. The results of this study provide evidences of the development of safer, more efficient, and interpretable autonomous driving systems. Finally, key achievements are summarized, limitations are discussed, and future research directions are proposed.
Understanding Visual Attention to Button Design Utilizing Eye-Tracking: An Experimental Investigation
As graphical user interfaces continue to become more complex; it is becoming increasingly important for user interface (UI) and user experience (UX) designers to understand how design elements influence user attention. This study investigates the impact of button shape on user perception, focusing on shape preferences, attention distribution, and perceived pleasantness. To isolate the effect of shape, buttons with five different corner radii (completely angular to completely curved) were presented without contextual influences in a pairwise comparison. The research combined eye-tracking technology with digital questionnaires to collect both objective and subjective data. The results obtained revealed a preference for buttons with moderate corner radii, while buttons with completely angular corners received the least attention and were the least favored. Notably, discrepancies emerged between subjective preferences and objective attention rankings, particularly for wireframe buttons. This research demonstrates the effectiveness of eye-tracking in UI/UX design studies and provides valuable insights into the relationship between attention and preference for abstract design elements. The findings offer fundamental theory for creating more intuitive and effective graphical user interfaces, while also highlighting the limitation and importance of examining design elements within relevant contexts in future studies.
Using eye tracking to study the takeover process in conditionally automated driving and piloting systems
In a conditionally automated environment, human operators are often required to resume manual control when the autonomous system reaches its operational limits — a process referred to as takeover. This takeover process can be challenging for human operators, as they must quickly perceive and comprehend critical system information and successfully resume manual control within a limited amount of time. Following a period of autonomous control, human operators’ Situation Awareness (SA) may be compromised, thus potentially impairing their takeover performance. Consequently, investigating potential approaches to enhance the safety and efficiency of the takeover process is essential. Human eyes are vital in an individual’s information gathering, and eye tracking techniques have been extensively applied in the takeover studies in previous research works. The current study aims at enhancing the takeover procedure by utilizing operators’ eye tracking data. The data analysis methods include machine learning techniques and the statistical approach, which will be applied to driving and piloting domains, respectively. Simulation experiments were conducted in two domains: a level-3 semi-autonomous vehicle in the driving domain and an autopilot-assisted aircraft landing scenario in the piloting domain. In both domains, operators ’eye tracking data and simulator-derived operational data were recorded during the experiments. The eye tracking data went through two categories of feature extractions: eye movement features linked predominantly to fixation and saccades, and Area-of-Interest (AOI) features associated with which AOI the gaze was located. Eye tracking features were analyzed using both traditional statistical techniques and machine learning models. Key eye tracking features included fixation-based metrics and AOI features, such as dwelling time, entry count, and gaze entropy. Operators’ SA and takeover performance were measured by a series of domain specific metrics, including Situation Awareness Global Assessment Technique (SAGAT) score, Hazard Perception Time (HPT), Takeover Time (TOT) and Resulting acceleration. Three research topics were discussed in the current thesis and each topic included one driving study and one piloting study. In topic 1, significant differences in eye movement patterns were found between operators with higher versus lower SA, as well as between those with better and worse takeover performance. Besides the notable differences in various Area-of-Interests (AOIs) across three pre-defined Time windows (TWs), in the driving domain, drivers with a better SA and better takeover performance showed inconsistent eye movement patterns after the Takeover Request (TOR) and before they perceived hazards. In the piloting domain, pilots with shorter TOT showed more distributed and complex eye movement pattern before the malfunction alert and after resuming control. During the intervening period, their eye movements were more focused and predictable, indicating vifast identification of necessary controls with minimal visual search. In topic 2, significant differences in eye movement patterns were observed between younger and older drivers, as well as between learner and expert pilots. As for driving domain, older drivers exhibited more extensive visual scanning, indicating difficulty in effectively prioritizing information sources under time pressure. In piloting domain, expert pilots not only allocate more attention to critical instrument areas but also dynamically adjust their scanning behavior based on the current tasks. In topic 3, machine learning models trained on eye tracking features successfully performed binary classification for both SA-related and takeover performance-related metrics. Model performance was evaluated using standard classification metrics, including accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC). Finally, comparisons were made across Topics 1 and 2, as well as between the driving and piloting domains. The results suggest that better operators can flexibly adapt their gaze strategies to meet task demands, shifting between broad visual scanning and focused searching when appropriate. This shift in patterns underscores the importance of accounting for the specific Time window (TW) when interpreting operators’ eye movements. Overall, this thesis advances the understanding of different eye movement patterns during the takeover process by exploring a range of eye tracking features. The findings support the development of operator training programs and the design of customized interfaces to enhance the safety and efficiency of takeover performance.
An exploration of drivers’ lane position after adding buffered cycling lanes in Guelph, Ontario
Dedicated cycling infrastructure, such as a buffered cycling lane, is implemented more frequently with the goal of improving cyclist safety by decreasing cyclist-vehicle interactions. While previous research has focused on evaluating driver lane position through passing events (when drivers overtake slower cyclists), little research has evaluated how drivers interact with novel cycling infrastructure in the absence of cyclists. Through an analysis of instrumented vehicle data from an on-road study in Guelph, Ontario, this study compares driver behaviors before-and-after modifying an existing cycling lane into a cycling lane with a painted buffer. It was found that drivers were significantly further from the marking of the cycling lane by an average of 31.6 cm when there was a traditional painted cycling lane, as opposed to a buffered cycling lane. This difference was greater than the change in vehicle lane width (narrowed on average by 22.7 cm). However, this may not change overall distance from cyclists when accounting for additional space from the buffer. Drivers did not differ in the standard deviation of their lane position, or in their speeds, between the two types of cycling lane. Findings from this research have implications for decisions regarding infrastructure and the development of automated driving systems.
Association between length of upstream tunnels and visual load in connection zones of highway tunnel groups
To investigate drivers' visual load and comfort in the distance between adjacent tunnels (tunnel group connection zones), the maximum transient vibration value (MTVV) of the pupil area is used in this study as the index to analyze the visual load characteristics of the driver throughout the connection zones in highway tunnel groups. Data was collected using field driving experiments during which the pupil area change rate is measured as an additional indicator to evaluate the sufficiency of the length of the connection zones from the perspective of drivers’ visual adaptation. The findings show that the length of the upstream tunnel affects the visual strain of the drivers when they enter the connection zone. The visual load and its association with the length of the upstream tunnel appeared to be in the following descending order: short > extra-long > long > medium tunnel. The visual discomfort level in the short upstream tunnel has shown to be “uncomfortable,” while the level of comfort slightly rises to “fairly uncomfortable,” in the connection zone when the upstream tunnel is extra long and long. Departing from medium upstream tunnel resulted in the highest level of comfort “a little uncomfortable level” in the connection zone. When the upstream tunnels are short and medium in length, the required time for light adaptation is 5 s. The connection zone length threshold which is the minimum length of connection zone in order for two consecutive tunnels not to affect each other in terms of visual load of drivers is calculated to be 713.89 m. The driver's pupil area change during light adaptation when the upstream tunnel is short and medium is in the range of 30–40 %. When upstream tunnel is long and extremely long, the light adaptation time is 8 s and 9 s, respectively, and the respective thresholds for connection zone are 797.22 m and 825 m. The drivers' pupil area change in long and extremely long tunnels during light adaptation is in the range of 38–50 % and 43–50 %, respectively. Findings in this study can be used for the design of connection zones between tunnels in a highway tunnel group.
Biosignals Monitoring for Driver Drowsiness Detection using Deep Neural Networks
Drowsy driving poses a significant risk to road safety, necessitating the development of reliable drowsiness detection systems. In particular, the advancement of Artificial Intelligence based neuroadaptive systems is imperative to effectively mitigate this risk. Towards reaching this goal, the present research focuses on investigating the efficacy of physiological indicators, including heart rate variability (HRV), percentage of eyelid closure over the pupil over time (PERCLOS), blink rate, blink percentage, and electrodermal activity (EDA) signals, in predicting driver drowsiness. The study was conducted with a cohort of 30 participants in controlled simulated driving scenarios, with half driving in a non-monotonous environment and the other half in a monotonous environment. Three deep learning algorithms were employed: sequential neural network (SNN) for HRV, 1D-convolutional neural network (1D-CNN) for EDA, and convolutional recurrent neural network (CRNN) for eye tracking. The HRV-Based Model and EDA-Based Model exhibited strong performance in drowsiness classification, with the HRV model achieving precision, recall, and F1-score of 98.28%, 98%, and 98%, respectively, and the EDA model achieving 96.32%, 96%, and 96% for the same metrics. The confusion matrix further illustrates the model's performance and highlights high accuracy in both HRV and EDA models, affirming their efficiency in detecting driver drowsiness. However, the Eye-Based Model faced difficulties in identifying drowsiness instances, potentially attributable to dataset imbalances and underrepresentation of specific fatigue states. Despite the challenges, this work significantly contributes to ongoing efforts to improve road safety by laying the foundation for effective real-time neuro-adaptive systems for drowsiness detection and mitigation.
Comparing eye–hand coordination between controller-mediated virtual reality, and a real-world object interaction task
Virtual reality (VR) technology has advanced significantly in recent years, with many potential applications. However, it is unclear how well VR simulations mimic real-world experiences, particularly in terms of eye–hand coordination. This study compares eye–hand coordination from a previously validated real-world object interaction task to the same task re-created in controller-mediated VR. We recorded eye and body movements and segmented participants’ gaze data using the movement data. In the real-world condition, participants wore a head-mounted eye tracker and motion capture markers and moved a pasta box into and out of a set of shelves. In the VR condition, participants wore a VR headset and moved a virtual box using handheld controllers. Unsurprisingly, VR participants took longer to complete the task. Before picking up or dropping off the box, participants in the real world visually fixated the box about half a second before their hand arrived at the area of action. This 500-ms minimum fixation time before the hand arrived was preserved in VR. Real-world participants disengaged their eyes from the box almost immediately after their hand initiated or terminated the interaction, but VR participants stayed fixated on the box for much longer after it was picked up or dropped off. We speculate that the limited haptic feedback during object interactions in VR forces users to maintain visual fixation on objects longer than in the real world, altering eye–hand coordination. These findings suggest that current VR technology does not replicate real-world experience in terms of eye–hand coordination.
