Driver state monitoring using consumer electronic devices: Innovation report
An impaired mental and physical state such as fatigue, high level of workload, or distraction, can make a driver prone to errors and lead to sub-optimal driving performance. If a human remains in full or partial control of a vehicle, drivers’ state is an important aspect of driving and cannot be neglected, given its significant impact on road safety. It is, therefore, beneficial for future automobiles to be fitted with a feature which enables detection of any physical and mental abnormalities in drivers’ state in real time using physiological and emotional indicators. Such a feature is often referred to as Driver State Monitoring (DSM) system. It is forecasted that DSM is expected to become a standard passenger car feature by 2025 and its integration is encouraged by the standards authorities. Previous research has predominantly considered the use of medical grade devices for the purpose of DSM. Instead, this research project has considered the potential use of Consumer Electronic Devices (CEDs) as part of DSM. However, the literature lacks evidence that this can be accomplished in a valid and reliable manner. Thus, the research, presented in this doctorate, aims to provide knowledge that describes feasibility and integration of CEDs into the vehicles for the purpose of DSM, from both technological and human factors perspectives. Firstly, this research project has produced a model of a hybrid DSM system. The model combines physiological and emotional sensing within CEDs. This can be used to enhance validity and reliability of DSM in a flexible and cost-efficient manner. The model also acknowledges barriers of introduction of hybrid DSM into the automotive market. Acceptance, one of the important adoption barriers of DSM technology, was studied and behaviour intention to use the system was statistically appraised using the Unified Theory of Acceptance and Use of Technology (UTAUT) model. It was found that social influence is a significant factor affecting drivers’ behaviour intention to use hybrid DSM in the near future. On the other hand, it was demonstrated that there is no significant negative attitude towards the use of hybrid DSM technology due to apprehension, intimidation, or fear of making mistakes. These findings indicate viability of DSM in the driving context. To further deepen understanding of CED-based DSM, three driving simulator user trials were conducted. Overall, supporting evidence for adoption of CEDs in DSM was provided by utilising state of the art methodology in DSM while characterising sensory capabilities of CEDs. The studies were specifically aiming to (1) determine the reliability and validity of wearable CEDs to measure human physiology while driving, (2) provide supporting evidence for employing CEDs in physiological and emotional evaluation of common driving activities, and (3) explore the effect of cognitive and visual workload on drivers’ state and driving performance during the automated to manual control transition scenarios. All three studies have demonstrated evidence of CEDs being well suited to reliably monitor drivers’ state. For instance, it was demonstrated how an extent of workload can be reliably measured using heart rate variability, captured by means of CEDs in the driving context. This approach could enable cost-efficient access to drivers’ state outside of driving activities. To facilitate this, a modular and cost-effective mobile DSM toolkit was designed and developed in-house. The toolkit enabled driver-state-related data collection, filtering, on-board analysis, storage, and synchronisation. It can be concluded that this EngD has successfully demonstrated that CEDs can be used for the purpose of DSM.
Effects of mental demands on situation awareness during platooning: A driving simulator study
Previous research shows that drivers of automated vehicles are likely to engage in visually demanding tasks, causing impaired situation awareness. How mental task demands affect situation awareness is less clear. In a driving simulator experiment, 33 participants completed three 40-min runs in an automated platoon, each run with a different level of mental task demands. Results showed that high task demands (i.e., performing a 2-back task, a working memory task in which participants had to recall a letter, presented two letters ago) induced high self-reported mental demands (71% on the NASA Task Load Index), while participants reported low levels of self-reported task engagement (measured with the Dundee Stress State Questionnaire) in all three task conditions in comparison to the pre-task measurement. Participants’ situation awareness, as measured using a think-out-loud protocol, was affected by mental task demands, with participants being more involved with the mental task itself (i.e., to remember letters) and less likely to comment on situational features (e.g., car, looking, overtaking) when task demands increased. Furthermore, our results shed light on temporal effects, with heart rate decreasing and self-constructed mental models of automation growing in complexity, with run number. It is concluded that mental task demands reduce situation awareness, and that not only type-of-task, but also time-on-task, should be considered in Human Factors research of automated driving.
Effects of searching for street parking on driver behaviour, physiology, and visual attention allocation: An on-road study
On-street parking is a major aspect of the urban street system, and entails important costs to drivers, in terms of time, inconvenience and energy to find a parking space. The efforts needed to park on-street can create frustrations and stress among drivers, which can contribute to unsafe driving behavior and ultimately affect road safety. Additionally, the search for available spaces creates disturbances to traffic, delays to other vehicles, and increased pollution due to extra fuel consumption. In this study, the effects of searching for on-street parking on driver behavior, physiology, and visual attention allocation were investigated using an on-road experimental approach. A total of 32 drivers participated in the study, during which their driving behavior, physiological responses, and visual attention were monitored while they searched for parking on urban streets in Toronto, Canada. Results indicated that searching for on-street parking led to significant changes in drivers' behavior, including reduced speeds and abrupt stops, as well as increases in physiological stress markers and visual attention diversion. Understanding these effects is crucial for urban traffic management and designing policies to mitigate the negative impacts associated with on-street parking search.
Effects of written peer-feedback content and sender’s competence on perceptions, performance, and mindful cognitive processing
Peer-feedback efficiency might be influenced by the oftentimes voiced concern of students that they perceive their peers’ competence to provide feedback as inadequate. Feedback literature also identifies mindful processing of (peer)feedback and (peer)feedback content as important for its efficiency, but lacks systematic investigation. In a 2 × 2 factorial design, peer-feedback content (concise general feedback [CGF] vs. elaborated specific feedback [ESF]) and competence of the sender (high vs. low) were varied. Students received a scenario containing an essay by a fictional student and fictional peer feedback, a perception questionnaire, and a text revision, distraction, and peer-feedback recall task. Eye tracking was applied to measure how written peer feedback was (re-)read, e.g., glance duration on exact words and sentences. Mindful cognitive processing was inferred from the relation between glance duration and (a) text-revision performance and (b) peer-feedback recall performance. Feedback by a high competent peer was perceived as more adequate. Compared to CGF, participants who received ESF scored higher on positive affect towards the peer feedback. No effects were found for peer-feedback content and/or sender’s competence level on performance. Glance durations were negatively correlated to text-revision performance regardless of condition, although peer-feedback recall showed that a basic amount of mindful cognitive processing occurred in all conditions. Descriptive findings also hint that this processing might be dependent on an interaction between peer-feedback content and sender’s competence, signifying a clear direction for future research.
Ensuring the take-over readiness of the driver based on the gaze behavior in conditionally automated driving scenarios
Conditional automation is the next step towards the fully automated vehicle. Under prespecified conditions an automated driving function can take-over the driving task and the responsibility for the vehicle, thus enabling the driver to perform secondary tasks. However, performing secondary tasks and the resulting reduced attention towards the road may lead to critical situations in take-over situations. In such situations, the automated driving function reaches its limits, forcing the driver to take-over responsibility and the control of the vehicle again. Thus, the driver represents the fallback level for the conditionally automated system. At this point the question arises as to how it can be ensured that the driver can take-over adequately and timely without restricting the automated driving system or the new freedom of the driver. To answer this question, this work proposes a novel prototype for an advanced driver assistance system which is able to automatically classify the driver’s take-over readiness for keeping the driver ”in-the-loop”. The results show the feasibility of such a classification of the take-over readiness even in the highly dynamic vehicle environment using a machine learning approach. It was verified that far more than half of the drivers performing a low-quality take-over would have been warned shortly before the actual take-over, whereas nearly 90% of the drivers performing a high-quality take-over would not have been interrupted by the driver assistance system during a driving simulator study. The classification of the take-over readiness of the driver is performed by means of machine learning algorithms. The underlying features for this classification are mainly based on the head and eye movement behavior of the driver. It is shown how the secondary tasks currently being performed as well as the glances on the road can be derived from these measured signals. Therefore, novel, online-capable approaches for driver-activity recognition and Eyes-on-Road detection are introduced, evaluated, and compared to each other based on both data of a simulator and real-driving study. These novel approaches are able to deal with multiple challenges of current state-of-the-art methods such as: i) only a coarse separation of driver activities possible, ii) necessity for costly and time-consuming calibrations, and iii) no adaption to conditionally automated driving scenarios.
Evaluation of Decision Making Processes in Critical Situations
Emergency services, policemen and members of the armed forces act in high workload environments where fast and correct decisions are essential. Therefore, decision criterions and automated processes play an important role in drill and training. The evaluation of the trainees’ performance is usually based on subjective ratings by the instructors, which depend on the availability of many instructors to provide individual feedback. The goal of our ongoing research work is to develop evaluation techniques that enable objective ratings for the trainees’ performance. As gaze behavior is seen as key element of trainees’ performance, tracing methods are evaluated and head-mounted eye tracking is ascertained as promising. Furthermore, we give an overview about ongoing work, including software (laboratory experiment) and hardware development (field experiment) of an objective rating tool.
Eye tracking in naturalistic badminton play: comparing visual gaze pattern strategy in world-rank and amateur player
A professional player's expertise rests on the ability to predict action by optimally extracting the opponent's postural cues. Eye tracking (head-mounted system) data in a naturalistic singles badminton play was collected from one professional world-ranked player facing five amateur players (10 serves or 50 trials) and two amateurs playing against four other amateur players each (10 serves or 80 trials). The visual gaze on the opponent body, segregated into 3 areas-of-interest covering the feet, face/torso, and hand/racket of the opponent and the shuttle, was analysed for a) the period just before the serve, b) while receiving the serve and c) the entire rally. The comparative analysis shows the first area-of-interest for professional player as the opponent's feet while executing the serve and the hand/racket when receiving a serve. On the other hand, the amateur players show no particular strategy of fixation location either for the serve task or while facing a serve. The average fixation duration (just before serve) for the professional was 0.96s and for the amateurs it was 1.48s. The findings highlight the differences in the postural cue considered important and the preparatory time in professional and amateur players. We believe, analytical models from dynamic gaze behavior in naturalistic game conditions as applied in this study can be used for enhancing perceptual-cognitive skills during training.
