Cover not available
Article published In:
Interaction Studies
Vol. 19:3 (2018) ► pp.487–498

Frustration in the face of the driver

A simulator study on facial muscle activity during frustrated driving

| Institute of Transportation Systems, German Aerospace Center
| Institute of Transportation Systems, German Aerospace Center
| Institute of Flight Guidance, German Aerospace Center
| Institute of Transportation Systems, German Aerospace Center

Frustration in traffic is one of the causes of aggressive driving. Knowledge whether a driver is frustrated may be utilized by future advanced driver assistance systems to counteract this source of crashes. One possibility to achieve this is to automatically recognize facial expressions of drivers. However, only little is known about the facial expressions of frustrated drivers. Here, we report the results of a driving simulator study investigating the facial muscle activity that comes along with frustration. Twenty-eight participants were video-taped during frustrated and non-frustrated driving situations. Their facial muscle activity was manually coded according to the Facial Action Coding System. Participants showed significantly more facial muscle activity in the mouth region. Thus, recording facial muscle behavior potentially provides traffic researchers and assistance system developers with the possibility to recognize frustration while driving.

Keywords: frustration, Facial Action Coding System, driving simulator, facial expressions
Article outline
  • Methods
    • Participants
    • Experimental set-up
    • Experimental design and cover story
    • Subjective ratings
    • Video coding
  • Results
    • Ratings
    • Activity in facial AUs
  • Discussion
  • Acknowledgements
  • References
Published online: 13 March 2019
DOI logo
https://doi.org/10.1075/is.17005.ihm
References
Barrett, L. F.
(2016) The theory of constructed emotion: an active inference account of interoception and categorization. Social Cognitive and Affective Neuroscience, nsw154. DOI logoGoogle Scholar
Bruce, V.
(1992) What the human face tells the human mind: some challenges for the robot-human interface. In Proceedings of IEEE International Workshop on Robot and Human Communication (pp. 44–51). DOI logoGoogle Scholar
Bradley, M. M., & Lang, P. J.
(1994) Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. DOI logoGoogle Scholar
D’Mello, S., Craig, S., Gholson, B., Franklin, S., Picard, R., & Graesser, A.
(2005) Integrating affect sensors in an intelligent tutoring system. In Affective Interactions: The Computer in the Affective Loop Workshop at 2005 International Conference on Intelligent User Interfaces. New York: AMC Press.Google Scholar
Deffenbacher, J. L., Lynch, R. S., Oetting, E. R., & Swaim, R. C.
(2002) The Driving Anger Expression Inventory: a measure of how people express their anger on the road. Behaviour Research and Therapy, 40(6), 717–737. DOI logoGoogle Scholar
Donkor, R., Burnett, G., & Sharples, S.
(2014) Measuring the emotional validity of driving simulators. Advances in Transportation Studies, (Special, Issue Special Vol1), 51–64.Google Scholar
Ekman, P.
(1992) An argument for basic emotions. Cognition & Emotion, 6(3), 169–200. DOI logoGoogle Scholar
Ekman, P., & Friesen, W. V.
(2003) Unmasking the face: A guide to recognizing emotions from facial clues. Cambridge, MA: Malor Books.Google Scholar
Ekman, P., Friesen, W. V., & Hager, J.
(2002) The Investigator’s Guide for the Facial Action Coding System. Salt Lake City: A Human face.Google Scholar
Gao, H., Yuce, A., & Thiran, J.-P.
(2014) Detecting emotional stress from facial expressions for driving safety. In IEEE International Conference on Image Processing (ICIP) (pp. 5961–5965). DOI logoGoogle Scholar
Gehrig, T., & Ekenel, H. K.
(2011) A common framework for real-time emotion recognition and facial action unit detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops) (pp. 1–6).Google Scholar
Gosselin, P., Perron, M., & Beaupré, M.
(2010) The voluntary control of facial action units in adults. Emotion, 10(2), 266–271. DOI logoGoogle Scholar
Grafsgaard, J. F., Wiggins, J. B., Boyer, K. E., Wiebe, E. N., & Lester, J. C.
(2013) Automatically recognizing facial indicators of frustration: A learning-centric analysis. In Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 159–165). DOI logoGoogle Scholar
Hamm, J., Kohler, C. G., Gur, R. C., & Verma, R.
(2011) Automated Facial Action Coding System for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200(2), 237–256. DOI logoGoogle Scholar
Hart, S. G., & Staveland, L.
(1988) Development of the NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 521, 139–183. DOI logoGoogle Scholar
Healey, J., & Picard, R.
(2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6(2), 156–166. DOI logoGoogle Scholar
Hoque, M. E., McDuff, D. J., & Picard, R. W.
(2012) Exploring temporal patterns in classifying frustrated and delighted smiles. IEEE Transactions on Affective Computing, 3(3), 323–334. DOI logoGoogle Scholar
Lazarus, R. S.
(1991) Progress on a cognitive-motivational-relational theory of emotion. American Psychologist, 46(8), 819–834. DOI logoGoogle Scholar
Lee, Y.-C.
(2010) Measuring drivers’ frustration in a driving simulator. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54(19), 1531–1535. DOI logoGoogle Scholar
Lee, Y.-C., & LaVoie, N.
(2014) Relationship between frustration justification and vehicle control behaviors ? A simulator study. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58(1), 2235–2239. DOI logoGoogle Scholar
Malta, L., Miyajima, C., Kitaoka, N., & Takeda, K.
(2011) Analysis of real-world driver’s frustration. IEEE Transactions on Intelligent Transportation Systems, 12(1), 109–118. DOI logoGoogle Scholar
Russell, J. A.
(1980) A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. DOI logoGoogle Scholar
Scherer, K. R.
(2005) What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729. DOI logoGoogle Scholar
Tews, T.-K., Oehl, M., Siebert, F. W., Höger, R., & Faasch, H.
(2011) Emotional human-machine interaction: cues from facial expressions. In D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell, … (Eds.), Lecture Notes in Computer Science. Human Interface and the Management of Information. Interacting with Information (Vol. 67711, pp. 641–650). Berlin: Springer. DOI logoGoogle Scholar
Cited by

Cited by 14 other publications

Bustos, Cristina, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza & Rosalind Picard
2021. 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), ► pp. 1 ff. DOI logo
Franz, Oliver, Uwe Drewitz & Klas Ihme
2020. Facing Driver Frustration: Towards Real-Time In-Vehicle Frustration Estimation Based on Video Streams of the Face. In HCI International 2020 - Posters [Communications in Computer and Information Science, 1226], ► pp. 349 ff. DOI logo
Jipp, Meike & Jochen Steil
2021. Steuern wir oder werden wir gesteuert? Chancen und Risiken von Mensch-Technik-Interaktion. In Zusammenwirken von natürlicher und künstlicher Intelligenz, ► pp. 17 ff. DOI logo
Krüger, Sandra, Esther Bosch, Klas Ihme & Michael Oehl
2021. In-Vehicle Frustration Mitigation via Voice-User Interfaces – A Simulator Study. In HCI International 2021 - Posters [Communications in Computer and Information Science, 1421], ► pp. 241 ff. DOI logo
Lemmer, Karsten, Meike Jipp, Heiner Bubb, Hans-Jörg Vögel, Matthias Jung, Georg Laukart & Thomas Vorberg
2021. Mensch-Technik-Kooperation und Fahrzeuginnenraum. In Vieweg Handbuch Kraftfahrzeugtechnik, ► pp. 1161 ff. DOI logo
Nadri, Chihab, Jingyi Li, Esther Bosch, Michael Oehl, Ignacio Alvarez, Michael Braun & Myounghoon Jeon
2020. 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ► pp. 106 ff. DOI logo
Ortoncelli, Andre Roberto, Luciano Silva, Olga Regina Perreira Bellon, Tiago Mota de Oliveira & Juliana Daga
2020. 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), ► pp. 587 ff. DOI logo
Pape, Anna-Antonia, Sonja Cornelsen, Victor Faeßler, Klas Ihme, Michael Oehl, Uwe Drewitz, Franziska Hartwich, Frank Schrödel, Andreas Lüdtke & Martin Schramm
2020. Empathic assistants – Methods and use cases in automated and non-automated driving. In 20. Internationales Stuttgarter Symposium [Proceedings, ], ► pp. 435 ff. DOI logo
Siddiqi, Muhammad Hameed, Khalil Khan, Rehan Ullah Khan & Amjad Alsirhani
2022. Face Image Analysis Using Machine Learning: A Survey on Recent Trends and Applications. Electronics 11:8 ► pp. 1210 ff. DOI logo
Sukhavasi, Susrutha Babu, Suparshya Babu Sukhavasi, Khaled Elleithy, Ahmed El-Sayed & Abdelrahman Elleithy
2022. Deep Neural Network Approach for Pose, Illumination, and Occlusion Invariant Driver Emotion Detection. International Journal of Environmental Research and Public Health 19:4 ► pp. 2352 ff. DOI logo
Weidemann, Alexandra & Nele Russwinkel
2019. Proceedings of Mensch und Computer 2019, ► pp. 819 ff. DOI logo
Weidemann, Alexandra & Nele Rußwinkel
2021. The Role of Frustration in Human–Robot Interaction – What Is Needed for a Successful Collaboration?. Frontiers in Psychology 12 DOI logo
Zepf, Sebastian, Tobias Stracke, Alexander Schmitt, Florian van de Camp & Juergen Beyerer
2019. 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), ► pp. 202 ff. DOI logo
Zhou, Xin, Liang Ma & Wei Zhang
2022. Event-related driver stress detection with smartphones among young novice drivers. Ergonomics ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 15 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.