Facial Paresis Analysis
Team
Tim Büchner, Oliver Mothes, Sven Sickert
Introduction

The human face is a complex and diverse part of the human body. All facial motions for individual mimics and emotional expressions are possible using 43 different muscles controlled respectively by one facial nerve on both halves of the face. However, this nerve can be injured by external or internal factors, which in turn can cause a facial palsy. A unilateral facial palsy essentially leads to a one-sided motor dysfunction of the facial muscles, a movement disorder. The resulting defective eye closure leads to the drying out of the eye, eye inflammation and visual impairment. The paralysis of the perioral musculature hinders regular dient and speech. Medical experts evaluate facial palsy in a subjective manner. But ideally, these functional deficits should be measured objectively by exploiting machine learning methods. The following projects tackle this task.

Project: Bridging the Gap – Mimics and Muscles
Time frame: 2019 – 2023

The aim of the project is the synthesis of the facial surface model with a model of the facial muscles. High-resolution 3D video sequences, which are recorded synchronously together with electromyography sequences, are intended to facilitate understanding of the facial muscles. The resulting model should be used for an automatic objective evaluation of facial palsy.

Project: Irritationsfreies und emotionssensitives Trainingssystem (IRESTRA)
Time frame: 2016 – 2019

The interaction between humans and machines is an ongoing research field in many scientific disciplines. To develop an optimal strategy for machines to interact with humans the interactions between humans can be used to detect signals for the mood of the interaction partner and react in an adequate way. Especially elderly people can benefit from an easy and intuitive handling of technical devices. The aim of the project is to develop an irritation free and non invasive emotion sensitive training device for elderly people and Facial Paresis patients. Older persons can use this system for memory training (e.g. person recognition), whereas paresis patients can do their daily facial training sessions with a direct feedback from the machine and motivation if necessary. Within this project these tasks are tackled by combining medical, psychological and neurological knowledge with machine learning and computer vision techniques both in 2D and 3D in coorperation with the Fraunhofer Institute for Applied Optics and Precision Engineering IOF, the Jena University Hospital (namely the Institute of Psychosocial Medicine and Psychological Therapy, the Department of Neurology and the Department of Otolaryngology) and the Department for General Psychology from the Friedrich Schiller University Jena.

Publications
2024
Tim Büchner, Oliver Mothes, Orlando Guntinas-Lichius, Joachim Denzler:
JeFaPaTo - A Joint Toolbox for Blinking Analysis and Facial Features Extraction.
arXiv preprint 2402.08439. 2024. (submitted to JOSS)
[bibtex] [pdf] [doi] [code] [abstract]
2023
Lena Mers, Oliver Mothes, Joachim Denzler, Orlando Guntinas-Lichius, Christian Dobel:
Der zeitliche Verlauf des emotionalen menschlichen Gesichtsausdruckes - die Entwicklung eines künstliche Intelligenz basierten Paradigmas zur Quantifizierung.
94. Jahresversammlung der Deutschen Gesellschaft für Hals-Nasen-Ohren-Heilkunde, Kopf- und Hals-Chirurgie e.V., Bonn. 2023.
[bibtex] [web] [doi] [abstract]
Tim Büchner, Orlando Guntinas-Lichius, Joachim Denzler:
Improved Obstructed Facial Feature Reconstruction for Emotion Recognition with Minimal Change CycleGANs.
Advanced Concepts for Intelligent Vision Systems (Acivs). Pages 262-274. 2023. Best Paper Award
[bibtex] [web] [doi] [abstract]
Tim Büchner, Sven Sickert, Gerd F. Volk, Christoph Anders, Orlando Guntinas-Lichius, Joachim Denzler:
Let’s Get the FACS Straight - Reconstructing Obstructed Facial Features.
International Conference on Computer Vision Theory and Applications (VISAPP). Pages 727-736. 2023.
[bibtex] [pdf] [web] [doi] [abstract]
Tim Büchner, Sven Sickert, Gerd F. Volk, Orlando Guntinas-Lichius, Joachim Denzler:
From Faces To Volumes - Measuring Volumetric Asymmetry in 3D Facial Palsy Scans.
International Symposium on Visual Computing (ISVC). Pages 121-132. 2023. Best Paper Award
[bibtex] [web] [doi] [abstract]
Tim Büchner, Sven Sickert, Roland Graßme, Christoph Anders, Orlando Guntinas-Lichius, Joachim Denzler:
Using 2D and 3D Face Representations to Generate Comprehensive Facial Electromyography Intensity Maps.
International Symposium on Visual Computing (ISVC). Pages 136-147. 2023.
[bibtex] [web] [doi] [code] [abstract]
2022
Gabriel Meincke, Johannes Krauß, Maren Geitner, Dirk Arnold, Anna-Maria Kuttenreich, Valeria Mastryukova, Jan Beckmann, Wengelawit Misikire, Tim Büchner, Joachim Denzler, Orlando Guntinas-Lichius, Gerd F. Volk:
Surface Electrostimulation Prevents Denervated Muscle Atrophy in Facial Paralysis: Ultrasound Quantification [Abstract].
Abstracts of the 2022 Joint Annual Conference of the Austrian (ÖGBMT), German (VDE DGBMT) and Swiss (SSBE) Societies for Biomedical Engineering, including the 14th Vienna International Workshop on Functional Electrical Stimulation. 67 (s1) : pp. 542. 2022.
[bibtex] [doi] [abstract]
Johannes Krauß, Gabriel Meincke, Maren Geitner, Dirk Arnold, Anna-Maria Kuttenreich, Valeria Mastryukova, Jan Beckmann, Wengelawit Misikire, Tim Büchner, Joachim Denzler, Orlando Guntinas-Lichius, Gerd F. Volk:
Optical Quantification of Surface Electrical Stimulation to Prevent Denervation Muscle Atrophy in 15 Patients with Facial Paralysis [Abstract].
Abstracts of the 2022 Joint Annual Conference of the Austrian (ÖGBMT), German (VDE DGBMT) and Swiss (SSBE) Societies for Biomedical Engineering, including the 14th Vienna International Workshop on Functional Electrical Stimulation. 67 (s1) : pp. 541. 2022.
[bibtex] [doi] [abstract]
Tim Büchner, Sven Sickert, Gerd F. Volk, Orlando Guntinas-Lichius, Joachim Denzler:
Automatic Objective Severity Grading of Peripheral Facial Palsy Using 3D Radial Curves Extracted from Point Clouds.
Challenges of Trustable AI and Added-Value on Health. Pages 179-183. 2022.
[bibtex] [web] [doi] [code] [abstract]
2020
Anish Raj, Oliver Mothes, Sven Sickert, Gerd F. Volk, Orlando Guntinas-Lichius, Joachim Denzler:
Automatic and Objective Facial Palsy Grading Index Prediction using Deep Feature Regression.
Annual Conference on Medical Image Understanding and Analysis (MIUA). Pages 253-266. 2020.
[bibtex] [pdf] [web] [doi] [abstract]
Gerd F. Volk, Maren Geitner, Katharina Geißler, Jovanna Thielker, Ashraf Raslan, Oliver Mothes, Christian Dobel, Orlando Guntinas-Lichius:
Functional outcome and quality of life after hypoglossal-facial jump nerve suture.
Frontiers Surgery - Otorhinolaryngology - Head and Neck Surgery. 2020.
[bibtex] [abstract]
2019
Gerd F. Volk, Martin Thümmel, Oliver Mothes, Dirk Arnold, Jovanna Thielker, Joachim Denzler, Valeria Mastryukova, Winfried Mayr, Orlando Guntinas-Lichius:
Long-term home-based Surface Electrostimulation is useful to prevent atrophy in denervated Facial Muscles.
Vienna Workshop on Functional Electrical Stimulation (FESWS). 2019.
[bibtex] [pdf] [abstract]
Oliver Mothes, Luise Modersohn, Gerd F. Volk, Carsten Klingner, Otto W. Witte, Peter Schlattmann, Joachim Denzler, Orlando Guntinas-Lichius:
Automated objective and marker-free facial grading using photographs of patients with facial palsy..
European Archives of Oto-Rhino-Laryngology. 2019.
[bibtex] [pdf]
Orlando Guntinas-Lichius, Oliver Mothes, Gerd F. Volk, Carsten M. Klingner, Otto W. Witte, Peter Schlattmann, Joachim Denzler:
Machine Learning based Classification of Facial Palsies using Standard Still Photografies.
Laryngo-Rhino-Otologie. 98 (S02) : pp. 360. 2019.
[bibtex] [web] [doi]
2018
Gerd F. Volk, Anika Steinerstauch, Annegret Lorenz, Luise Modersohn, Oliver Mothes, Joachim Denzler, Carsten M. Klingner, Farsin Hamzei, Orlando Guntinas-Lichius:
Facial motor and non-motor disabilities in patients with central facial paresis: a prospective cohort study.
Journal of Neurology. 2018.
[bibtex]
2017
Rebecca Anna Schaede, Gerd F. Volk, Luise Modersohn, Jodie M. Barth, Joachim Denzler, Orlando Guntinas-Lichius:
Video Instruction for Synchronous Video Recording of Mimic Movement of Patients with Facial Palsy.
Laryngo-Rhino-Otologie. 2017.
[bibtex] [web] [doi] [abstract]
2016
Luise Modersohn, Joachim Denzler:
Facial Paresis Index Prediction by Exploiting Active Appearance Models for Compact Discriminative Features.
International Conference on Computer Vision Theory and Applications (VISAPP). Pages 271-278. 2016.
[bibtex] [pdf] [abstract]