Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration
Quan Wen1
Derek Bradley2
Thabo Beeler2
Seonwook Park3
Otmar Hilliges3
1 Tsinghua University
2 Disney Research
3 ETH Zurich
3D gaze tracking from a single RGB camera is very challenging due to the lack of information in determining the accurate gaze target from a monocular RGB sequence. The eyes tend to occupy only a small portion of the video, and even small errors in estimated eye orientations can lead to very large errors in the triangulated gaze target. We overcome these difficulties with a novel lightweight eyeball calibration scheme that determines the user-specific visual axis, eyeball size and position in the head. Unlike the previous calibration techniques, we do not need the ground truth positions of the gaze points. In the online stage, gaze is tracked by a new gaze fitting algorithm, and refined by a 3D gaze regression method to correct for bias errors. Our regression is pre-trained on several individuals and works well for novel users. After the lightweight one-time user calibration, our method operates in real time. Experiments show that our technique achieves state-of-the-art accuracy in gaze angle estimation, and we demonstrate applications of 3D gaze target tracking and gaze retargeting to an animated 3D character.
Note
@article{wen2020accurate,
title={Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration},
author={Wen, Quan and Bradley, Derek and Beeler, Thabo and Park, Seonwook and Hilliges, Otmar and Xu, Feng and Yong, Jun-Hai},
journal={Computer Graphics Forum},
year={2020}
}
title={Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration},
author={Wen, Quan and Bradley, Derek and Beeler, Thabo and Park, Seonwook and Hilliges, Otmar and Xu, Feng and Yong, Jun-Hai},
journal={Computer Graphics Forum},
year={2020}
}
Quan Wen, Derek Bradley, Thabo Beeler, Seonwook Park, Otmar Hilliges, Feng Xu, Jun-Hai Yong. 2020. "Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration". Computer Graphics Forum. (Proceedings of Eurographics 2020).
Abstract
Files
[video]
[paper]
The gaze dataset is available for non-commercial research purposes only. Please contact wenq013@gmail.com or feng-xu@tsinghua.edu.cn for the full dataset.
You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the models, any portion of the images and any portion of the derived data.
You agree not to further copy, publish or distribute any portion of the gaze dataset.
You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the models, any portion of the images and any portion of the derived data.
You agree not to further copy, publish or distribute any portion of the gaze dataset.
Citation