UTokyo-IIS researchers will introduce their research.

In recent years, gaze estimation technology, which interprets the direction of a person’s gaze from camera images, has made significant progress and is anticipated to be applied in various fields, such as autonomous driving, robotics, and medical applications. However, a significant challenge has been the bias in the training data necessary for AI to learn gaze estimation. For instance, many existing research datasets are collected in university laboratories, predominantly featuring student participants, resulting in a significant age bias.

In response, Associate Professor Yusuke Sugano’s laboratory at the Institute of Industrial Science, the University of Tokyo, and the DLX Design Lab have proposed a novel approach that transforms the collection of AI training data into an enjoyable gaming experience and demonstrated its effectiveness. The game developed by the research team involves two players facing each other with a transparent board between them, communicating solely through gaze directions. Through this gaming experience, facial images and gaze direction data for training gaze estimation AI are naturally collected. The aim is for participants to discover the potential of the technology while contributing to data collection through an engaging and enjoyable game.

AICOM / DLX Design Lab + Sugano Lab, IIS, The University of Tokyo

The experiment was conducted over five days at the “Dining Lab/Shokudo Komani ” on the University of Tokyo’s Komaba Research Campus. A diverse group of 47 participants, aged 18 to 79, took part, enjoying the opportunity to engage with cutting-edge research during meals or breaks. In a survey conducted after the experiment, many participants reported enjoying the game and gaining a deeper understanding of gaze estimation technology. Furthermore, the accuracy of the gaze labels in the data collected, crucial for training models, was comparable to traditional laboratory methods.

Experiments conducted in the “Dining Lab” Credit: ONESTORY

The fact that this experiment was conducted in a dining lab visited by not only students and researchers but also a diverse range of people holds significant importance. This research project aims for diverse participants to enjoy contributing to AI training data collection outside the laboratory. Evaluating the system created in this way only with specially gathered participants in a lab setting would be meaningless. By assessing it in a format that includes the general public, a more realistic verification aligned with the original intent of the research was made possible. As the role of AI technology increasingly expands in our daily lives, research activities in places like the dining lab suggest new possibilities for the future of scientific and technological research.

This initiative presents a novel way of connecting AI research with society. It offers valuable insights into the relationship between AI and people by allowing data to be provided enjoyably while simultaneously deepening understanding of the technology. Future prospects include introducing automatic game difficulty adjustment features, long-term public exhibitions, and combinations with workshops. Moreover, this method of collecting training data through games could be applied to AI research fields beyond gaze estimation, potentially having a wide-ranging impact.

This study was published in the International Journal of Human-Computer Interaction on September 12, 2024.
DOI：https://doi.org/10.1080/10447318.2024.2399873

Related Link> DLX Design Lab
Related Link> Dining Lab

Text: Associate Professor Yusuke Sugano