SCIENCE@home

Preliminary study on the feasibility of approximating children's engagement level from their emotions estimation by a picture-based, three-model AI in a family-robot cohabitation scenario

G. A. García,Guillermo Pérez,Rohan K. Laycock-Narayan,Leigh M Levinson,J. G. Amores,Gloria Alvarez-Benito,Manuel Castro-Malet,Mario Castaño-Ocaña,M. J. L. D. Quevedo,Ricardo Durán-Viñuelas,Randy Gomez,Selma Šabanović

Published 2024 in Adv. Robotics

ABSTRACT

Haru4Kids (H4K) is a system that emulates the physical, social, family-oriented robot Haru, designed with the goal of cohabitating with children in their homes for extended periods of time. In a previous experiment [Garcia GA, Perez G, Levinson L, et al. Living with Haru4Kids: study on children's activity and engagement in a family-robot cohabitation scenario. In: 2023 IEEE ROMAN; 2023 Aug. p. 1428–1435], seven families kept H4K for a span of two weeks in their homes. Throughout this period of cohabitation, we collected child-robot interaction data, including images that were later hand-annotated to estimate user engagement. In this present work, we used a novel AI-based, four-stage framework available from Roboflow for the automatic estimation of children's level of engagement from their inferred emotions. We did a deep study of the performance and behaviour of that framework over our dataset of users' pictures and characterized its response in order to understand its advantages and limitations, including the technique used to translate emotions into engagement levels. We also tested a different approach for that mapping, using a machine learning technique based on Support Vector Machine (SVM). The framework yielded promising results just ‘off-the-shelf’: 0.47–0.68 accuracy and 0.46–0.70 F1 using the original mapping, and 0.39–0.75 and 0.37–0.78 respectively using SVM. Therefore, we propose this emotion-based approach for engagement estimation from pictures. GRAPHICAL ABSTRACT

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-65 of 65 references · Page 1 of 1

CITED BY

  • No citing papers are available for this paper.

Showing 0-0 of 0 citing papers · Page 1 of 1