SCIENCE@home

Culture-Aware Multimodal Personality Prediction using Audio, Pose, and Cultural Embeddings

Islam J A M Samiul,Khalid Zaman,Marius Funk,Masashi Unoki,Yukiko I. Nakano,Shogo Okada

Published 2025 in ICMI Companion

ABSTRACT

Personality trait prediction from behavioral data is a growing challenge in affective computing, particularly across culturally diverse populations where expressions of traits like Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism vary significantly. Existing methods often assume cultural universality, limiting their generalizability. In this study, we introduce a culture-aware multimodal framework that integrates audio and skeletal pose characteristics with cultural embeddings. We evaluated three modeling strategies: a culture-agnostic model trained on pooled data, culture-specific models trained within individual cultural groups, and a culture-conditioned model that incorporates culture as an embedding vector into the network. All models utilize a shared LSTM-based fusion backbone. Experiments on data from five regions—France, China, Japan, England, and Germany—show that culture-specific models outperform others in intra-cultural settings but fail to generalize. The culture-conditioned model achieves stable, competitive performance across all test sets, especially improving results in low-resource cultures.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-22 of 22 references · Page 1 of 1

CITED BY