Project Overview
This project aims to develop enabling technologies for conversational Metahumans with social gaze behavior. By combining real-time speaker recognition, gaze modeling, and large language model (LLM)-based dialogue systems, we will create immersive and trustworthy human–Metahuman interactions.
•
Research Subject: Research on Enabling Technologies for Conversational Metahumans with
Social Gaze Behavior
•
Program: NRF Excellent Young Researcher (Seed Research)
•
Funding Agency: National Research Foundation of Korea (NRF)
•
Research Period: Sep 2025 – Aug 2026 (12 months)
•
Participants:
◦
PI: Prof. Hyun Ahn
◦
Researchers: Jin Hak Kim (M.S. student)
Research Background
As digital transformation accelerates, the demand for realistic digital humans is rapidly increasing across education, customer service, entertainment, and healthcare. Current Metahuman technologies provide high-fidelity appearance and speech capabilities, but they lack social interaction elements such as mutual gaze, gaze aversion, and joint attention.
Without these, users often fail to feel trust or presence in conversations with digital humans. To overcome this limitation, our research integrates theories from social psychology, HCI, and AI to implement social gaze behavior as a critical element of conversational Metahumans.
Research Objectives
•
Real-time Speaker Recognition: Detect user head pose, face, and gaze direction using lightweight RGB camera–based vision techniques.
•
Social Gaze Behavior Modeling: Implement natural gaze actions (mutual gaze, gaze aversion, joint attention) to enhance immersion and trust.
•
LLM-based Dialogue Pipeline: Build a Korean-specialized, low-latency conversational system (STT–LLM–TTS) for natural multi-turn interaction.
•
Integrated Metahuman System: Connect recognition, gaze, and dialogue modules with Unreal Engine to realize responsive, lifelike Metahumans.
