X's Day: Personality-Driven Virtual Human Behavior Generation

IEEE TVCG (Special Issue on IEEE VR 2025)

Haoyang Li1, Zan Wang1, Wei Liang1, 2†, Yizhuo Wang1†
1 Beijing Institute of Technology
 2 Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing
†indicates corresponding authors
Paper Supplementary Code Video
↓

X's day. Our approach generates a one-day activity sequence for the virtual human X by integrating personality traits, attributes, and 3D scene information. The demo video illustrates a sequence of activities performed in the morning.

Abstract

Developing convincing and realistic virtual human behavior is essential for enhancing user experiences in virtual reality (VR) and augmented reality (AR) settings. This paper introduces a novel task focused on generating long-term behaviors for virtual agents, guided by specific personality traits and contextual elements within 3D environments. We present a comprehensive framework capable of autonomously producing daily activities autoregressively. By modeling the intricate connections between personality characteristics and observable activities, we establish a hierarchical structure of Needs, Task, and Activity levels. Integrating a Behavior Planner and a World State module allows for the dynamic sampling of behaviors using large language models (LLMs), ensuring that generated activities remain relevant and responsive to environmental changes. Extensive experiments validate the effectiveness and adaptability of our approach across diverse scenarios. This research makes a significant contribution to the field by establishing a new paradigm for personalized and context-aware interactions with virtual humans, ultimately enhancing user engagement in immersive applications.

Overview

PIPELINE

Overview of our framework. (a) The input comprises the virtual human's personality, attributes, and 3D scene information. We utilize the Big Five personality traits to represent the personality, while the attributes described by natural language encompass physiological traits, social characteristics, hobbies, and preferences. The 3D scene information includes scene layout (represented by orange and blue dots), object relations (indicated by yellow dots), and the spatial positioning of the virtual human and objects. (b) In the generator, the World State continuously monitors the environmental status, while the Behavior Planner generates activities autoregressively. (c) The output is an activity sequence, with the activity labeled in blue indicating the current activity, while the others represent completed activities.

BibTeX


@article{li2025x,
  title= {X's Day: Personality-Driven Virtual Human Behavior Generation},
  author = {Li, Haoyang and Wang, Zan and Liang, Wei and Wang, Yizhuo},
  journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)},
  year = {2025},
  publisher = {IEEE}
}