Egocentric World Models
Learning how dynamic environments evolve from first-person video, and how an agent can predict and act within them.
Computer Vision · Generative AI
Zhenyu Xie
Postdoctoral Researcher, Computer Vision Department
Mohamed bin Zayed University of Artificial Intelligence
I am a postdoctoral researcher in the Computer Vision Department at MBZUAI, advised by Prof. Hao Li. My research centers on human-centric generative modeling across 2D images, 3D content, and video, with an emphasis on creating realistic, controllable, and interactive representations of humans and their environments.
Recently, I have focused on world models, particularly egocentric world models and world action models that understand, predict, and interact with dynamic environments from a first-person perspective. These models have broad potential for virtual reality, embodied intelligence, and interactive human-centered simulation.
Previously, I was a visiting Ph.D. researcher at the Robotics Institute, Carnegie Mellon University, advised by Prof. Fernando De la Torre. I completed my doctoral studies at Sun Yat-sen University, supervised by Prof. Xiaodan Liang.
Research Interests
My research connects generative modeling, 3D vision, and embodied perception.
Human-Centric Generation
Controllable synthesis of identity, pose, motion, and interaction across images and video.
3D Humans & Avatars
High-fidelity 3D representations for avatars, virtual try-on, immersive media, and interactive simulation.
News
Recent research and career updates.
-
HairOrbit is accepted by ECCV 2026.
-
GRPO-Guard is accepted by CVPR 2026.
-
I joined the Computer Vision Department at MBZUAI as a postdoctoral researcher, advised by Prof. Hao Li.
-
G3D-VTON is accepted by TPAMI 2025.
-
Robust-MVTON is accepted by CVPR 2025.
-
I completed my doctoral studies at Sun Yat-sen University and was named an Honors Graduate.
-
ViTon-GUN is accepted by TVCG 2025.
-
DreamVTON is accepted by ACM MM 2024.
-
GUESS is accepted by TVCG 2024.
-
B2A-HDM is accepted by AAAI 2024.
-
I began a visiting research period at the Robotics Institute of Carnegie Mellon University.
-
GP-VTON is accepted by CVPR 2023.
-
3D-GCL is accepted by NeurIPS 2022.
-
ARMANI is accepted by ACM MM 2022.
-
wFlow is accepted by CVPR 2022.
-
PASTA-GAN is accepted by NeurIPS 2021.
-
CPF-Net is accepted by TIP 2021.
-
M3D-VTON is accepted by ICCV 2021.
-
WAS-VTON is accepted by ACM MM 2021.
Selected Publications
Selected peer-reviewed publications. My name is highlighted; * indicates equal contribution.
2026
HairOrbit: Multi-view Aware 3D Hair Modeling from Single Portraits
European Conference on Computer Vision (ECCV), 2026
GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026
2025
Monocular-to-3D Virtual Try-On with Generative Semantic Articulated Fields
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025
Robust-MVTON: Learning Cross-Pose Feature Alignment and Fusion for Robust Multi-View Virtual Try-On
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
ViTon-GUN: Person-to-Person Virtual Try-on via Garment Unwrapping
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2025
2024
DreamVTON: Customizing 3D Virtual Try-on with Personalized Diffusion Models
ACM International Conference on Multimedia (ACM MM), 2024
Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model
AAAI Conference on Artificial Intelligence (AAAI), 2024
GUESS: Gradually Enriching Synthesis for Text-Driven Human Motion Generation
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2024
2023
GP-VTON: Towards General Purpose Virtual Try-On via Collaborative Local-Flow Global-Parsing Learning
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
2022
Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning
Advances in Neural Information Processing Systems (NeurIPS), 2022
ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design
ACM International Conference on Multimedia (ACM MM), 2022
Dressing in the Wild by Watching Dance Videos
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
2021
Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN
Advances in Neural Information Processing Systems (NeurIPS), 2021
WAS-VTON: Warping Architecture Search for Virtual Try-on Network
ACM International Conference on Multimedia (ACM MM), 2021
Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
IEEE Transactions on Image Processing (TIP), 2021
M3D-VTON: A Monocular-to-3D Virtual Try-On Network
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
2020
Fashion Editing with Adversarial Parsing Learning
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
Experience & Education
Academic appointments, training, and industry research experience.
Academic
-
Mohamed bin Zayed University of AI
Postdoctoral Researcher, Computer Vision Department
Advisor: Prof. Hao Li
-
Carnegie Mellon University
Visiting Ph.D., Robotics Institute
Advisor: Prof. Fernando De la Torre
-
Sun Yat-sen University
Ph.D. studies, Intelligent Systems Engineering
Advisor: Prof. Xiaodan Liang
-
Sun Yat-sen University
M.S., Computer Science and Engineering
Advisor: Prof. Jianhuang Lai
-
Sun Yat-sen University
B.S., Computer Science and Engineering
Advisor: Prof. Xiaohua Xie
Industry Research
-
ByteDance · Shenzhen
Research Intern, Intelligence Creation Platform
High-fidelity and 3D virtual try-on
-
Tencent · Shenzhen
Research Intern, Artificial Intelligence Platform Department
Cross-modal human motion synthesis
-
ByteDance · Beijing
Research Intern, Intelligence Creation Platform
High-fidelity 2D garment-to-person virtual try-on
Academic Service & Honors
Reviewing
Conferences: CVPR, ICCV, ECCV, NeurIPS
Journal: International Journal of Computer Vision (IJCV)
Community
Organizer, CVPR 2020 Workshop on Human-centric Image/Video Synthesis.
Teaching Assistant, Artificial Intelligence Experiment, Sun Yat-sen University, 2020–2021.
Talk & Recognition
VALSE Webinar on scalable unpaired virtual try-on, Jan. 2022.
Honors Graduate, Sun Yat-sen University, May 2025.