Haoyi Zhu 朱皓怡
Ph.D student in Computer Science
Shanghai Artificial Intelligence Laboratory
University of Science and Technology of China
Hi, there!
I am a second-year Ph.D. student in Computer Science jointly at Shanghai AI Lab and USTC advised by Prof. Xiaogang Wang and Prof. Wanli Ouyang. I work closely with Prof. Tong He. I earned my B.S. degree in Artificial Intelligence Honor Class at Shanghai Jiao Tong University, advised by Prof. Cewu Lu. I also have had the privilege of working with Dr. Hao-Shu Fang and Dr. Jim Fan.
Research for fun and truth. My current research interests focus on embodied AI, robot manipulation, and 3D vision. Feel free to follow me on and for latest research announcements and updates!
In my personal life, I am passionate (but amateur) about football, music, literature, philosophy, traditional Chinese painting, and modern Chinese poems!
“The philosophers have only interpreted the world, in various ways. The point, however, is to change it.”
✨ News ✨
- Oct. 2024 SPA has been announced! SPA is a novel representation learning framework that emphasizes the importance of 3D spatial awareness in embodied AI. Paper,code, and pre-trained models are all open-sourced! Check it out!
- Sep. 2024: PointCloudMatters is accepted by NeurIPS D&B 2024! We prove that explicit representation like point cloud can significantly enhance the performance and generalization ability of robot learning policies. Codes are open-sourced!
- Feb. 2024: UniPAD is accepted by CVPR 2024! Check out our code on !
- Oct. 2023: PonderV2 and UniPAD has been announced! PonderV2 is a universal pre-training paradigm for 3D vision, paving the way for 3D foundation model. It achieves SOTA on 11 indoor and outdoor benchmarks. Check out our paper and code!
- Jul. 2023: RH20T has been announced! RH20T is a large-scale open-source robotic dataset for learning diverse skills in one-shot, comprising over 110,000 contact-rich robot manipulation sequences across diverse skills, contexts, robots, and camera viewpoints, all collected in the real world. Please check out our website for latest updates!
- Nov. 2022: MineDojo has won 🎉 Outstanding Paper Award 🎉 at NeurIPS announcement!
- Nov. 2022: AlphaPose paper is accepted by TPAMI! AlphaPose is an accurate multi-person pose estimator, which has received more than 6.5K stars on Github. Check out the paper for more details and feel free to star on !
- Oct. 2022: X-NeRF is accepted by WACV 2023! Checkout our code on !
- Jun. 2022: MineDojo has been announced! MineDojo is a new framework for building generally capable agents with internet-scale knowledge in Minecraft. Paper, code, and databases are all open access. Check it out today!
- Embodied AI
- Computer Vision
- Robot Learning
B.S. in Artificial Intelligence Honor Class, 2019 - 2023
Shanghai Jiao Tong University
Ph.D. in Computer Science, 2023 - Present
Shanghai AI Lab & USTC
Featured
Research Highlights
Publications
Visit my Google Scholar page for a comprehensive listing!
Experience
- Joint Ph.D. at Shanghai AI Lab and USTC.
- Doctoral advisor: Prof. Wanli Ouyang and Prof. Xiaogang Wang
- Conducting AI research on 3D vision, foundation model and Embodied AI.
- Joint Ph.D. with USTC.
- Co-author of AlphaPose, an accurate multi-person pose estimator. Paper accpted by TPAMI.
- Proposed X-NeRF, paper accepted by WACV 2023.
- Major in the first AI Honor class.
- Research on computer vision and robotics at MVIG advised by Dr. Hao-shu Fang and Prof. Cewu Lu.
- Co-developed MineDojo for open-ended agent learning in Minecraft advised by Dr. Jim Fan and Prof. Anima Anandkumar.
- Paper anounced as Outstanding Paper Award at NeurIPS Dataset & Benchmark 2022.
- Lead a project on unsupervised multi-task learning for subtomogram images advised by Dr. Xiangrui Zeng and Prof. Min Xu.
- Paper accepted by ICIP 2022.
Gallery
Some of my traditional Chinese paintings
