Assistant Professor at EECS, DGIST
Email: mks0601@gmail.com
Lab website
Google Scholar
CV
Github
Research Interests
Designing artificial intelligence to perceive, model, and represent human-centric 3D worlds through state-of-the-art computer vision, computer graphics, and machine learning techniques.
Publications (Selected)
Codec Avatar Studio: Paired Human Captures for Complete, Driveable, and Generalizable Avatars
Julieta Martinez, Emily Kim, Javier Romero, Timur Bagautdinov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollhöfer, Te-Li Wang, Shaojie Bai, Shih-En Wei, Rohan Joshi, Wyatt Borsos, Tomas Simon, Jason Saragih, Paul Theodosis, Alexander Greene, Anjani Josyula, Silvio Mano Maeta, Andrew I Jewett, Simion Venshtain, Christopher Heilman, Yueh-Tung Chen, Sidi Fu, Mohamed Ezzeldin A. Elshaer, Tingfang Du, Longhua Wu, Shen-Chi Chen, Kai Kang, Michael Wu, Youssef Emad, Steven Longay, Ashley Brewer, Hitesh Shah, James Booth, Taylor Koska, Kayla Haidle, Joanna Ching-Hui Hsu, Thomas Dauer, Peter Selednik, Tim Godisart, Scott Ardisson, Matthew Cipperly, Ben Humberston, Lon Farr, Bob Hansen, Peihong Guo, Dave Braun, Steven Krenn, He Wen, Lucas Evans, Natalia Fadeeva, Matthew Stewart, Gabriel Schwartz, Divam Gupta, Gyeongsik Moon, Kaiwen Guo, Yuan Dong, Yichen Xu, Takaaki Shiratori, Fabian Andres Prada Nino, Bernardo R Pires, Bo Peng, Julia Buffalini, Autumn Trimble, Kevyn Alex Anthony McPhail, Melissa Robinson Schoeller, Yaser Sheikh
NeurIPS 2024 (Datasets and Benchmarks Track)
[WEBSITE1] [WEBSITE2]
Expressive Whole-Body 3D Gaussian Avatar
Gyeongsik Moon, Takaaki Shiratori, Shunsuke Saito
ECCV 2024
[WEBSITE] [ARXIV] [PAPER] [VIDEO] [CODE]
3D Hand Sequence Recovery from Real Blurry Images and Event Stream
JoonKyu Park, Gyeongsik Moon, Takaaki Shiratori, Weipeng Xu, Evan Kaseman, Kyoung Mu Lee
ECCV 2024
[PAPER] [DATASET] [VIDEO]
Authentic Hand Avatar from a Phone Scan via Universal Hand Model
Gyeongsik Moon, Weipeng Xu, Rohan Joshi, Chenglei Wu, Takaaki Shiratori
CVPR 2024
[ARXIV] [PAPER] [CODE] [VIDEO]
URHand: Universal Relightable Hands
Zhaoxi Chen, Gyeongsik Moon, Kaiwen Guo, Chen Cao, Stanislav Pidhorskyi, Tomas Simon, Rohan Joshi, Yuan Dong, Yichen Xu, Bernardo Pires, He Wen, Lucas Evans, Bo Peng, Julia Buffalini, Autumn Trimble, Kevyn McPhail, Melissa Schoeller, Shoou-I Yu, Javier Romero, Michael Zollhöfer, Yaser Sheikh, Ziwei Liu, Shunsuke Saito
CVPR 2024 (Oral presentation (top 0.78%))
[WEBSITE] [ARXIV] [PAPER]
Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer
Hyeongjin Nam*, Daniel Sungho Jung*, Gyeongsik Moon, Kyoung Mu Lee (* equal contribution)
CVPR 2024
[ARXIV] [PAPER] [CODE]
A Dataset of Relighted 3D Interacting Hands
Gyeongsik Moon, Shunsuke Saito, Weipeng Xu, Rohan Joshi, Julia Buffalini, Harley Bellan, Nicholas Rosen, Jesse Richardson, Mallorie Mize, Philippe de Bree, Tomas Simon, Bo Peng, Shubham Garg, Kevyn McPhail, Takaaki Shiratori
NeurIPS 2023 (Datasets and Benchmarks Track)
[DATASET] [ARXIV] [PAPER] [VIDEO]
Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wild
Gyeongsik Moon
CVPR 2023
[ARXIV] [PAPER] [VIDEO] [CODE]
Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding
Yeonguk Oh*, JoonKyu Park*, Jaeha Kim*, Gyeongsik Moon, Kyoung Mu Lee (* equal contribution)
CVPR 2023
[ARXIV] [PAPER] [CODE/DATASET]
Rethinking Self-Supervised Visual Representation Learning in Pre-training for 3D Human Pose and Shape Estimation
Hongsuk Choi*, Hyeongjin Nam*, Taeryung Lee, Gyeongsik Moon, Kyoung Mu Lee (* equal contribution)
ICLR 2023
[ARXIV] [PAPER]
MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels
Taeryung Lee*, Gyeongsik Moon*, Kyoung Mu Lee (* equal contribution)
AAAI 2023 (Oral presentation)
[ARXIV] [PAPER] [CODE]
MonoNHR: Monocular Neural Human Renderer
Hongsuk Choi*, Gyeongsik Moon*, Vincent Leroy, Matthieu Armando, Kyoung Mu Lee, Gregory Rogez (* equal contribution)
3DV 2022
[ARXIV] [PAPER]
3D Clothed Human Reconstruction in the Wild
Gyeongsik Moon*, Hyeongjin Nam*, Takaaki Shiratori, Kyoung Mu Lee (* equal contribution)
ECCV 2022
[ARXIV] [PAPER] [CODE]
HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network
JoonKyu Park*, Yeonguk Oh*, Gyeongsik Moon*, Hongsuk Choi, Kyoung Mu Lee (* equal contribution)
CVPR 2022
[ARXIV] [PAPER] [CODE]
Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes
Hongsuk Choi, Gyeongsik Moon, JoonKyu Park, Kyoung Mu Lee
CVPR 2022
[ARXIV] [PAPER] [CODE]
Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation
Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee
CVPRW 2022
[ARXIV] [PAPER] [VIDEO] [CODE]
NeuralAnnot: Neural Annotator for 3D Human Mesh Training Sets
Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee
CVPRW 2022
[ARXIV] [PAPER] [WEBSITE]
Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video
Hongsuk Choi, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee
CVPR 2021
[ARXIV] [PAPER] [CODE] [VIDEO]
IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos
Gyeongsik Moon*, Heeseung Kwon*, Kyoung Mu Lee, Minsu Cho (* equal contribution)
CVPRW 2021
[ARXIV] [PAPER] [CODE]
DeepHandMesh: A Weakly-Supervised Deep Encoder-Decoder Framework for High-Fidelity Hand Mesh Modeling
Gyeongsik Moon, Takaaki Shiratori, Kyoung Mu Lee
ECCV 2020 (Oral presentation (top 2%))
[DATASET] [ARXIV] [PAPER] [CODE] [VIDEO]
InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image
Gyeongsik Moon, Shoou-i Yu, He Wen, Takaaki Shiratori, Kyoung Mu Lee
ECCV 2020
[DATASET] [ARXIV] [PAPER] [CODE] [VIDEO]
I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image
Gyeongsik Moon, Kyoung Mu Lee
ECCV 2020
Winners and 2nd place of the 3D human pose estimation in the wild (3DPW) challenge (workshop in conjunction with ECCV 2020)
[ARXIV] [PAPER] [CODE] [VIDEO]
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose
Hongsuk Choi*, Gyeongsik Moon*, Kyoung Mu Lee (* equal contribution)
ECCV 2020
[ARXIV] [PAPER] [CODE] [VIDEO]
Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image
Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee
ICCV 2019
[ARXIV] [PAPER] [CODE (ROOTNET)] [CODE (POSENET)]
PoseFix: Model-agnostic General Human Pose Refinement Network
Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee
CVPR 2019
[ARXIV] [PAPER] [CODE]
V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map
Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee
CVPR 2018
Winners of the HANDS 2017 3D hand pose estimation challenge (workshop in conjunction with ICCV 2017)
[ARXIV] [PAPER] [CODE]
Depth-based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Shanxin Yuan, Guillermo Garcia-Hernando, Björn Stenger, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee, Pavlo Molchanov, Jan Kautz, Sina Honari, Liuhao Ge, Junsong Yuan, Xinghao Chen, Guijin Wang, Fan Yang, Kai Akiyama, Yang Wu, Qingfu Wan, Meysam Madadi, Sergio Escalera, Shile Li, Dongheui Lee, Iason Oikonomidis, Antonis Argyros, Tae-Kyun Kim
CVPR 2018
[ARXIV] [PAPER]
Experiences
- 2024.04 - : Assistant Professor, EECS, DGIST
- 2022.06 - 2024.03: Postdoctoral Research Scientist, Reality Labs Research, Meta (Manager: Takaaki Shiratori)
- 2022.03 - 2022.05: Visiting Researcher, CLOVA AI Lab, Naver
- 2021.06 - 2021.12: Visiting Researcher, Naver Labs Europe (Host: Gregory Rogez)
- 2021.03 - 2022.02: Senior Researcher, ASRI, SNU
- 2019.04 - 2019.09: Research Scientist Intern, Facebook Reality Labs (Manager: Takaaki Shiratori)
- 2015.09 - 2021.02: Ph.D. student, ECE, SNU (Advisor: Kyoung Mu Lee)
- 2011.03 - 2015.08: Undergraduate student, CSE, POSTECH
Honors
- Outstanding Reviewer (CVPR 2023)
- Top Reviewer (NeurIPS 2022)
- Doctoral Consortium (CVPR 2021. Mentor: Micheal Black of MPII)
- Samsung Humantech Paper Award Silver Prize (CSE, 2021)
- SNU Distinguished Ph.D. Dissertation Award / Best Thesis Award (2021)
- Qualcomm Innovation Fellowship (2020)
- Google Ph.D. Fellowship (Computer Vision. 2020)
- 1st and 2nd place at the 3D human pose estimation in the wild (3DPW) challenge (workshop in conjunction with ECCV 2020)
- 1st place at the HANDS 2017 3D hand pose estimation challenge (workshop in conjunction with ICCV 2017)
Service
- Area Chair
- NeurIPS Datasets and Benchmarks (2023-)
- Session Chair
- AAAI (2023)
- Reviewer
- CVPR (2020-)
- ICCV (2021-)
- ECCV (2020-)
- SIGGRAPH Asia (2022-)
- NeurIPS (2022-)
- ICLR (2022-)
- ICML (2023-)
- AAAI (2022-)
- TPAMI (2021-)