DeepHandMesh: A Weakly-Supervised Deep Encoder-Decoder Framework for High-Fidelity Hand Mesh Modeling
Introduction
This repo is official PyTorch implementation of DeepHandMesh: A Weakly-Supervised Deep Encoder-Decoder Framework for High-Fidelity Hand Mesh Modeling (ECCV 2020. Oral.).
Dataset download (only provide data of subject 4)
Images
- Images.
- To verify downloaded files using
md5sum
, runpython verify_download.py
. - To unzip images, run
sh unzip.sh
. - All image files take 9.2 GB.
Annotations
- Annotation.
- To verify downloaded 3D scan files using
md5sum
, runpython verify_download_3d_scan.py
. - To verify downloaded depthmap files using
md5sum
, runpython verify_download_depthmap.py
. - To unzip 3D scan files, run
sh unzip_3d_scan.sh
. - To unzip depthmap files, run
sh unzip_depthmap.sh
.
* 3D_scans_decimated: decimated 3D scans (uncompressed files take 114 GB)
|-- [frame_index].ply
* depthmaps: multi-view depth maps (uncompressed files take 421 GB)
|-- [frame_index]
| |-- depth[camera_index].pkl
* keypoints.zip: 3D hand joint coordinates (in world coordinate system. milimeter scale.)
|-- keypoints[frame_index].pts: (joint index, x, y, z, summation of scores of the annotator, the number of views used for the triangulation)
The joint index follows this order: 'b_r_thumb_null', 'b_r_thumb3', 'b_r_thumb2', 'b_r_thumb1', 'b_r_index_null', 'b_r_index3', 'b_r_index2', 'b_r_index1', 'b_r_middle_null', 'b_r_middle3', 'b_r_middle2', 'b_r_middle1', 'b_r_ring_null', 'b_r_ring3', 'b_r_ring2', 'b_r_ring1', 'b_r_pinky_null', 'b_r_pinky3', 'b_r_pinky2', 'b_r_pinky1', 'b_r_wrist'
* KRT_512: camera parameters
|-- [camera index, intrinsic matrix, dist, extrinsic matrix]
* hand_model.zip: contains an initial 3D hand model
Tips for a Google Drive download
To download multiple files from Google drive without compressing them, try this. If you have a problem with ‘Download limit’ problem when tried to download dataset from google drive link, please try this trick.
* Go the shared folder, which contains files you want to copy to your drive
* Select all the files you want to copy
* In the upper right corner click on three vertical dots and select “make a copy”
* Then, the file is copied to your personal google drive account. You can download it from your personal account.
Codes and pre-trained models
Go to github.
Results
Effect of Identity- and Pose-Dependent Correctives
Comparison with MANO
Reference
@InProceedings{Moon_2020_ECCV_DeepHandMesh,
author = {Moon, Gyeongsik and Shiratori, Takaaki and Lee, Kyoung Mu},
title = {DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modeling},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}