Akash Karthikeyan · Robert Ren · Yash Kant · Igor Gilitschenski
WACV 2024
Reconstructing realistic human avatars from monocular videos is a challenge that demands intricate modeling of 3D surface and articulation. In this paper, we introduce a comprehensive approach that synergizes three pivotal components: (1) a Signed Distance Field (SDF) representation with volume rendering and grid-based ray sampling to prune empty raysets, enabling efficient 3D reconstruction; (2) faster 3D surface reconstruction through a warmup stage for human surfaces, which ensures detailed modeling of body limbs; and (3) temporally consistent subject-specific forward canonical skinning, which helps in retaining correspondences across frames, all of which can be trained in an end-to-end fashion under 15 minutes.
Leveraging warmup and grid-based ray marching, along with a faster voxel-based correspondence search, our model streamlines the computational demands of the problem. We further experiment with different sampling representations to improve ray radiance approximations and obtain a floater free surface. Through rigorous evaluation, we demonstrate that our method is on par with current techniques while offering novel insights and avenues for future research in 3D avatar modeling. This work showcases a fast and robust solution for both surface modeling and novel-view animation.
conda create -n av3d -f environment.yaml
conda activate av3d
cd third_party/nerfacc
pip install -e .
cd ../../
python setup.py develop
- See docs/dataset.md on how to accquire data used in this paper.
- See docs/experiment.md on using this code base for training and evaluation.
Our code is mainly built based on TAVA. We thank the authors for sharing the code!
We also use the external repositories listed below in this project. A big thanks to them for their code!
BSD 3-clause (see LICENSE.txt).
@InProceedings{Karthikeyan_2024_WACV,
author = {Karthikeyan, Akash and Ren, Robert and Kant, Yash and Gilitschenski, Igor},
title = {AvatarOne: Monocular 3D Human Animation},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {January},
year = {2024},
pages = {3647-3657}
}