HashNeRF-pytorch

🌟 Update 🌟

Get answers to any questions about this repository using this HuggingFace Chatbot.

Instant-NGP recently introduced a Multi-resolution Hash Encoding for neural graphics primitives like NeRFs. The original NVIDIA implementation mainly in C++/CUDA, based on tiny-cuda-nn, can train NeRFs upto 100x faster!

This project is a pure PyTorch implementation of Instant-NGP, built with the purpose of enabling AI Researchers to play around and innovate further upon this method.

This project is built on top of the super-useful NeRF-pytorch implementation.

Convergence speed w.r.t. Vanilla NeRF

HashNeRF-pytorch (left) vs NeRF-pytorch (right):

https://user-images.githubusercontent.com/8559512/154065666-f2eb156c-333c-4de4-99aa-8aa15a9254de.mp4

After training for just 5k iterations (~10 minutes on a single 1050Ti), you start seeing a crisp chair rendering. :)

Instructions

Download the nerf-synthetic dataset from here: Google Drive.

To train a chair HashNeRF model:

python run_nerf.py --config configs/chair.txt --finest_res 512 --log2_hashmap_size 19 --lrate 0.01 --lrate_decay 10

To train for other objects like ficus/hotdog, replace configs/chair.txt with configs/{object}.txt:

Extras

The code-base has additional support for:

• Total Variation Loss for smoother embeddings (use --tv-loss-weight to enable)
• Sparsity-inducing loss on the ray weights (use --sparse-loss-weight to enable)

ScanNet dataset support

The repo now supports training a NeRF model on a scene from the ScanNet dataset. I personally found setting up the ScanNet dataset to be a bit tricky. Please find some instructions/notes in ScanNet.md.

TODO:

• Voxel pruning during training and/or inference
• Accelerated ray tracing, early ray termination

Citation

Kudos to Thomas Müller and the NVIDIA team for this amazing work, that will greatly help accelerate Neural Graphics research:

@article{mueller2022instant,
    title = {Instant Neural Graphics Primitives with a Multiresolution Hash Encoding},
    author = {Thomas M\"uller and Alex Evans and Christoph Schied and Alexander Keller},
    journal = {arXiv:2201.05989},
    year = {2022},
    month = jan
}

Also, thanks to Yen-Chen Lin for the super-useful NeRF-pytorch:

@misc{lin2020nerfpytorch,
  title={NeRF-pytorch},
  author={Yen-Chen, Lin},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished={\url{https://github.com/yenchenlin/nerf-pytorch/}},
  year={2020}
}

If you find this project useful, please consider to cite:

@misc{bhalgat2022hashnerfpytorch,
  title={HashNeRF-pytorch},
  author={Yash Bhalgat},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished={\url{https://github.com/yashbhalgat/HashNeRF-pytorch/}},
  year={2022}
}

Star History