Berkeley lab developers created Nerfstudio, a Python framework that converts photographs into 3D navigable scenes using Neural Radiance Fields to speed up NeRF projects and make them more accessible to others.
The framework includes plug-and-play components for creating NeRF-based procedures, facilitating collaboration, and incorporating NeRF into projects more accessible. Kanazawa and her colleagues will submit their Nerfstudio paper at SIGGRAPH 2023.
Kanazawa said, “Advancements in NeRF have contributed to its growing popularity and use in applications such as computer vision, robotics, visual effects, and gaming. But support for development has been lagging.”Â
The framework was created to make it easier to develop new NeRF algorithms, handle real-world data, and interact with reconstructions. Nerfstudio is already assisting a diverse range of engineers that use interactive computer graphics in their work, particularly those attempting to produce 3D reconstructions in real-world environments. Roboticists that use NeRF for manipulation, motion planning, simulation, and mapping are included, as are gaming companies and news organizations that use interactive graphics to tell stories.
Nerfstudio provides an open-source framework in addition to the most recent NeRF research, which is why researchers and business organizations use it. People don’t have to start from scratch when using NeRFs, which makes it simpler.
Researchers worldwide have been striving to advance NeRF’s basic technology since its launch, from accelerating real-time image rendering and training to creating new editing tools. They have also been experimenting with new NeRF methods, like when the light changes between photographs or when things move around in a scene. However, this work is frequently carried out by research teams using exclusive repositories, which makes it challenging to share these contributions with the wider NeRF community.
Nerfstudio solves these issues using a modular architecture that “consolidates these research innovations.” Furthermore, it promotes “community-driven development” by making the accompanying code and data accessible via open-source licensing.
Matt Tancik, the paper’s lead author and a Ph.D. student in Kanazawa’s lab, said, “We set out to create a platform in which people can create new modules and techniques that others can then use.” The idea is for Nerfstudio to become an open-source community project that scholars will want to collaborate with and help drive forward.
20 Berkeley researchers are actively contributing to and maintaining Nerfstudio, with up to 100 others outside the institution contributing to the core code since its October 2022 launch. Nerfstudio also makes it simple for users to run NeRFs on real-world data they acquire, a common challenge for developers. It also makes this technology more accessible to those who do not have NeRF knowledge, such as special effects studios and social media users.
The researcher said, “It’s kind of exciting that everything is out in the open; it’s incorporating the cutting-edge research you have, with both researchers wanting to push it forward and people who just want to use the tech.”
Journal Reference:
- Matthew Tancik, Ethan Weber, et al. Nerfstudio: A Modular Framework for Neural Radiance Field Development.arXiv. DOI: 10.48550/arXiv.2302.04264