VR with Panda3D: panda3d-openvr

Following rdb’s thread update on VR support in Panda3D, my curiosity was aroused again and I googled around to learn what is required to have a VR compatible application. As expected it is not trivial in most case :slight_smile:
The only exception is OpenVR which not only provides drivers for the headset hardware but also an application layer that take care of all the projection, compositing and extra work needed to have a proper VR rendering. Better, there is a python binding pyopenvr with a decent implementation. (The downside is that the only implementation of OpenVR is SteamVR, so you need Steam and a steam account to use it. So not really open after all…)

So, after some tinkering, I managed to properly bind pyopenvr, configure the camera in Panda3D, retrieve the textures from the draw context to submit to the compositor and voilà :

Link

Installation

The installation is as usual :

git clone https://github.com/el-dee/panda3d-openvr
cd panda3d-openvr
python3 setup.py install

(I might create a pypi package later on)

Examples

Minimal

The first example, showcases how to initialize the VR system and retrieve the information about the HMD, controllers, tracking stations, …
It displays the panda avatar in front of you, the default rgb cube for the controllers and cameras for the tracking stations (if any).

Actions

This example is pretty similar to the minimal example, except that it uses actions manifest to manage the interaction with the user. It’s a little bit more complicated that using directly events, but that allows the user to rebind the actions from the Steam interface.

Ralph

The last example is the mandatory Roaming Ralph demo, except this time you are Ralph :slight_smile: This example shows how to move the tracking space in your scene to simulate displacement.

You can use the trackpad of the left controller to move forward and backward in the direction you’re facing, or turn yourself around inworld.

Final notes

This has been only tested by me on my setup (Linux + HTC Vive) so it may not work for you (yet), but theoretically it should work on Windows and macOS too.

Also, on Linux the NVidia controller does not seem to allow an app to disable v-sync. If you get 45 fps instead of 90 fps, that means you have to open the NVidia control panel and deactivate V-Sync there.

4 Likes

Forgot to mention it, I created the project in PyPi : https://pypi.org/project/panda3d-openvr/ You can now use pip to install it easily.

1 Like

After quite a while, here is a new release of this project ! This new release contains some new important features, a couple of bug fixes and a new API :slight_smile:

Here is the changelog with some explanation :

  • Hands animation using skeleton : This is the most important feature of this release, and you can see the result above
  • Hidden area mesh to allow early-z discard of invisible pixel, improving the fill rate
  • Add application manifest and identification : This allows SteamVR to remember the settings of your application and displays its actual name in the dashboard.
  • Allow an action to be restricted to a specific device : This allows better action rebinding by the user
  • Support colour space configuration : Now linear or sRGB buffers can be submitted to OpenVR
  • Complete code documentation
  • Cleanup and document better the samples
  • Include default action manifest from SteamVR : This contains binding for all the common controllers.
  • New API now uses handlers instead of inheritance : This improves a lot the usability, the old method are deprecated and will be removed in a next version.

The next release should includes teleportation and (or) basic physics, depending on the time I can spend on this side project.

Enjoy :slight_smile:

4 Likes

Ah, I see that the crucially-important snoot-booping logic has been implemented. Excellent. :stuck_out_tongue_winking_eye:

More seriously, this looks like rather good work! Well done on it! :slight_smile:

1 Like

This is incredible work! I see you were originally testing with the Vive, and I am curious about other headsets. Proton support, for “flat” games has been improving, at least, even if Steam is not exactly open. Still, your implementation may be the most minimal way I know of developing a VR application. Cool stuff.

Thanks ! I merely leveraged the features and flexibility that Panda3D offer, that should explain why the implementation is so minimal (Also, OpenVR does a lot of the dirty work now)

This should work on any OpenVR compatible headsets and controllers, but I only got feedback about the Vive, the Index and the Rift so far.

There is an open-source implementation of OpenVR, but only for the Oculus : Campbell Suter (ZNix) / OpenComposite · GitLab
There is also a fully open source compositor, OpenHMD, GitHub - OpenHMD/OpenHMD: Free and Open Source API and drivers for immersive technology., which is included in Monado, https://monado.freedesktop.org/, an open-source OpenXR implementation. But the python bindings for both are either not maintained or inexistent. But I hope this will change in the future and we then have a fully open-source VR system :slight_smile:

4 Likes