Allowing Torch/Tensorflow to directly access rendered image in GPU

General Discussion
7

Perfect! The second way works by setting the correct GL_TEXTURE_2D from PyOpenGL. Now I manage to build a pipeline as follows:

  1. Panda3D render to texture
  2. Map resource with cudaGraphicsRegisterImage
  3. get the resource pointer and make a copy via CUDA-Python
  4. convert CUDA-Python to CuPy array and then to torch

Now everything happens only on GPU! But I am still debugging, as nothing is contained in the final torch tensor which is full with 0. So far, I am not sure what causes this problem, but I am really happy. At least the whole pipeline works. Thank you!

By the way, I find a pretty interesting example: CuPy/OpenGL interop example ยท GitHub it shows how to register VBO and EBO buffer created by glGenBuffers. And then modify the buffer content with CuPy to do the rendering. Do you think is it possible to do similar things in Panda3D with pure Python code? I think that register from buffer would be more efficient, right?