Nothing in Texture Buffer under offscreen mode

General Discussion
1

Hi, everyone.
I have some texture buffers in my game, and they work well in “onscreen” mode, that is, I can get the images of my cameras from texture buffer. However, by using the similar way to get the images in “offscreen” mode, the images obtained from these texture buffers are all blank. It’s strange that I can get them from texture buffer, and the x-size and y-size is correct, but nothing in the images. To find the problem, I use the same code to get the image of base.cam from base.win in “offscreen” mode. Fortunately, base.win works well in “offscreen” mode, and my scene is shown on the image. Maybe there are some differences between base.win and buffers which are made by makeTextureBuffer().

The following code is how I create my buffer and get the images from buffer

from panda3d.core import NodePath, Vec3, Vec4
import logging
from typing import Union
import numpy as np
from panda3d.core import Texture, GraphicsOutput, PNMImage, Ostream, PNMFileType


class ImageBuffer:
    CAM_MASK = None
    BUFFER_L = 800
    BUFFER_W = 800
    BKG_COLOR = Vec4(0., 0., 0., 0.1)
    display_bottom = 0.8
    display_top = 1

    def __init__(
        self, length: float, width: float, pos: Vec3, bkg_color: Union[Vec4, Vec3], make_buffer_func, make_camera_func,
        parent_node: NodePath
    ):
        assert self.CAM_MASK is not None, "define a camera mask for every image buffer"
        self.texture = Texture()
        self.buffer = make_buffer_func("camera", length, width, tex=self.texture, to_ram=True)
        # now we have to setup a new scene graph to make this scene

        self.node_path = NodePath("new render")

        # this takes care of setting up their camera properly
        self.cam = make_camera_func(self.buffer, clearColor=bkg_color)
        self.cam.reparentTo(self.node_path)
        self.cam.setPos(pos)
        self.cam.node().setCameraMask(self.CAM_MASK)
        self.node_path.reparentTo(parent_node)

    def get_image(self):
        """
        Bugs here! when use offscreen mode, thus the front cam obs is not from front cam now
        """
        # raise PermissionError
        from panda3d.core import Texture, GraphicsOutput, PNMImage, Ostream, PNMFileType
        import numpy as np
        img = PNMImage()
        self.buffer.getScreenshot(img)
        #  save for debug
        # img.write("img.jpg")
        return img

    def get_gray_pixels_array(self) -> np.ndarray:
        img = self.get_image()
        img.makeGrayscale()
        numpy_array = np.array([[img.getGray(i, j) for j in range(img.getYSize())] for i in range(img.getXSize())])
        return np.clip(numpy_array, 0, 1)

    def __del__(self):
        logging.debug("{} is destroyed".format(self.__class__.__name__))

I have tried to use addRenderToTexture() method and trigger_copy mode mentioned by @David and @rdb, instead of PNMImage, but the result is same. The images I get under “offscreen” mode are still blank.
Really appreciate your help!

2

For the offscreen buffer I think you need to call in advance. base.graphicsEngine.renderFrame()

1 Like
3

Thanks for your help! I am sure that I call renderFrame() before I read the offscreen buffer data to RAM, but the image is still empty, :no_mouth: . Actually, after calling renderFrame(), I also read the data in base.win to the RAM and save it into PNMImage, which shows the data captured by base.cam correctly. Thus, there must be some errors when I create my texture buffer.

4

oh, shit. I solved it. The camera of my buffer is not attached to the scene graph, and thus nothing is shown in the images. :sweat: However, It also shows that my code works well :joy:. Hope it can help someone who wants to get images under “offscreen” mode and transfer it to numpy array.

1 Like