How to use multiple cameras with simplepbr

Hi, @Moguri

I am testing my program with simplepbr. Everything works fine except for the rendering from the new camera and the new buffer made bymakeTextureBuffer().

As shown in the following image, the result of the main window is perfect, while the circled display region which is associated with the new buffer/camera seems missing the tone-mapping and thus dark. Could you give me some hints to address it?

Though I found this issue on github: Use multiple Camera nodes with the PBR shaders (actually just for Tone Mapping) · Issue #47 · Moguri/panda3d-simplepbr · GitHub, calling init() on the new buffer and the new camera still doesn’t work.

I don’t believe simplepbr supports multiple init() . You would need to apply a set of shaders to that separate region yourself, to my understanding. It is possible but it gets somewhat out of the “simple” and “easy” realm.

Also, nice Z-car.

1 Like

Thank you, dude. I copied some source code from simplepbr and finally made it. As for these beautiful cars, so happy you like em, it actually took me a lot of effort to get these cars :joy:

1 Like

Happy to assist. The cars do look quite good there I must say. I’m usually adept at identifying cars, but I can’t place that red one in center frame – what is it?

1 Like

Ahh, it is Ferrari Dino: Ferrari Dino: not a simply Ferrari, but a memorial to a lost son - Influx
Very beautiful, I love old oil cars.

image

Out of curiosity, what about simplepbr prevents multiple init calls?

Good question, and to be honest I am not sure, though I would speculate that simplepbr’s initialization procedure is setting up the original DisplayRegion such that consecutive DisplayRegions are ignored, and cause interference.

To test this, I tried the following:

simplepbr.init()

region = base.win.makeDisplayRegion(0.5, 1, 0, 1)

…which fails with:

if hasattr(i.node(), 'is_shadow_caster') and i.node().is_shadow_caster() AssertionError: !is_empty() at line 228 of c:\buildslave\sdk-windows-amd64\build\built1.10\include\nodePath.I

I don’t have a ton of time to test this today, but I did go a little further.

This works:

        # simplepbr.init()

        region = base.win.makeDisplayRegion(0.5, 1, 0, 1)
        camNode = Camera('cam')
        camNP = NodePath(camNode)
        region.setCamera(camNP)
        region = base.win.makeDisplayRegion()
        render2 = NodePath('render2')
        camNP.reparentTo(base.render)
        env = loader.loadModel('models/ramp_1.gltf')
        env.reparentTo(render2)

This fails with the same ‘is_shadow_caster’ error:

        simplepbr.init()

        region = base.win.makeDisplayRegion(0.5, 1, 0, 1)
        camNode = Camera('cam')
        camNP = NodePath(camNode)
        region.setCamera(camNP)
        region = base.win.makeDisplayRegion()
        render2 = NodePath('render2')
        camNP.reparentTo(base.render)
        env = loader.loadModel('models/ramp_1.gltf')
        env.reparentTo(render2)

In fact, the simplepbr library is well written and it is possible to use it outside of ShowBase, I had a successful experience.

The init function requires a number of parameters that can be passed. You just need to duplicate some of them, that is, create a buffer to bind the camera and so on.

import simplepbr
import simplepbr as simplepbr_1

simplepbr.init(set 1 arguments scene data and buffer and so on)
simplepbr_1.init(set 2 arguments scene data and buffer and so on)
2 Likes

Let me clarify this at first, now I am 100% sure that simplepbr.init() can be called for multiple Buffers/CameraNode.

Several days before, I tried calling simplepbr.init(cameraNode=my_new_cam, window=my_new_buffer) so that the buffer content can have the correct tone map when rendering to a new display region of showbase.win. The new buffer is created by my_new_buffer=showbase.win.makeTextureBuffer(h, w, fpb) and camera is created by my_new_cam=showbase.makeCamera(self.buffer). However, I found that simplepbr doesn’t work on this new buffer, and tonemap shader is missing when running the following code:

self.display_region = showBase.win.makeDisplayRegion(*display_region)
self.display_region.setCamera(my_new_cam)

I then made a wrong conclusion that simplepbr can not be applied for >2 times. Then I copied the code for setting tonemap shader and ran it for my new buffer. Unfortunately, it still doesn’t work until I made the following modification :

 self.display_region.setCamera(my_new_buffer.getDisplayRegions()[1].camera)

After this, the new display region on my window can show correct content with tonemap. So the problem is that I set the wrong camera when making the display region, the simplebr itself is perfect without any problem

1 Like

Good to know. Perhaps you could post your full setup code for other users who might be interested in using simplepbr in this way?

1 Like

Good suggestion. The following code will make a new buffer and a new camera with pbr enabled. The rendered content can be accessed by get_rgb_array(). Besides, one can use add_display_region() and remove_display_region() to make (remove) a new display region and render the content of this buffer to the main window.

from typing import List

import numpy as np
import simplepbr
from direct.showbase.ShowBase import ShowBase
from panda3d.core import NodePath

engine = ShowBase()


class ImageBuffer:
    BUFFER_W = 84
    BUFFER_H = 84

    def __init__(
            self,
            width: float,
            height: float,
            frame_buffer_property=None,
    ):
        if frame_buffer_property is None:
            self.buffer = engine.win.makeTextureBuffer("camera", width, height)
        else:
            self.buffer = engine.win.makeTextureBuffer("camera", width, height, fbp=frame_buffer_property)

        self.origin = NodePath("new render")
        # this takes care of setting up their camera properly
        self.cam = engine.makeCamera(self.buffer)
        self.cam.reparentTo(self.origin)
        self.display_region = None

        # see init() accepts more parameters, like msaa, etc.
        self.pbrpipe=simplepbr.init(camera_node=self.cam, window=self.buffer, render_node=self.origin)

    def get_rgb_array(self):
        engine.graphicsEngine.renderFrame()
        origin_img = self.buffer.getDisplayRegion(1).getScreenshot()
        img = np.frombuffer(origin_img.getRamImage().getData(), dtype=np.uint8)
        img = img.reshape((origin_img.getYSize(), origin_img.getXSize(), 4))
        img = img[::-1]
        img = img[..., :-1]
        return img

    def add_display_region(self, display_region: List[float]):
        self.display_region = engine.win.makeDisplayRegion(*display_region)
        self.display_region.setCamera(self.buffer.getDisplayRegions()[1].camera)

    def remove_display_region(self):
        engine.win.removeDisplayRegion(self.display_region)
        self.display_region = None

1 Like

This error occurs if you have a DisplayRegion without a camera. simplepbr should be checking this, I will file a PR. Assigning a camera to the display region should make this go away.