Lighting limits with p3d_LightSourceParameters

Scripting Issues
,
21

I think it’s useless to use methods that depend on the shader generator.

I think it will be useful.

As for the rest of the questions, this is banal algebra. You need to learn vector operations.
The concept of screen space, it simply the position of the object relative to the camera.

An example of calculating a vector and a unit vector.

cam_pos = LVecBase3f(0, 0, 0)
light_pos = LVecBase3f(0, 0, 10)

# The direction from the light source to the camera.
vector_to_camera = light_pos - cam_pos
print(vector_to_camera)
print(vector_to_camera.normalized())

# The direction from the camera to the light source.
vector_to_light = cam_pos - light_pos
print(vector_to_light)
print(vector_to_light.normalized())
1 Like
22

Do you reckon ShaderBuffers are a better alternative to 1D textures?

I consider it more appropriate for the simple reason that you are not working on image data. Packing data into a texture for processing was how GPGPU started out so that fragment shaders could be made to do math; With the advent of compute shaders and SSBOs, we do not have to deal with these complications anymore.

As for updating the buffer every frame:

  1. If the problem says that you have to, then you have to. The question then becomes how to do it efficiently, e.g. whether caching data encodings from previous frames, and only updating those elements that have changed, is actually significantly faster. If in fact most of your lights are static (or “static” enough to move calculating their actions onto the GPU), having one large buffer for static lights, and a small one for dynamic ones, may be pretty efficient.
  2. Are we talking about a significant effort to begin with? Encoding e.g. 100 lights isn’t that much work. If we’re talking about tens of thousands of lights, that may be more difficult to do on the CPU side, but I suspect that before you run into this problem, you will have to dig deeply into light assignment (Tiled/Clustered Shading), or the light loop in your fragment shader will take loooong.
  3. While it’s not efficient with your CPU-side computing, I did write a library to make working with SSBOs easier for Python developers.
2 Likes
23

Originally, I was also looking for some compatibility with the 3.30 version of OpenGL. Shader buffers require version 4.30. Do you reckon UBOs are the way to go. I could split light information into multiple UBOs, like you said, static lights into one, dynamic lights into another. Its just for SSBOs, they aren’t supported on mac.

24

AFAIK UBOs are typically limited to 64kB, and use the more inefficient std140 packing rules, so you’ll be limited to thousands or even just hundreds of lights, depending on how much data per light you’re packing.

25

I think that is perfectly fine. having over a hundred lights is enough for me. If I’m correct, I can create multiple UBOs in my shader as each UBO has its own independent storage so if one fills up, can use the others.

26

Hmm, for me, binding UBOs don’t work with the default panda3d framework. I had to resort to PyOpenGL to bind the UBO in my shader. Will this be fixed in version 1.11?

27

Can you go a bit into detail inhowfar it doesn’t work? Like, code and stack trace?

28

currently this is what I am doing:

class ManyLightsManager():
    def __init__(self, render, max_lights = 100):
        self.MAX_LIGHTS = max_lights
        width = self.MAX_LIGHTS
        self.render = render

        self.spotlight_data = np.zeros((self.MAX_LIGHTS * 4, 4), dtype=np.float32)

        self.spotlight_buffer = glGenBuffers(1)
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        glBufferData(GL_UNIFORM_BUFFER, self.spotlight_data.nbytes, self.spotlight_data, GL_DYNAMIC_DRAW)
        glBindBufferBase(GL_UNIFORM_BUFFER, 0, self.spotlight_buffer)
        glBindBuffer(GL_UNIFORM_BUFFER, 0)


        self.spotlights = []
        self.spotlight_index_count = 0
        self.spotlight_task_sequence = []

    def re_input(self):
        pass

    def update_lights_transforms(self):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)

        for i, light in enumerate(self.spotlights):
            index = i * 4
            pos = light._lightnp.getPos(self.render)
            self.spotlight_data[index, :3] = [pos[0], pos[1], pos[2]]
            #self.spotlight_data[index+1] = [light._light_color[0], light._light_color[1], light._light_color[2], 0.0]
            dir = light._lightnp.getQuat(self.render).getForward()
            self.spotlight_data[index+2, :3] = [dir[0], dir[1], dir[2]]
            #self.spotlight_data [index+3] = [light._light.attenuation[0], light._light.attenuation[1], light._light.attenuation[2], 1.0]

            glBufferSubData(GL_UNIFORM_BUFFER, (index + 0) * 4 * 4, self.spotlight_data[index].nbytes, self.spotlight_data[index])
            #glBufferSubData(GL_UNIFORM_BUFFER, (index + 1) * 4 * 4, self.spotlight_data[index+1].nbytes, self.spotlight_data[index+1])
            glBufferSubData(GL_UNIFORM_BUFFER, (index + 2) * 4 * 4, self.spotlight_data[index+2].nbytes, self.spotlight_data[index+2])
            #glBufferSubData(GL_UNIFORM_BUFFER, (index + 3) * 4 * 4, self.spotlight_data[index+3].nbytes, self.spotlight_data[index+3])

        glBindBuffer(GL_UNIFORM_BUFFER, 0)

        self.render.setShaderInput('SpotLightCount', len(self.spotlights))


    def add_spotlight(self, light):
        self.spotlights.append(light)

    def remove_spotlight(self, light):
        self.spotlights.remove(light)


    def change_spotlight_fov(self, light):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        cos = math.cos(math.radians(light._light.getLens().get_fov()[0] * 0.5))
        index = self.spotlights.index(light) * 4
        self.spotlight_data[index+2, 3] = cos
        glBindBuffer(GL_UNIFORM_BUFFER, 0)
        #print('CHANGING FOV!', cos)


    def change_spotlight_color(self, light):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        index = self.spotlights.index(light) * 4
        self.spotlight_data[index+1, :3] = [light._light_color[0], light._light_color[1], light._light_color[2]]
        glBufferSubData(GL_UNIFORM_BUFFER, (index + 1) * 4 * 4, self.spotlight_data[index + 1].nbytes,self.spotlight_data[index + 1])
        glBindBuffer(GL_UNIFORM_BUFFER, 0)
        #print('CHANGING COLOR!')


    def change_spotlight_exponent(self, light):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        #print('CHANGING EXPONENT!')
        index = self.spotlights.index(light) * 4
        self.spotlight_data[index + 1, 3] = light._light.getExponent()
        glBufferSubData(GL_UNIFORM_BUFFER, (index + 1) * 4 * 4, self.spotlight_data[index + 1].nbytes,self.spotlight_data[index + 1])
        glBindBuffer(GL_UNIFORM_BUFFER, 0)


    def change_spotlight_attenuation(self, light):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        #print('CHANGING ATTENUATION!')
        index = self.spotlights.index(light) * 4
        self.spotlight_data[index+3, :3] = [light._light.attenuation[0], light._light.attenuation[1], light._light.attenuation[2]]
        glBufferSubData(GL_UNIFORM_BUFFER, (index + 3) * 4 * 4, self.spotlight_data[index + 3].nbytes,self.spotlight_data[index + 3])
        glBindBuffer(GL_UNIFORM_BUFFER, 0)


    def change_spotlight_on(self, light):
        glBindBuffer(GL_UNIFORM_BUFFER, self.spotlight_buffer)
        #print('CHANGING LIGHT ON / OFF!')
        index = self.spotlights.index(light) * 4
        self.spotlight_data[index, 3] = float(light._on)
        glBindBuffer(GL_UNIFORM_BUFFER, 0)

glsl shader code:

uniform sampler2DArray SpotLightShadows;

struct PointLightParameters {
    vec4 position_on;
    vec4 color_exponent;
    vec4 direction_cutoff;
    vec4 attenuation;
};

layout(std140) uniform SpotLightBuffer {
    PointLightParameters pointlights[64];
};

uniform int SpotLightCount;

when I use panda3ds ShaderBuffer Object, it no longer works. I don’t change anything in my shader code. Just the python side.

#Cutt down version
def update_lights_transforms():
  ubo = ShaderBuffer("buffer", data, ShaderBuffer.UHDynamic)
  
  #then
  
  render.setShaderInput("SpotLightBuffer", ubo)

There aren’t any errors, but my glsl shader doesn’t seem to be retrieving any values.
SSBOs work perfectly but when I switch to a UBO, it all fails when using panda3ds system.

29

I’ve tested both texture based and UBO based data transmission to the and actually found that 1D textures perform faster than UBOs. The only weird trouble I had with textures is that their channels were reversed from RGBA to BGRA. I tried forcing RGBA with texture.setRamImageAs(data, "RGBA"), but it still didn’t work. I think this may have something to do with either panda3d, or my graphics driver uploading the texture to ram as BGRA. Is there a way to force RGBA?

30

If you want to support older GPUs, you should use buffer textures. Buffer textures are similar to SSBOs in that they provide buffer-backed storage to a shader, except they require an OpenGL 3 level GPU instead of an OpenGL 4 level GPU and they will work on macOS.

ShaderBuffer is basically a bit more ergonomic to use from the shader since you can use a struct.

Uniform buffers aren’t supported, so your previous shader code won’t work.

1 Like
31

yea I guess I will stick with that. however do you maybe have a solution to the BGRA texture upload problem? No matter what I do, I cant get the texture forced as RGBA.

basic code:

        self.spotlight_texture = p3d.Texture()
        self.spotlight_texture.set_compression(p3d.Texture.CMOff)
        self.spotlight_texture.setup_1d_texture(self.MAX_LIGHTS * 9, p3d.Texture.T_float, p3d.Texture.F_rgba32)
        self.spotlight_texture.set_keep_ram_image(True)

        self.spotlight_data = np.ndarray(shape=(self.MAX_LIGHTS * 9, 4),dtype=np.float32) #np.zeros(self.MAX_LIGHTS * 5 * 3, dtype=np.float32)

        self.spotlight_texture.set_ram_image_as(self.spotlight_data.tobytes(),"RGBA")

        self.spotlight_data = np.ndarray(shape=(self.MAX_LIGHTS * 9, 4),dtype=np.float32, buffer=self.spotlight_texture.modify_ram_image())

when edit the code:

    def change_spotlight_value(self, light, channel, value):
        if light in self.spotlights.keys():
            index = self.spotlights[light] * 9
            self.spotlight_data[index + channel] = [value[0], value[1], value[2], 1.0]

then call:

self.spotlight_texture.modifyRamImage()

to update the texture.

32

set_ram_image_as should be working, but honestly, it’s more efficient to just accept BGRA order for now and do the switcheroo in the shader. It’s more efficient that way.
This will be remedied in a future version of Panda.

33

Its just I’m a bit worried for compatibility reasons. I’m not sure that every computer will upload the texture as BGRA. Like you said, I have some a swizzle switch in my shader code, but will this be safe to do for every device that runs this code?

34

Yes, the behaviour will not be different for different devices.

35

I will sum up this thread for anyone also interested in making a lighting system as I will mark this thread as solved:

If you want to make a compatible lighting system that supports many lights, use texture buffers with RGB float32 or RGBA float 32, as I found they are faster to update than UBOs (and they aren’t supported with the current version 1.10.16). Just watch out for the BGRA upload format in your shader.

If you are wanting the best performance and are using modern OpenGL (4.30+), then go for SSBOs. @Baribal made a fantastic library with examples for Shader Buffers where you can start.

Both techniques will require you to have either shadow texture arrays, or shadow atlases’.

Again, thank you everyone in this thread for all your information and help!