Help with texture processing using RenderToTexture

Hi All,

I’m trying to implement the jump flooding algorithm explained here:

(There are several shadertoys included in the article but they’re not written in GLSL)

using render to texture with panda3d and Python as part of a GPU port of a larger project, but I haven’t been able to piece together how to make basic render to texture work yet.

Currently, I’m importing:

import panda3d.core as pc
from direct.showbase.ShowBase import ShowBase

Setting up a frame buffer to render to:

texW = 1024
texH = 512

base = ShowBase()
# Request 8 RGB bits, 8 alpha bits, and a depth buffer.
fb_prop = pc.FrameBufferProperties()
fb_prop.setRgbColor(True)
fb_prop.setRgbaBits(8, 8, 8, 8)
fb_prop.setDepthBits(16)

# Create a WindowProperties object set to size.
win_prop = pc.WindowProperties(size=(texW, texH))

# Don't open a window - force it to be an offscreen buffer.
flags = pc.GraphicsPipe.BF_refuse_window

# Create a GraphicsBuffer to render to, we'll get the textures out of this
tempTex = pc.Texture()
buffer = base.graphicsEngine.makeOutput(base.pipe, "Buffer", -100, fb_prop, win_prop, flags, base.win.getGsg(), base.win)
buffer.addRenderTexture(tex=tempTex, mode=pc.GraphicsOutput.RTMBindOrCopy, bitplane=pc.GraphicsOutput.RTPColor)

Creating a scene graph, card, and camera:

# Create a scene graph, a camera, and a card to render to
cm = pc.CardMaker('card')
canvas = pc.NodePath("Scene")
card = canvas.attachNewNode(cm.generate())
cam2D = base.make_camera(buffer)
lens = pc.OrthographicLens()
lens.setFilmSize(2, 2)
lens.setNearFar(-1000, 1000)
cam2D.node().setLens(lens)
cam2D.reparentTo(canvas)

Loading and attaching the shaders:

voronoiShader = pc.Shader.load(pc.Shader.SL_GLSL, vertex="quad.vert", fragment="jumpflood.frag")
card.set_shader(voronoiShader)

My vertex shader looks like:

#version 150

// Vertex inputs
in vec4 p3d_Vertex;
in vec2 p3d_MultiTexCoord0;

// Output to fragment shader
out vec2 texcoord;

void main() {
  gl_Position = p3d_Vertex;
  texcoord = p3d_MultiTexCoord0;
}

My fragment shader looks like:

#version 430

// Input from vertex shader
in vec2 texcoord;

// Output to the buffer
out vec4 p3d_FragColor;

void main()
{
    p3d_FragColor.rgba = vec4(1.0);
}

I’m rendering to the buffer:

base.graphicsEngine.renderFrame()
tex = buffer.getScreenshot()

And I’m writing the texture to the disk:

tex.write("jumpflood.png")

This doesn’t throw any errors, but the following message is printed to the console:

Known pipe types:
  wglGraphicsPipe
(all display modules loaded.)

A black window appears and promptly vanishes (it’s not 1024x512, it looks more like 800x600), and jumpflood.png is output as a 1024x512 completely transparent image. I suspect that I’ve made quite a few mistakes, but I’ve been trawling through the documentation for a few days now with no luck in fixing them thus far, so I thought I’d ask here for help.

I also wasn’t sure how alpha was handled in the fragment shader so I tried:

p3d_FragColor.rgba = vec4(1.0, 1.0, 1.0, 0.0);

as well, but the output was still transparent.

I am also particularly confused by the fact that a window pops up every time I run the program even though my buffer uses the flag:

pc.GraphicsPipe.BF_refuse_window

Apologies if this post is a bit long, but I thought my mistake(s) could be anywhere in my code, so I wanted to cover all my bases by providing as much as possible up front.

You should not use a base instance. You will need a template something like this.

Hi, thanks for your reply, sorry for the delay in getting back to you:

I made the following changes:

# Setup the engine stuff
engine = pc.GraphicsEngine.get_global_ptr()
pipe = pc.GraphicsPipeSelection.get_global_ptr().make_module_pipe("pandagl")
# Create a GraphicsBuffer to render to, we'll get the textures out of this
tempTex = pc.Texture("tempTex")
buffer = engine.makeOutput(pipe, "Buffer", 0, fb_prop, win_prop, flags)
buffer.addRenderTexture(tex=tempTex, mode=pc.GraphicsOutput.RTMBindOrCopy, bitplane=pc.GraphicsOutput.RTPColor)
# Create a scene graph, a camera, a display region, and a card to render to
cm = pc.CardMaker('card')
canvas = pc.NodePath("Scene")
card = canvas.attachNewNode(cm.generate())
cam2D = pc.Camera("Camera")
lens = pc.OrthographicLens()
lens.setFilmSize(2, 2)
lens.setNearFar(-1000, 1000)
cam2D.setLens(lens)
camera = pc.NodePath(cam2D)
camera.reparentTo(canvas)
display_region = buffer.makeDisplayRegion()
display_region.camera = camera
# Do the rendering
engine.renderFrame()
tex = buffer.getScreenshot()

I haven’t changed anything else. When I run this now I don’t get any console message, no errors or warnings, and the jumpflood.png is generated but is still completely transparent.

I think the problem is that the camera and the card are at the same point.
camera.setPos(0, -1, 0)
Try to move the camera back.

Here is a slightly modified code.

import panda3d.core as pc

# Setup the engine stuff
engine = pc.GraphicsEngine.get_global_ptr()
pipe = pc.GraphicsPipeSelection.get_global_ptr().make_module_pipe("pandagl")

texW = 1024
texH = 512

# Request 8 RGB bits, 8 alpha bits, and a depth buffer.
fb_prop = pc.FrameBufferProperties()
fb_prop.setRgbColor(True)
fb_prop.setRgbaBits(8, 8, 8, 8)
fb_prop.setDepthBits(16)

# Create a WindowProperties object set to size.
win_prop = pc.WindowProperties(size=(texW, texH))

# Don't open a window - force it to be an offscreen buffer.
flags = pc.GraphicsPipe.BF_refuse_window

# Create a GraphicsBuffer to render to, we'll get the textures out of this
tempTex = pc.Texture("tempTex")
buffer = engine.makeOutput(pipe, "Buffer", 0, fb_prop, win_prop, flags)
#buffer.addRenderTexture(tex=tempTex, mode=pc.GraphicsOutput.RTMBindOrCopy, bitplane=pc.GraphicsOutput.RTPColor)
buffer.add_render_texture(tempTex, pc.GraphicsOutput.RTM_copy_ram)

# Create a scene graph, a camera, a display region, and a card to render to
cm = pc.CardMaker('card')
canvas = pc.NodePath("Scene")

card = canvas.attachNewNode(cm.generate())

cam2D = pc.Camera("Camera")
lens = pc.OrthographicLens()
lens.setFilmSize(2, 2)
lens.setNearFar(-1000, 1000)
cam2D.setLens(lens)
camera = pc.NodePath(cam2D)
camera.reparentTo(canvas)
camera.setPos(0, -1, 0)
display_region = buffer.makeDisplayRegion()
display_region.camera = camera

engine.render_frame()

frame = pc.PNMImage()
tempTex.store(frame)
frame.write(pc.Filename('render.png'))

Hi, thanks for getting back to me so quickly.

Thanks to your code I’ve isolated my problem to my shader program. If I add:

voronoiShader = pc.Shader.load(pc.Shader.SL_GLSL, vertex="quad.vert", fragment="jumpflood.frag")
card.set_shader(voronoiShader)

before

engine.render_frame()

in your code, I get a transparent output, if I comment it out I get a white quad in the upper left quarter of render.png, which I’m assuming is either to do with the display region or the card coords.

This occurs for both:

p3d_FragColor.rgba = vec4(1.0, 1.0, 1.0, 0.0);

and:

p3d_FragColor.rgba = vec4(1.0, 1.0, 1.0, 1.0);

I think that this is because CardMaker, by default, produces cards with their origin at one corner. You should be able to centre the card by offsetting the card in the x- and y- by an appropriate amount, I daresay.

It does seem odd that it’s in the upper-left–I would have expected it to be in the upper-right, I think.

Looking at your code, I do wonder whether it’s not because–as serega says–you have your camera at the same position as your card, with your camera then having a negative near-value. I’m wondering, in short, whether the card isn’t being rendered as though behind your camera.

(Which may perhaps also be related to your output being transparent in some cases: you might be rendering the back-face of your card. However, that then prompts the question of why you’re seeing your card at all, in any case…)

So, what happens if you move your card forward a bit (i.e. something like card.setY(1)), and change your “setNearFar” call to something like lens.setNearFar(0.1, 1000)?

[edit]
Although a quick test on my end results in the card always being rendered at the top-right–even when placed explicitly behind the camera. Thus I suspect that there’s something wrong elsewhere in your setup–perhaps in your frame-buffer properties, or your display region, or something like that…

Hi, thanks for your reply. I got my lefts and rights mixed up, you’re correct, that card is always in the top right.

I tried card.setY(1) and lens.setNearFar(0.1, 1000) and it still renders in the top right, if I do camera.setPos(0.5, -1, 0.5) however, the card is centred in the image, but is still 1/4 full size. This happens whether I use card.setY(1) or not, which leads me to believe that the card’s y was already 1.

I still haven’t fixed the shader issue though, as soon as I use the shader I can’t see the card at all.

Vertex:

#version 150

// Vertex inputs
in vec4 p3d_Vertex;
in vec2 p3d_MultiTexCoord0;

// Output to fragment shader
out vec2 texcoord;

void main() {
  gl_Position = p3d_Vertex;
  texcoord = p3d_MultiTexCoord0;
}

Fragment:

#version 430

// Input from vertex shader
in vec2 texcoord;

// Output to the buffer
out vec4 p3d_FragColor;

void main()
{
    p3d_FragColor.rgba = vec4(1.0, 1.0, 1.0, 1.0);
}

I don’t know if its because I’m not using the camera’s projection matrix in the vertex shader or what.

That’s actually the expected behaviour–much of my post was me being thrown by the left-right mix-up! ^^;

As I said:

(You can also adjust the CardMaker object to produce a centred card, but moving the generated card may be (slightly) simpler.)

This I don’t know about offhand, I’m afraid, so I’ll leave it to others for now, I think!

[edit]
Ah, sorry, I missed this element earlier:

That’s likely just the console popping up; I’m guessing that your program is a command-line one, and hence runs from a console. Since the program exits almost immediately, the console lasts only a very brief time, and so flashes onto the screen as you’ve seen.

The flag that you mention should prevent the program from opening a graphical window, and I imagine has no effect on your program being command-line-based.

I managed to figure it out, the vertex shader needs to include the model view projection matrix:

Vertex shader:

#version 150

// Uniform inputs
uniform mat4 p3d_ModelViewProjectionMatrix;

// Vertex inputs
in vec4 p3d_Vertex;
in vec2 p3d_MultiTexCoord0;

// Output to fragment shader
out vec2 texcoord;

void main() {
  gl_Position = p3d_ModelViewProjectionMatrix * p3d_Vertex;
  texcoord = p3d_MultiTexCoord0;
}

It’s as simple as that.

I also got the card to appear correctly in the frame using:

card.setZ(-1)
card.setX(-1)
card.setScale(2)
2 Likes