Rendering problem on Beyond TV

firefly · March 22, 2022, 8:44am

I have a problem with running my Panda application on the computer of someone I work with. He has 4 monitors connected. the left one is a Dell P2419H, and the other 3 are, as far as I can see via a TeamViewer login, Beyond TV. In my application, the leftmost TV is at [1920,0], 2nd TV is at [3840,0] and the third TV is at [5760,0]. So all 4 displays are 1920x1080. I also tested this with the Fireflies demo and positioned the window are various positions, I added these 3 lines with props:

class FireflyDemo(ShowBase):
def init(self):
# Initialize the ShowBase class from which we inherit, which will
# create a window and set up everything we need for rendering into it.
ShowBase.init(self)
self.setBackgroundColor((0, 0, 0, 0))
props = WindowProperties()
props.setSize(500, 500)
props.setOrigin(600, 0)
base.win.requestProperties(props)

and set the position at various positions. As long as the position of the window was in the region of the regulal monitor (Dell), so props.setOrigin(x, 0), with x between 0 and 1920, then the rendering was fine. However, if I set it to larger than 1920, so it was rendered on either one of the TVs, then nothing happened. No rendering at all, and Panda seemed to be in an infinite loop (spinning circle when mouse hovers over the application.
So, its as if Panda can’t find out where to render the images.
All displays are set at 60 Hz, there’s one thing that is different for the TVs: HDR Certification reports Not Found.
Does anyone have an idea what could cause this?

rdb · March 22, 2022, 9:41am

Interesting. You have not specified which OS, so I’m going to assume Windows?

You could set notify-level-display spam in Config.prc to perhaps get a clue why Panda is not rendering.

firefly · March 22, 2022, 11:08am

Yes, its Windows 10. The things gets even weirder. I added this to the Fireflies demo.:
loadPrcFileData("", “notify-level-display spam” )
ShowBase.init(self)
self.setBackgroundColor((0, 0, 0, 0))
props = WindowProperties()
props.setSize(500, 500)
props.setOrigin(1921, 0)

Then it renders normally, and gives a lot of debug information, however, it does not start at 1921,0 but at 0,0.
When I add # loadPrcFileData("", “notify-level-display spam” ), so I comment it out, then it hangs again. So, no output, no rendering.
The console output of spam is gigantic, and I can attach the file with the output to this message. But this is the part where it changes the origin to 1921:
:display:windisplay(debug): reshape to origin: (1921,0), size: (500,500)
:display(debug): system_changed_properties(origin=(1921, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)
:display:windisplay(spam): 1.84697 window_proc(0000017756D29870, 00000000000A0752, 124, 18446744073709551600, 471552940544)
:display:windisplay(spam): 1.848 window_proc(0000017756D29870, 00000000000A0752, 125, 18446744073709551600, 471552940544)
:display:windisplay(spam): 1.84942 window_proc(0000017756D29870, 00000000000A0752, 131, 1, 471552940528)
:display:windisplay(spam): 1.85044 window_proc(0000017756D29870, 00000000000A0752, 133, 1, 0)
:display:windisplay(spam): 1.85188 window_proc(0000017756D29870, 00000000000A0752, 20, 1694567575, 0)
:display:windisplay(spam): 1.85273 window_proc(0000017756D29870, 00000000000A0752, 71, 0, 471552940576)
:display:windisplay(debug): WM_WINDOWPOSCHANGED: 00000000000A0752, 0
:display:windisplay(debug): reshape to origin: (1921,0), size: (500,500)
:display(debug): system_changed_properties(origin=(1921, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)
:display:windisplay(spam): 1.85581 window_proc(0000017756D29870, 00000000000A0752, 12, 0, 471552940704)
:display:windisplay(spam): 1.85655 window_proc(0000017756D29870, 00000000000A0752, 174, 9, 0)
:display:windisplay(spam): 1.85753 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 3934549)
:display:windisplay(spam): 1.85846 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 5309117)
:display:windisplay(spam): 1.85925 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 985461)
:display:windisplay(spam): 1.86001 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 9503881)
:display:windisplay(spam): 1.86075 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 3213319)
:display:windisplay(spam): 1.86143 window_proc(0000017756D29870, 00000000000A0752, 255, 0, 1313101)
:display:windisplay(spam): 1.86217 window_proc(0000017756D29870, 00000000000A0752, 15, 0, 0)
:display:windisplay(spam): Got update regions: 0000017756D29870
:display(spam): begin_frame(refresh): GLGraphicsBuffer model buffer 000001775AC65F50
:display(spam): begin_frame(parasite): wglGraphicsWindow window1 0000017756D29870
:display:wgldisplay(spam): Drawing 0000017756D29870: exposed.
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display:gsg:glgsg(debug): Binding texture to depth attachment.
:display:gsg:glgsg(spam): glBindTexture(0xde1, 1):
:display:gsg:glgsg(debug): loading texture with NULL image
:display:gsg:glgsg(debug): loading new texture object for , 512 x 512 x 1, z = 0, mipmaps 0, uses_mipmaps = 0
:display:gsg:glgsg(debug): (initializing NULL image)
:display:gsg:glgsg(spam): glBindTexture(0xde1, 1):
:display:gsg:glgsg(debug): Binding texture to stencil attachment.
:display:gsg:glgsg(spam): glBindTexture(0xde1, 1):
:display:gsg:glgsg(debug): Binding texture to color attachment.
:display:gsg:glgsg(spam): glBindTexture(0xde1, 2):
:display:gsg:glgsg(debug): loading texture with NULL image
:display:gsg:glgsg(debug): loading new texture object for , 512 x 512 x 1, z = 0, mipmaps 0, uses_mipmaps = 0
:display:gsg:glgsg(debug): (initializing NULL image)
:display:gsg:glgsg(spam): glBindTexture(0xde1, 2):
:display:gsg:glgsg(debug): Binding texture to color attachment.
:display:gsg:glgsg(spam): glBindTexture(0xde1, 3):
:display:gsg:glgsg(debug): loading texture with NULL image
:display:gsg:glgsg(debug): loading new texture object for , 512 x 512 x 1, z = 0, mipmaps 0, uses_mipmaps = 0
:display:gsg:glgsg(debug): (initializing NULL image)
:display:gsg:glgsg(spam): glBindTexture(0xde1, 3):
:display(spam): end_frame(refresh): GLGraphicsBuffer model buffer 000001775AC65F50
:display(spam): end_frame(parasite): wglGraphicsWindow window1 0000017756D29870

firefly · March 22, 2022, 11:14am

And when I use loadPrcFileData("", “notify-level-display warning” ) instead of spam, a panda3d window is opened at the center of the main display (left one) with nothing in it, no messages in the dos window that starts the rendering and panda3d hangs (no response).

firefly · March 22, 2022, 11:33am

Oh, by the way, its windows 11 Home.
Panda version: 1.10.10-x64.
GPU: RTX3090
CPU: Intel core i7-12700F

rdb · March 22, 2022, 12:03pm

Are you saying it does not hang with spam set? Or is it simply so slow due to the amount of console output that it’s never getting to that point? If you set notify-output file.txt then the execution should not be slowed down significantly.

Are all the displays hooked to the same graphics card, or to different ones?

I am travelling until next week, so I can unfortunately not test anything until I get back.

firefly · March 22, 2022, 12:37pm

Yes, it does not hang with spam set but it does hang with warning. The firefly demo works normally, you see the fireflies coming up with loadPrcFileData("", “notify-level-display spam” ). Except ofcourse that the display comes up at [0,0], so the upper left corner of the leftmost display instead of at 1921, 0. So it comes up at the wrong display.

firefly · March 22, 2022, 12:59pm

This:
loadPrcFileData("", “notify-level-display spam\n”
“notify-output file.txt” )
ShowBase.init(self)
results in immediate hanging, alsthough data are writte to file.txt. The panda window stays completely empty.

This, however:
loadPrcFileData("", “notify-level-display spam” )
ShowBase.init(self)
lets the demo run perfectly fine, you see all the fireflies coming up, and it gives a continuous stream of messages to the console.

I know, very strange.
All 4 displays are connected to the same graphics card. I can send you the file.txt output file, but I don’t think I can attach that to this message…

rdb · March 22, 2022, 1:11pm

Please attach or send the file, right up to the hang (I assume the output stops there).

I will privately send you my email address.

firefly · May 1, 2022, 2:35pm

After it was solved by a complete re-installation of windows, the same problem occured with another computer. I have noticed this now with computers with fast processors during the last couple of months, now an intel i9-12900 KF processor, before that with an computer with an intel i7 12700 F CPU, and before that with computers with Ryzen processors. It occurs with windows 10 and windows 11, and with Panda1.10.10, 1.10.11 and 1.10.12. I haven’t tested it with older versions. So the problem is that when opening a panda window on any monitor that is not the main monitor, panda hangs as if it is in an endless loop (panda not responding) immediately after the start of the second frame.
I use the samples\firefly demo for that, but it occurs on any panda application. Its not a matter of repositioning the window to another position, it also occurs with this added to the firefly demo:

class FireflyDemo(ShowBase):
def init(self):
ShowBase.init(self, windowType=“none”)
props = WindowProperties(WindowProperties.getDefault())
props.setOrigin(1920, 0)
self.openDefaultWindow(props=props)
…

In this last case it was on the following hardware:
GPU: NVidia RTX3080 Ti
both tried windows 10 and windows 11 64 bits, no difference
128 GB RAM
CPU: intel 19-12900 KF
4 x 1920x1080 monitors connected.

Does anyone have any idea what could be he problem ? It always worked properly on a large number of installations before that, also on RTX3080 GPU’s…

Thanks.

firefly · May 2, 2022, 8:13am

I ran the exact same test on the computer that hangs and one that renderes normally with:
loadPrcFileData("", “notify-level-display spam\n”
“notify-output file.txt” )
ShowBase.init(self, windowType=“none”)
etc…
While the output is mostly the same, there are a few differences:

[from line 459 in the computer that hangs]
:display:windisplay(debug): reshape to origin: (1920,0), size: (500,500)
:display(debug): system_changed_properties(origin=(1920, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display(debug): DisplayRegion::do_compute_pixels(500, 500)
:display:windisplay(debug): reshape to origin: (1920,0), size: (500,500)
:display(debug): system_changed_properties(origin=(1920, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)
:display:windisplay(spam): 0.204916 window_proc(000002562F7F2D10, 00000000001001C8, 124, 18446744073709551600, 732635782768)
:display:windisplay(spam): 0.205078 window_proc(000002562F7F2D10, 00000000001001C8, 125, 18446744073709551600, 732635782768)
:display:windisplay(spam): 0.20555 window_proc(000002562F7F2D10, 00000000001001C8, 131, 1, 732635782752)
:display:windisplay(spam): 0.206412 window_proc(000002562F7F2D10, 00000000001001C8, 133, 1, 0)
:display:windisplay(spam): 0.207229 window_proc(000002562F7F2D10, 00000000001001C8, 20, 251725638, 0)
:display:windisplay(spam): 0.20786 window_proc(000002562F7F2D10, 00000000001001C8, 71, 0, 732635782800)
:display:windisplay(debug): WM_WINDOWPOSCHANGED: 00000000001001C8, 0
:display:windisplay(debug): reshape to origin: (1920,0), size: (500,500)
:display(debug): system_changed_properties(origin=(1920, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)

[from line 647 in the computer that hangs]
:display(spam): render_frame() - frame 2
:display(spam): Culling window model buffer
:display(spam): Culling window model buffer
:display(spam): Culling window window1
:display(spam): Flipping window model buffer

[and here it stops, while normally the output continues with:]
:display(spam): begin_frame(render): GLGraphicsBuffer model buffer 000001F8235A6900
:display(spam): begin_frame(parasite): wglGraphicsWindow window1 000001F821C1D150
:display:wgldisplay(spam): Drawing 000001F821C1D150: exposed.
:display(spam): clear(): GLGraphicsBuffer model buffer 000001F8235A6900
:display:gsg:glgsg(spam): glDisable(GL_SCISSOR_TEST)
:display(spam): Drawing window model buffer
:display:gsg:glgsg(spam): glEnable(GL_SCISSOR_TEST)
etc…

So
:display(debug): system_changed_properties(origin=(1920, 0) size=(500, 500) )
:display(debug): system_changed_size(500, 500)

occurs three times in a row on the computer that hangs, while it only occurs once in the computer that works normally.

the last debug line is
:display(spam): Flipping window model buffer
after fendering frame 2 in the computer that hangs.

firefly · May 3, 2022, 2:34pm

So, I did a number of things to check if that helped. So, 4 monitors are connect. As long as I render the firefly demo on the main display (which starts at [0,0]), the rendering is completely fine. If I render outside the main display the rendering hangs completely (panda not responding). It does not matter which monitor is made main display, for example if the second monitor is main display, then it renders fine on the 2nd monitor, but not if I set props.setOrigin(1920, 0) or props.setOrigin(-500, 0) or anything outside of the main display.
I tried the following:

installed the latest NVidia driver version 512.59: not helping
installed an older version driver 466.77 : not helping
switched vertical sync off and loadPrcFileData("", “sync-video 0”): not helping
loadPrcFileData("", “auto-flip 1”): not helping
loadPrcFileData("", “support-threads 0”): not helping
Also, a known issue with freezing of the rendering outside the main display is when Instant replay (of Geforce experience) is off, but I have tried switching Geforce experience off and I have checked whether instant replay is of when Geforce experience is running and neither of these helps.

This is the 2nd time this problem occurs during the last few weeks on 2 different new computers with the latest CPU’s of intel and latest GPU models (RTX3080 ti and RTX3090), but I have not experienced this problem on other RTX3080 GPU’s.

I think we will see this problem more, but I am out of ideas for now, any help appreciated.

rdb · May 5, 2022, 9:02am

The new output confirms that the hang most likely occurs in the SwapBuffers call.

Hmm, this is vexing. At this point it is hard for me to diagnose the problem without physically having access to the set-up or otherwise being able to use development tools to diagnose the issue and try code changes on the affected machine itself. I don’t suppose that the installation is situated in the Netherlands?

That said, let’s first determine whether the problem has anything to do with Panda3D. From googling around it seems that this problem has also been reported with other toolkits:
https://bugreports.qt.io/browse/QTBUG-96628
https://bugreports.qt.io/browse/QTBUG-72557
To further confirm that it is not something related to Panda3D, you could try and run other OpenGL-based applications on the non-primary monitor and see if they hang. If they run fine, then there’s something we can do in Panda3D, if not, then there is no use looking further at making changes to Panda3D.

In the case of that bug, the problem was an “Asus Sonic Studio” service that was uninstalled to resolve the issue. Perhaps you could open up services.msc on the machine and see if a service like that exists? Or if there is a difference in running/installed services between the machines?

firefly · May 5, 2022, 9:38am

The computer is in Lousiana USA and I access it via teamviewer. I will check your suggetsions.
Another thing I would like to check is to see if hyperthreading is the issue, because that is enabled on that computer while on most other systems an i7-9700 CPU without hyperthreading is used.

firefly · May 6, 2022, 4:53pm

I have tested with hyperthreading disabled: no effect
I tried services.msc but there were no suspicious remnants of drivers, no traces of Asus Sonic Studio or Fanatec drivers
I installed the heaven benchmark 4.0 for OpenGL en did the Wall Auto test with 4 monitors, and there was no problem with SwapBuffers or any other problem. The OpenGL test went fine.

So I still don’t know why Panda hangs when opening on a non-main display. Its not a pure OpenGL things, because the heaven benchmark in OpenGL mode works fine on non-main monitors. So from what I can see, its a Panda3D thing.

firefly · May 6, 2022, 5:33pm

I just found out that if I reposition the origin (for example from 500,0 to 1920,0) a bit later, for example a second after the start of the program, then it renders fine, there’s only a jump of the window which is a bit annoying, but if that is needed to work around the problem then I can live with that.

rdb · May 9, 2022, 8:45am

Interesting! I’m glad you found a workaround at least.

I wonder if just rendering a single frame (base.graphicsEngine.renderFrame()) before positioning is enough? Maybe we could build something like that into the engine.

firefly · May 9, 2022, 5:40pm

rendering a single frame before positioning also does not work. And it seems that I spoke a bit too soon. Although the firefly demo works when postponing the window positioning a few seconds, I did not have that luck with my application, which still hangs even if I postpone the opening of the window on the other monitors for a couple of seconds. I don’t think I will be able to solve this so I will refund this customer.

firefly · May 11, 2022, 5:56pm

And now I have the exact same problem with yet another computer. It has an RTX2080 Super CPU and an i9-9900KS CPU, 16 cores, not in hyperthreading. A similarity with the previous computer where I had this problem is that that one had an i9-12900KF processor. So both computers where this problem of a rendering freezing occurs (with both the fireflies demo and my application) have an i9 intel CPU. Computers with an i7 CPU have never given this problem. Does anyone have any ideas?

rdb · May 11, 2022, 8:37pm

I’m sorry, I don’t have any obvious ideas. I can imagine this must be quite frustrating.

It’s certainly interesting that other OpenGL applications did not suffer from the same problem. If I had access to the set-up I would probably be trying to make random changes to the Panda source code to see which part may be causing the problem.

Is it only the Fireflies sample (besides your application) that is exhibiting the problem? Or do simpler sample programs also suffer from the issue?