Crash on CullTraverser C++ -- (Actually a NVidia-cg-toolkit issue)

Hi everyone,

I’m facing a strange crash. I’m using Panda version 1.10.9 (commit 51b6a90fcb27ae338838b68d14cc09cc3463486e) compiled from source on ubuntu 20.04.
Graphic card is a Quadro RTX4000 (proprietary driver)

The crash seems to be on the cull_to_bin, here the stack:

(gdb) bt full
#0  0x00007fff60746886 in CullTraverser::traverse_below(CullTraverserData&) [clone .cold] () at .../panda3d/build/lib/libpanda.so.1.11
#1  0x00007fff60bb91b9 in CullTraverser::do_traverse(CullTraverserData&) () at .../panda3d/build/lib/libpanda.so.1.11
#2  0x00007fff60ba5e99 in CullTraverser::traverse_below(CullTraverserData&) () at .../panda3d/build/lib/libpanda.so.1.11
#3  0x00007fff60bb91b9 in CullTraverser::do_traverse(CullTraverserData&) () at .../panda3d/build/lib/libpanda.so.1.11
#4  0x00007fff60ba5b9b in CullTraverser::traverse(NodePath const&) () at .../panda3d/build/lib/libpanda.so.1.11
#5  0x00007fff60833951 in GraphicsEngine::do_cull(CullHandler*, SceneSetup*, GraphicsStateGuardian*, Thread*) ()
    at .../panda3d/build/lib/libpanda.so.1.11
#6  0x00007fff60833b3e in GraphicsEngine::cull_to_bins(GraphicsOutput*, GraphicsStateGuardian*, DisplayRegion*, SceneSetup*, CullResult*, Thread*) ()
    at .../panda3d/build/lib/libpanda.so.1.11
#7  0x00007fff60849eef in GraphicsEngine::cull_to_bins(ov_set<PointerTo<GraphicsOutput>, IndirectLess<GraphicsOutput>, pvector<PointerTo<GraphicsOutput> > >, Thread*) ()
    at .../panda3d/build/lib/libpanda.so.1.11
#8  0x00007fff6084a546 in GraphicsEngine::WindowRenderer::do_frame(GraphicsEngine*, Thread*) () at .../panda3d/build/lib/libpanda.so.1.11
#9  0x00007fff6084ae6b in GraphicsEngine::render_frame() () at .../panda3d/build/lib/libpanda.so.1.11
#10 0x00007fff7c214855 in PandaFramework::task_igloop(GenericAsyncTask*, void*) () at .../panda3d/build/lib/libp3framework.so.1.11
#11 0x00007fff6088628f in AsyncTask::unlock_and_do_task() () at .../panda3d/build/lib/libpanda.so.1.11
#12 0x00007fff60894fa6 in AsyncTaskChain::service_one_task(AsyncTaskChain::AsyncTaskChainThread*) () at .../panda3d/build/lib/libpanda.so.1.11
#13 0x00007fff608972f3 in AsyncTaskChain::do_poll() () at .../panda3d/build/lib/libpanda.so.1.11
#14 0x00007fff60897b43 in AsyncTaskManager::poll() () at .../panda3d/build/lib/libpanda.so.1.11
#15 0x00007fff7c21585f in PandaFramework::main_loop() () at .../panda3d/build/lib/libp3framework.so.1.11

I use a single Task to run all my code, I don’t have any other threads running in parallel.

taskMgr->add(new GenericAsyncTask(“Run all code”, &Runcode, (void*) &data));

I do use some culling-related features, like bins, depth_offset and

set_bounds(new OmniBoundingVolume());
node()->set_final(true);

Anyway it seemed to work pretty well until recently. I haven’t done big changes other than adding shadows, which shouldn’t be related (BTW thanks for helping me to make shadows working).

Unfortunately I’m not able to create a dummy example to reproduce this issue, and it’s not deterministic. Everything works well for like 40minutes and then the crash happens.

I’have not specified any threading model in config file, the only config changes are:

  • framebuffer-multisample 1
  • multisamples 1
  • sync-video 1

What may I do to find the cause of this crash? Any hint to what to look for?

Many thanks!

Hi guys,

it seems I found out the issue. It was NOT related to the CullTraverser, not sure why gdb was pointing me there.

It was instead related to the shadows I recently introduced via the Nvidia-cg-toolkit. Disabling them the crash disappeared. Reading throughout the documentation I found the ‘known issues’ page about shaders and I got the ‘basic-shaders-only’ option.

Enabling this option everything seems to work correctly, I have the shadows and no more random crashes.

Thank you for providing this option, it really saved my life.

Anyway I was wondering if there are alternatives to this Nvidia-cg-toolkit, since it is no longer supported. In my case, my application was working fine on 2 different PCs with different NVidia cards, but was crashing on a different one. My fear is that getting more recent graphic cards at a certain point even the ‘basic-only’ option will no longer be effective.

Thanks

1 Like