Issues with ppackage and p3d

I’m attempting to make a couple of packages for the community to use (starting with yaml and google protocol buffers) and I am running into a problem. After compiling a program with the yaml package as a requirement the program stalls.

p3dcore.log

_root_dir = /home/croxis/.panda3d, _temp_directory = /tmp/, platform = linux_amd64, host_url = http://runtime.panda3d.org/, verify_contents = 0
Plugin version: 1.0.0
Plugin distributor: cmu
Core API host URL: http://runtime.panda3d.org/
Core API date: Sat Jan 23 06:46:06 2010

Creating P3DInstance 0xef6f60:  console_output="1" auto_start="1"
setting background to download, splash_window = 0
All 8 extracts of images seem good.
report_package_info_ready: images
Done installing images: success = 1
p3d_basename = citymania.p3d
p3d trusted
_matches_run_origin = 1
_matches_script_origin = 0
_auto_install = 1, _auto_start = 1, _stop_on_ready = 0
All 54 extracts of panda3d seem good.
report_package_info_ready: panda3d
Beginning install of 0 packages, total 0 bytes required.
All 0 extracts of models seem good.
report_package_info_ready: models
All 30 extracts of morepy seem good.
report_package_info_ready: morepy
make_temp_filename: /tmp/p3d_fe8051.xml
set_wparams: 1 640 480
image_filename = /home/croxis/.panda3d/hosts/runtime.panda3d.org_96a0ec538744def6/images/download.png
notify: onpluginload 
notify: onauth 
notify: ondownloadbegin 
notify: ondownloadcomplete 
Downloading http://croxis.dyndns.org/panda3d/contents.xml?1267147082: 0M, 0xec7ad0
Downloaded http://croxis.dyndns.org/panda3d/contents.xml?1267147082: 100%, 0xec7ad0, success = 1
release_temp_filename: /tmp/p3d_fe8051.xml
/home/croxis/.panda3d/hosts/croxis.dyndns.org_872451fe61ca8cac/yaml/yaml.xml is stale.
Downloading http://croxis.dyndns.org/panda3d/yaml/yaml.xml: 0M, 0xec8000
Downloaded http://croxis.dyndns.org/panda3d/yaml/yaml.xml: 100%, 0xec8000, success = 1
All 0 extracts of yaml seem good.
report_package_info_ready: yaml

packs.pdef

import sys
# For google protobufs
sys.path.append('/usr/lib/pymodules/python2.6/')
# For python only yaml
sys.path.append('.')

class yaml(package):
    config(display_name = "PyYAML")
    packager.setHost('http://croxis.dyndns.org/panda3d/')
    require('morepy')
    module('yaml')
    #dir('yaml')

class protobuf(package):
    config(display_name = 'Protocol Buffers')
    packager.setHost('http://croxis.dyndns.org/panda3d/')
    module('google.protobuf')

packp3d command

panda3d packp3d.p3d -o citymania.p3d -d client/ -p /home/croxis/src/ -p /usr/lib/pymodules/python2.6/ -r models -r morepy -r yaml,,http://croxis.dyndns.org/panda3d/

And packages are at croxis.dyndns.org/panda3d/

Thank you!

Hmm, it works just fine for me–I’m able to build a p3d file with a dependency on your yaml package, and run it, and it launches without objection after successfully downloading your yaml package.

Are you sure the problem is related to the use of yaml? From the logs, the last message is indicating the successful installation of yaml; but maybe the problem happened sometime after. What happens if you remove the dependency on yaml?

David

Without the dependency it builds and runs just fine.

Hmm, that’s certainly mysterious. Can you break into the panda3d executable with gdb when the process stalls and report what the stack trace looks like?

David

I’ve never used gdb before so forgive me if I got this wrong. Panda3d spawned two processes. Here is the backtrace of the first:

#0  0x00007fa380f8282d in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x00007fa37f86d2e0 in P3DConditionVar::wait(double) ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#2  0x00007fa37f8704b7 in P3DInstanceManager::wait_request(double) ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#3  0x00007fa37f87e46a in P3D_check_request ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#4  0x0000000000417aa1 in ?? ()
#5  0x0000000000413bd4 in ?? ()
#6  0x00000000004153b8 in ?? ()
#7  0x00007fa380277abd in __libc_start_main () from /lib/libc.so.6
#8  0x00000000004101b9 in ?? ()
#9  0x00007fff2f0dd188 in ?? ()
#10 0x000000000000001c in ?? ()
#11 0x0000000000000002 in ?? ()
#12 0x00007fff2f0de5ca in ?? ()
#13 0x00007fff2f0de5d2 in ?? ()
#14 0x0000000000000000 in ?? ()

And the second

#0  0x00007fa3803313c3 in select () from /lib/libc.so.6
#1  0x00007fa37f89ff12 in P3DX11SplashWindow::subprocess_run() ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#2  0x00007fa37f8a04a4 in P3DX11SplashWindow::start_subprocess() ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#3  0x00007fa37f88f64b in P3DInstance::make_splash_window() ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#4  0x00007fa37f8adc2d in P3DInstance::set_wparams(P3DWindowParams const&) ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#5  0x00007fa37f8b18e5 in P3D_instance_setup_window ()
   from /home/croxis/.panda3d/coreapi/linux_amd64/p3d_plugin.so
#6  0x0000000000416087 in ?? ()
#7  0x00000000004140aa in ?? ()
#8  0x00000000004153b8 in ?? ()
#9  0x00007fa380277abd in __libc_start_main () from /lib/libc.so.6
#10 0x00000000004101b9 in ?? ()
#11 0x00007fff2f0dd188 in ?? ()
#12 0x000000000000001c in ?? ()
#13 0x0000000000000002 in ?? ()
#14 0x00007fff2f0de5ca in ?? ()
#15 0x00007fff2f0de5d2 in ?? ()
#16 0x0000000000000000 in ?? ()

Hmm, it appears to be waiting for the splash window to be drawn, or something. Do you see the visible splash window when it hangs?

David

Nope, no window is being drawn. I even put a print statement in the very first line and that isn’t being displayed either.

Are you in a position to compile the plugin yourself? Try editing p3dX11SplashWindow.cxx, and add a few output statements in subprocess_run(), using nout, like this:

void P3DX11SplashWindow::
subprocess_run() {
  nout << "start subprocess_run\n";
  // Since we're now isolated in a subprocess, we can safely make all
  // the X calls we like, and run independently of the browser
  // process.

  make_window();
  nout << "done make_window\n";
  setup_gc();
  nout << "done setup_gc, context = " << _graphics_context << "\n";

I’d like to get a hint as to where it’s getting locked up. I suspect something is a little funny about X11 in your case, possibly make_window() is not returning.

I’m not sure why this would have anything to do with yaml, though. Does it work if you completely blow away your ~/.panda3d directory and force it to redownload everything?

David

This may be a weird wild guess, but can you call “ldd” on the yaml library to see if it pulls in any weird X11-related dependencies?

Oh! That’s an excellent guess–I should have thought of that!

But looking at the package itself, as offered by croxis’ server, it’s pure Python, and doesn’t involve any .so’s at all. It doesn’t even need to unpack anything out of the .mf file, and so it’s hard to see how simply downloading the .mf could interfere with X11.

Edit: hmm, but yaml does depend on morepy. Maybe the problem is actually in there. What happens if you make your p3d file depend directly on morepy, instead of yaml?

David

The window created after I nuked .pdanda3d to install my yaml package, and proceeded to the program just fine. I closed and restarted the program and the window did not create.

I did just the morepy dependency and the window created just fine. I’m going to try adding the plugin code and compiling the plugin.

Ok, I added the nout lines, compiled the run time and it doesn’t even print “start_subprocess” to terminal or the p3dcore.log

And yet your stack trace shows that you were within subprocess_run(). It should have written this output to p3dcore.log, though it might be interleaved with other output, because this is happening within a separate process.

Hmm, try cerr instead of nout, maybe it will be able to write to the console.

David

Nothing in terminal using cerr either :confused:

_root_dir = /home/croxis/.panda3d, _temp_directory = /tmp/, platform = linux_amd64, host_url = http://runtime.panda3d.org/, verify_contents = 0
Plugin version: 1.0.0c
Plugin distributor: croxis
Core API host URL: http://runtime.panda3d.org/
Core API date: Sat Jan 23 06:46:06 2010

Creating P3DInstance 0x20ebfa0:  console_output="1" auto_start="1"
setting background to download, splash_window = 0
All 8 extracts of images seem good.
report_package_info_ready: images
Done installing images: success = 1
p3d_basename = test.p3d
p3d trusted
_matches_run_origin = 1
_matches_script_origin = 0
_auto_install = 1, _auto_start = 1, _stop_on_ready = 0
All 54 extracts of panda3d seem good.
report_package_info_ready: panda3d
Beginning install of 0 packages, total 0 bytes required.
make_temp_filename: /tmp/p3d_fb9b5a.xml
set_wparams: 1 640 480
image_filename = /home/croxis/.panda3d/hosts/runtime.panda3d.org_96a0ec538744def6/images/download.png
notify: onpluginload 
notify: onauth 
notify: ondownloadbegin 
notify: ondownloadcomplete 
Downloading http://croxis.dyndns.org/panda3d/contents.xml?1267223000: 0M, 0x20c0bf0
Downloaded http://croxis.dyndns.org/panda3d/contents.xml?1267223000: 100%, 0x20c0bf0, success = 1
release_temp_filename: /tmp/p3d_fb9b5a.xml
All 0 extracts of yaml seem good.
report_package_info_ready: yaml
All 30 extracts of morepy seem good.
report_package_info_ready: morepy

Ah, I bet we’re not running your newly-modified code. The panda3d executable by default will run whatever p3d_plugin.so it downloads from the server, not the one you built locally.

Are you using ppremake? You could define:

#define P3D_PLUGIN_P3D_PLUGIN /my/direct/built/lib/p3d_plugin.so

in your Sources.pp file to build a panda3d executable that links with your specific build p3d_plugin.so instead. Otherwise, you can just run ppackage coreapi.pdef to package up your p3d_plugin.so into a new downloadable, and host that downloadable on your own server (or on your local disk), and build your panda3d executable to reference your own server’s url (or your local file:// url).

David

I’m stumped over what is probably an easy step. I’ve built both the runtime and the sdk with makepanda and there is no p3d_plugin.so in either build directory. What am I missing?

In the case of makepanda, p3d_plugin.so is part of the rtdist build. Compile Panda3D with the --rtdist flag, and when it’s finished, you’ll notice a package structure in the “built_distributor/stage” directory.

When running your p3d, you can point to this stage directory with the -M (super-mirror) flag to the panda3d command-line.

And when one problem is fixed another crops up. I got the rdist build on my local server. Replaced all instances of 10.0.0.3/stage/
And it STILL is trying to do this

Mismatched download for runtime.panda3d.org/coreapi/linu … _plugin.so
Mismatched download for runtime.panda3d.org/coreapi/linu … 1267564869
Couldn’t download core API.
Unable to load Panda3D plugin.

I assume test.p3d (and packp3d) have some internal reference to runtime.panda3d.org?

Edit: Yup it does! I changed the references in test.p3d to 10.0.0.3/stage/ and deleted my .panda3d. Same problem

Order is important. Instead of:

 panda3d-1.7.0/built_croxis_rt/bin/panda3d test.p3d -M http://10.0.0.3/stage/

run:

 panda3d-1.7.0/built_croxis_rt/bin/panda3d -M http://10.0.0.3/stage/ test.p3d

David

Ok. I think I got it.


_root_dir = /home/croxis/.panda3d, _temp_directory = /tmp/, platform = linux_amd64, host_url = http://10.0.0.3/stage/, verify_contents = 0
Plugin version: 1.0.0c
Plugin distributor: cmu_1.7
Core API host URL: http://10.0.0.3/stage/
Core API date: Wed Mar  3 08:26:51 2010

super_mirror = http://10.0.0.3/stage/
Creating P3DInstance 0x23a99c0:  console_output="1" auto_start="1"
setting background to download, splash_window = 0
All 8 extracts of images seem good.
report_package_info_ready: images
Done installing images: success = 1
p3d_basename = test.p3d
p3d trusted
_matches_run_origin = 1
_matches_script_origin = 0
_auto_install = 1, _auto_start = 1, _stop_on_ready = 0
All 54 extracts of panda3d seem good.
report_package_info_ready: panda3d
Beginning install of 0 packages, total 0 bytes required.
make_temp_filename: /tmp/p3d_5ca87f.xml
set_wparams: 1 640 480
image_filename = /home/croxis/.panda3d/hosts/10.0.0.3_d7ee2fde0e80ce35/images/download.png
start subprocess_run
notify: onpluginload 
notify: onauth 
notify: ondownloadbegin 
notify: ondownloadcomplete 
done make_window
done setup_gc, context = 0x23d1d50
Downloading http://10.0.0.3/stage/contents.xml: 0M, 0x23d2500
Downloaded http://10.0.0.3/stage/contents.xml: 100%, 0x23d2500, success = 1
release_temp_filename: /tmp/p3d_5ca87f.xml
Couldn't find package yaml in contents file.
make_temp_filename: /tmp/p3d_884d04.xml
Downloading http://croxis.dyndns.org/panda3d/contents.xml?1267635622: 0M, 0x243c830
Downloaded http://croxis.dyndns.org/panda3d/contents.xml?1267635622: 100%, 0x243c830, success = 1
release_temp_filename: /tmp/p3d_884d04.xml
Redownloading contents.xml made a difference.
All 0 extracts of yaml seem good.
report_package_info_ready: yaml
All 44 extracts of morepy seem good.
report_package_info_ready: morepy
All 54 extracts of panda3d seem good.
report_package_info_ready: panda3d

No window