packaging c++ app for browser plugin

C++ coding using Panda3D
1

I’m trying to package a c++ app into a p3d and I’m getting an access violation. I didn’t find any documentation on how to do this apart from the fact that you have to make a python wrapper, so I’m sure I’m doing something wrong.

Anyway, this is what I’m doing.

  1. I changed my exe project in visual studio to dll by changing the setting, nothing else.

  2. I changed my entry point from:

void main(int argc, char *argv[]) {

to

#define DLLEXPORT extern "C" __declspec(dllexport)
DLLEXPORT void main() {

At this point I get a “game.dll” file.

  1. I wrote the following script, main.py:
from ctypes import *

#the print is for debugging, it shows a handle correctly
print windll.game
windll.game.main()

And I get an access violation when running it on the main() call.

So, before I waste a day figuring out how to debug the native dll from python, am I missing something trivial?

EDIT: Just for the record, I also tried like this, same result:

from ctypes import cdll
mydll = cdll.LoadLibrary('game.dll')
mydll.main()
2

Hmm, I don’t have any experience using ctypes to call a C++ function directly. Can anyone else answer to this?

David

3

It doesn’t have to be ctypes actually, it was just suggested to me by rdb. I’m clueless about the available options. How would you do it? Am I supposed to use interrogate?

Also, going back to ctypes, now I’m pretty sure I’m doing it well because I tested with a simple program and this method works perfectly. But when I bring my classes into the project I get an access violation.

4

My approach would have been to write a trivial Python-callable wrapper, something like this:

#include <Python.h>

static PyObject *
wrapper_main() {
  main();

  Py_RETURN_NONE;
}

static PyMethodDef mywrapper_methods[] = {
  {"main", (PyCFunction)wrapper_main, METH_NOARGS },
  {NULL}  /* Sentinel */
};

#ifndef PyMODINIT_FUNC	/* declarations for DLL import/export */
#define PyMODINIT_FUNC void
#endif
PyMODINIT_FUNC
initmywrapper(void) {
  PyObject *m;

  m = Py_InitModule3("mywrapper", mywrapper_methods, NULL);
}

Using API’s that are described more thoroughly at python.org. With this approach, you could import the dll as a module and call mywrapper.main() directly.

Still, that’s just a detail. It seems that the ctypes approach ought to work too, and that’s probably simpler. Perhaps there is some issue with compilation flags; it may be important to ensure that your app is compiled precisely the way that Python (and Panda) was compiled. But since you are already linking with Panda, it seems that it’s already so.

So, I don’t have any great ideas, sorry.

David

5

Ok, I’ll try that just in case it helps. And (AFAIK) it could be that I have a different configuration, since I’m using panda dynamically currently. I’ll check that too, thanks for the pointers.

And I guess there’s no easy way to remove the need for a python entry point from the p3d platform, right?

6

Right, not currently. In the next release we may have an easier solution.

David

7

Well, a hello world panda application works fine with the method explained above, so ctypes is OK.

But my full panda application doesn’t. I removed bullet so I have only panda libraries (from panda 1.7.0 official release) and libraries from the dependency package (like tinyxml, freetype and openssl). So one of those is probably the problem.

8

Sorry to double post but I got a hint. I’m gonna post it in case it gives somebody any idea.

#include "pandaFramework.h"
#include "pandaSystem.h"

PandaFramework framework;
 
#define DLLEXPORT extern "C" __declspec(dllexport)

DLLEXPORT int entry(){
  int fargc = 0;
  char**fargv = 0;

  framework.open_framework(fargc, fargv);
  framework.set_window_title("a window");
  WindowFramework *window = framework.open_window();

  //NodePath environ = window->load_model(framework.get_models(), "nonexistent");

  framework.main_loop();
  framework.close_framework();
  return (0);
}

This c++ code works, but if I uncomment that load_model() line, I get a crash. This crash doesn’t happen when running the same code as an exe. Note that the model doesn’t exist in both cases.

I’m in multithreaded-dll, release, and using only the following libs, from the 1.7.0 official binaries:

EDIT: And this is all the log, set to spam level:
pastebin.com/UyF9aSFg

9

Hmm, that function is the first time in the above code that you pass a string to a non-inline method, which is often the first time you detect a problem with differing build flags.

See if any other non-inline method with a string parameter can be called safely.

David

10

No, it’s not that, other calls like that work fine. Anyway, by using a panda built by me I can run this sample, but my full app still crashes.

In any case, I see that makepanda has a suspicious assortment of /NODEFAULTLIBRARY flags, for example libci is ignored but libc isn’t. Maybe it’s something like that. I’ll try rebuilding my app, panda, and the external dependencies while explicitly excluding everything except msvcrt.lib, I’ll let you know if it works.

EDIT: Btw, it now crashes when creating an ambient light, even though creating a directional light works… But nvm, as I think I’ll fix it by tinkering with the flags.

11

Still no luck, maybe I’m missing some configuration flag but I give up with this plugin system, I’m starting to think that maybe a full panda app is unable to run as a dll. Has this ever been done, per chance?

I’ll look into making my app into activex/npapi clients. That part is easy, but dealing with the resources and the installation folders is a pain :frowning:

If you can think of any quick hack to use the existing work into browser integration it would be appreciated, but I doubt such a thing is possible.

EDIT: Oh, maybe building panda statically helps.

12

Hmm, I don’t know why it shouldn’t just be easy. Let me try it myself to see if I have any luck.

David

13

You probably missed it cause I edited the message before you replied, so just in case:

Has a full panda app ever been run in a dll?

And thanks a lot for trying.

EDIT: Ah, and remember that my snippet above actually worked after I rebuilt panda. Before you try I think I think I should get you a new reproducible crash. I’ll get to it.

14

Actually nevermind, if you get to reproduce that crash with Panda 1.7.0 official window binaries, I guess we would find the root of the problem automatically.

15

For the record, I get the same access violations at random points with the PyObject method you described. I get a Panda window and lately it’s crashing when attaching an ambient light to render. This is the code of that part of the app, it’s curious because it doesn’t crash when I add the directional light:

    PT(DirectionalLight) d_light;
    d_light = new DirectionalLight("my d_light");
    d_light->set_color(LVecBase4f(1, 1, 1, 1));
    d_light->set_shadow_caster(true);
    NodePath dlnp = window->get_render().attach_new_node(d_light);
    dlnp.set_hpr (45, -45, 0);
    window->get_render().set_light(dlnp);


    PT(AmbientLight) alight = new AmbientLight("alight");
    alight->set_color(LVecBase4f(0.2, 0.2, 0.2, 1));
    NodePath alnp = window->get_render().attach_new_node(alight); //crashes here
    window->get_render().set_light(alnp);

EDIT: Ah, and at random times I get this when running the python script: “ImportError: DLL load failed: Recursion too deep; the stack overflowed.”

EDIT: Tested the PyObject method with a custom built panda and I’m getting this assertion now:

Assertion failed: size <= _buffer_size, file c:\devel\panda\stable\panda3d\dtool
\src\dtoolbase\deletedBufferChain.cxx, line 51

Also since I now have symbols I can debug the crash, it’s leaving me at:

  set_cg_device(NULL);

In line 5710 of dxGraphicsStateGuardian9.cxx, which is an old, unresolved bug. However, that’s at an atexit function so I don’t think it’s important. Anyway, call stack:

 	cgD3D9.dll!6a603ca1() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for cgD3D9.dll]	
 	cgD3D9.dll!6a602328() 	
>	libpandadx9.dll!DXGraphicsStateGuardian9::atexit_function()  Line 5710 + 0x11 bytes	C++
 	libpandadx9.dll!_CRT_INIT(void * hDllHandle=0x00000000, unsigned long dwReason=3290845538, void * lpreserved=0xc6ae5162)  Line 449	C
 	00000001()
16

Quadraposting. I finally got a meaningful call stack and break:

mutexSimpleImpl.cxx

  while ((_flags & F_lock_count) != 0) {
    manager->enqueue_block(thread, this); //violation here
    manager->next_context();
  }
>	libpanda.dll!MutexSimpleImpl::do_acquire()  Line 36 + 0x2e bytes	C++
 	libpanda.dll!CopyOnWritePointer::get_write_pointer()  Line 81 + 0x26 bytes	C++
 	libpanda.dll!CopyOnWritePointerTo<CopyOnWriteObj<pvector<GeomNode::GeomEntry> > >::get_write_pointer()  Line 311 + 0x12 bytes	C++
 	libpanda.dll!GeomNode::CData::modify_geoms()  Line 269 + 0xd bytes	C++
 	libpanda.dll!GeomNode::add_geom(Geom * geom=0x005d2988, const RenderState * state=0x0027f9a4)  Line 607 + 0xfd bytes	C++
 	gamedll.dll!Mesh::construct()  + 0x917 bytes	C++
 	gamedll.dll!Figure::getNodePath()  + 0x79 bytes	C++
 	gamedll.dll!entry()  + 0x6f3 bytes	C++

EDIT:

If I remove that geometry creation code I get another crash on the ambient light creation:

>	libpanda.dll!MutexSimpleImpl::do_acquire()  Line 36 + 0x2e bytes	C++
 	libpanda.dll!CopyOnWritePointer::get_read_pointer()  Line 39 + 0x28 bytes	C++
 	libpanda.dll!CopyOnWritePointerTo<CopyOnWriteObj1<ov_set<PandaNode::UpConnection,std::less<PandaNode::UpConnection> >,TypeHandle> >::get_read_pointer()  Line 285 + 0x12 bytes	C++
 	libpanda.dll!PandaNode::CData::get_up()  Line 1080 + 0x10 bytes	C++
 	libpanda.dll!PandaNode::get_component(NodePathComponent * parent=0x0315d2b4, PandaNode * child_node=0x031808f8, int pipeline_stage=0, Thread * current_thread=0x031413c4)  Line 3566 + 0xd bytes	C++
 	libpanda.dll!PandaNode::attach(NodePathComponent * parent=0x0315d2b4, PandaNode * child_node=0x031808f8, int sort=0, int pipeline_stage=0, Thread * current_thread=0x031413c4)  Line 3323 + 0x1a bytes	C++
 	libpanda.dll!NodePath::attach_new_node(PandaNode * node=0x031808f8, int sort=0, Thread * current_thread=0x031413c4)  Line 885 + 0x19 bytes	C++
 	gamedll.dll!entry()  + 0x9d7 bytes	C++
 	gamedll.dll!AmbientLight::write()  + 0x15 bytes	C++
 	python26.dll!1e015538() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for python26.dll]	
 	python26.dll!1e015d17() 	
 	python26.dll!1e014f07() 	
 	python26.dll!1e028861() 	
 	python26.dll!1e026b6a() 	
 	python26.dll!1e028954() 	
 	python26.dll!1e031694() 	
 	python26.dll!1e031622() 	
 	python26.dll!1e03368f() 	
 	python26.dll!1e03364e() 	
 	msvcr90.dll!71392201() 	
 	python.exe!1d001160() 	
 	kernel32.dll!76083677() 	
 	ntdll.dll!77529d72() 	
 	ntdll.dll!77529d45()

Will try to figure it out with this now, just posting in case it’s easy to see the problem for anybody.

17

It sure does seem like something is completely fubar. I don’t think the problem is likely to be geometry creation or ambient lights per se; there’s something at a lower level which is out of whack.

David

18

Right, but notice that in both cases the segfault occurs here:

@mutexSimpleImpl.cxx 

  while ((_flags & F_lock_count) != 0) { 
    manager->enqueue_block(thread, this); //violation here 
    manager->next_context(); 
  }
19

Ah, hmm. Good point. I don’t know why you would be blocking, either, since you presumably haven’t created any threads yet at startup.

Note that panda “threads” are not true OS-level threads, and exist only within Panda’s own address space. Panda implements context switching by munging the CPU registers. It’s just a hack to allow multithreaded algorithms like asynchronous loading, without imposing the overhead required by true OS-level threading.

Since you’re getting a crash at this point, perhaps something went wrong with the initialization of the threading subsystem. Maybe this is a static init ordering issue, and the threading subsystem in fact hasn’t been initialized yet? I’m still grasping at straws.

David

20

Oh, now that you mention it I remember that from the threading video lecture. That gives me something to try as a workaround (trying with real multi-threading mode), though supposedly I shouldn’t be using it, and I certainly don’t want to risk bringing instability when I don’t need any of the benefits of multi-threading. Is it really that bad?