Browser build in any plans?

[marrella]

I have seen several old threads about a build for browsers, but not recent, and not encouraging. Is it fair to conclude that work making a browser build is not moving ahead, or not soon?

By a browser build I mean help in packaging a panda3d game for running from a server in a browser.

[rdb]

The old browser plug-in is no longer supported by browsers. New efforts are focused on the WebGL port:

Panda3D WebGL Port!

Someone asked me about this possibility yesterday. So I tried it while I had some time left waiting for the 1.9 builds to finish. It surprisingly only took me a few hours to compile Panda3D to JavaScript using emscripten and create a WebGL back-end (based on our existing GLES2 back-end). So, here’s pview for you to try out: rdb.name/panda3d-webgl/pview.html (triangle) rdb.name/panda3d-webgl/pview-panda.html (animated panda) rdb.name/panda3d-webgl/simple_fps.html (FPS demo, with fancy cn04+…

This effort is alive on the webgl-port branch, and it does work, though it’s not currently a high priority. It currently lacks an easy way for people to build their games for this target. It requires a lot of manual compilation work at the moment.

[marrella]

Thank you, I did see that post, but did now know how to get from here to there.

‘It does work’ is great news, and I would like to try. I am happy to do ‘a lot of manual compilation’, not afraid of that. But my experience with all these tools (eg, emscripten, even WebGL) is pretty light. Just a matter of time, but lots (I am old). Is there a ‘recipe’ I can follow? I don’t mean do it for me or steps for beginners, I welcome doing it myself, just a list of the major steps that I could figure out the details (dependencies, installations, configuration) for myself?

An ignorant side question: is Pyiodide a future option, or overkill, or wrong-parts?

[rdb]

There’s no recipe other than what you can put together from the parts in that thread. You’d need to:

• Get the emscripten SDK
• Compile the Python interpreter using emscripten
• Check out the webgl-port branch from GitHub
• Build an application using a modified version of pfreeze, I think there is a zip file somewhere in that thread

Let me know if you need any help. The last time I tried building the WebGL port with Python was still with Python 2, but we no longer support that.

pyodide looks interesting, and it would be worth investigating whether it makes sense for Panda to integrate with that.

[marrella]

Thank you! I will get started probably next week and let you know. Thanks for your offer of help, I am sure I will be back asking.

(I remember the python 2 comment, that is why I didn’t pursue it deeper. I hadn’t realized all the steps were there.)

[rdb]

It looks like pyodide is still using an old version of the emscripten toolchain, it may be better to wait until they update to the newer pipeline before trying to integrate with them.

[marrella]

I finally got back to trying to do this. Seemed to all work (lots of little fixes, but I don’t think important), but then the final step gave me blanks.

1. Built Python with Emscripten. Resulted in python.bc and libpython3.4.a (8.8MB)
2. Built app with pfreeze. Resulted in Museum.c (53.8MB)
3. emcc libpython3.4.a Museum.c -o museum.html
   Result: museum.html, museum.js (117kb), museum.wasm (9kb)

Obviously from the size, something went wrong. (Loading on the server just makes a blank small black window.)

Variations of step 3 made no difference:
1. emcc Museum.c libpython3.4.a -o museum.html (though the emscripten notes says order matters)
2. emcc Museum.o libpython3.4.a -o museum.html (after emcc of the c file by itself)
3. emcc Museum.c python.bc -o museum.html – error does not recognize file type python.bc, though I noted that you used .bc files in your build

Still a lot I don’t know about emscripten, but obviously something basic i am doing wrong.

(identical results in linux and OSX machines after fixing paths)

==================
commands used to build:
config emscripten for python (makes Makefile for Python) –
BASECFLAGS=-m32 LDFLAGS=-m32 emconfigure ./configure --without-threads --without-pymalloc --disable-shared --disable-ipv6 --disable-toolbox-glue

then make, of course

freeze museum –
python pfreeze.py -s -o Museum.c -l -i Maze.py

[rdb]

I do recommend using Python 3.8 instead of 3.4, since Python 3.8 fixes some undefined behaviour in the Python interpreter that doesn’t work in WebGL.

I will share the wrapper script I used around pfreeze back in the day which includes the compiler command-line that I used: freezify.py (6.9 KB)
It will need some fixes since it was originally created for my (patched) version of Python 2.7.

[marrella]

I forgot to mention that I also did 3.9, but there were extra edits to get it to work.

But I used 3.9 for freezify in all cases, that is what is installed on my systems.

Thanks for the freezify. I think the problem is in the freeze, though I thought I copied what you did (you put your script in that other thread). I will try again more carefully, I guess. I am still puzzled by the .bc file not working on emcc.

[marrella]

A big difference in freezify.py is that you have .bc files for all the individual python modules as well as for all your panda3d files. Did you emscripten compile each python module and each panda3d module separately? That would certainly make a difference. I don’t understand from the Emscripten documentation why that would be necessary, but I guess that is what I missed. Or are these other .bc files only needed because you are making an exe file?

If so, I have a lot more work to do.

The other thing I don’t understand is that this example seems to create an exe, which doesn’t help. Does it also create a c file on the way (the inline text header seems to be the start)?

I’ll keep working with it, I understand maybe 85% of what is in the file.

Is it possible that the version of emscripten that you used in 2015 used .bc files and emscripten now uses .o files instead? I have all the .o files.

[rdb]

Yes, that’s possible, it may be that the standard is now to use .o files instead of .bc. Alternatively, .a files (which are really just archives containing .o files) may be used.

I’m using the ability of FreezeTool to produce an executable. Or rather, FreezeTool thinks it’s producing an executable, when it’s actually telling emscripten to produce a .js file that can be embedded in the browser. Producing the .c file is an intermediary step in that process, but it should get you a .js as output.

[marrella]

Thanks. It seems to be going ok after all the edits. I changed the .bc to .o before I wrote that and it seemed ok. Just tracking down odd errors in the Python source like these:

“LONG_BIT definition appears wrong for platform (bad gcc/glibc config?).”
#error “LONG_BIT definition appears wrong for platform (bad gcc/glibc config?).”
(I ‘fixed’ this by just commenting out the test)

incompatible pointer types passing ‘char [12]’ to parameter of type ‘const wchar_t *’ (aka ‘const int *’) [-Wincompatible-pointer-types]
Py_SetProgramName(“emscripthon”);
^~~~~~~~~~~~~
/Users/davidmarques/prog/Python-3.9.1/Include/pylifecycle.h:38:51: note: passing argument to parameter here
PyAPI_FUNC(void) Py_SetProgramName(const wchar_t *);

But that is just a warning.

Also have these: (pandas again)
WARNING: Python cannot import extension modules under frozen Python packages; pandas._libs.window.indexers will be inaccessible. passing either -l to link in extension modules or use -x pandas to exclude the entire package.

When I did the command manually, I just put in -l

It looks like the script thinks I am on a Windows system, have to track that down.

[rdb]

The LONG_BIT definition can maybe just be adjusted manually.

freezify.py contains embedded code to invoke the Python interpreter that is written using the Python 2 API. The Python 3 API has some changes, the one in question being that Py_SetProgramName now accepts a wchar_t instead of char. However, you can just comment out the Py_SetProgramName call, since it’s not consequential.

If at any point you feel you are stuck, let me know and I’ll see if I can find some time to fix this up myself.

I suggest you try building something without pandas for now, just a simple sample program, to make sure you can get that working before moving on to a more complex program. I think that you would also need to build pandas from source in order to get it to work in the browser.

[marrella]

I am building simpler now, skipping pandas. But it is just warning, and pandas is not invoked until after the start.

I did comment out the Py_SetProgramName yesterday, saw that it did not make a difference. Just noted it.

[cmake a red herring, fixed it]

All builds, creates editor.js and editor.wasm (7.3MB) which is more like it. I am sure it won’t run, but at least I trust the build a bit more.

Now I have to go back and build all the .o files I need but skipped for now (all the undefined symbols for the python modules you listed, and a few others).

BTW, if you ever get back to this (I hope you can), the cmake throws a warning:
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a … suffix to tell
CMake that the project does not need compatibility with older versions.

Ok for now, but you would have to fix that in your code eventually.

[marrella]

I think I am stuck now. I don’t know how to get core.o and direct.o for panda3d. I suppose I have to build all of panda3d from source.

using:

python makepanda/makepanda.py --everything --installer

In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/chrono:827:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:64:9: error:
no member named ‘clock’ in the global namespace
using ::clock;
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:65:9: error:
no member named ‘difftime’ in the global namespace
using ::difftime;
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:66:9: error:
no member named ‘mktime’ in the global namespace; did you mean ‘mktemp’?
using ::mktime;
~~^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:675:7: note:
‘mktemp’ declared here
char *mktemp(char *);

And many more.

I am building on OSX instead of on native linux, but I don’t think that should make this difference. I can try linux tomorrow.

[marrella]

The python libraries are not as straightforward as I thought either. I get these undefined globals:

__posix_spawnx
init_collections
init_hashlib
init_io
init_libOpenALAudio
init_random
initbinascii
initcStringIO
initcore
initdirect
initmath
inittime
initunicodedata
task_manager_poll

I include all the modules built by emscripten:

“posixmodule”, “errnomodule”, “pwdmodule”, “_sre”, “_codecsmodule”, “_weakref”, “_functoolsmodule”, “_operator”, “_collectionsmodule”, “_abc”, “itertoolsmodule”, “atexitmodule”, “signalmodule”, “_stat”, “timemodule”, “_threadmodule”, “_localemodule”, “_iomodule”, “iobase”, “fileio”, “bytesio”, “bufferedio”, “textio”, “stringio”, “faulthandler”, “_tracemalloc”, “_peg_parser”, “symtablemodule”, “xxsubtype”

And that makes no difference. I’ll try variations of the emscripten build, but shooting in the dark.

[marrella]

Not really any progress. I rebuilt Python using 3.8.6, explicitly including every module built by emscripten in the emcc command in pyfreeze. Still 14 undefined symbols (I think 2 or 3 are panda3d – initcore and initdirect) and the program when it runs gives:

Fatal Python error: init_importlib: importlib install failed

Next I will try the simplest python program without panda3d.

[marrella]

Same result with a python module with a single statement in it, print("time is: "+str(20449357))

Same list of undefined globals, same error (importlib install failed) in the javascript console.

Obviously I am on the wrong track.

[rdb]

core.o/bc will be in the built/panda3d folder of the panda3d build. If you built Panda3D with the --static flag then they will be in the form of libpy.panda3d.core.a or something like that.

It’s a good idea to start with just getting the basic Python modules to work. Did you not build those modules when you built Python from source? What built objects do you have in your Python source directory?

The init_importlib error is something we’ll need to debug further. Try setting the Py_VerboseFlag at the beginning of your C code. I think it suggests that the Python interpreter failed to import the _frozen_importlib module, but that should have been frozen as part of the FreezeTool process.

[marrella]

That is what I am trying to do, get a simple python program to work without panda3. I added every .o module that was in the Python-3.8.6/Modules directory to the emcc command.

“_abc”, “_codecsmodule”, “_collectionsmodule”, “_functoolsmodule”, “_iomodule”, “_localemodule”, “_operator”, “_sre”, “_stat”, “_threadmodule”, “_tracemalloc”, “_weakref”, “atexitmodule”, “bufferedio”, “bytesio”, “config”, “errnomodule”, “faulthandler”, “fileio”, “gcmodule”, “getbuildinfo”, “getpath”, “hashtable”, “iobase”, “itertoolsmodule”, “main”, “posixmodule”, “pwdmodule”, “signalmodule”, “stringio”, “symtablemodule”, “textio”, “timemodule”, “xxsubtype”

The object files in Python-3.8.6/Python give duplicate definition errors, I assume because they are bundled into libpython3.8.a

There are no .o or .a files corresponding closely to the undefined globals.

I don’t have a built folder for panda3d because it errored out when I tried pytnon makepanda (see above). I am sure there are config issues, but I have not gone deeper for now, just trying to get a simple python program to build and run with emscripten. A long way to go.