Browser build in any plans?

It looks like pyodide is still using an old version of the emscripten toolchain, it may be better to wait until they update to the newer pipeline before trying to integrate with them.

I finally got back to trying to do this. Seemed to all work (lots of little fixes, but I don’t think important), but then the final step gave me blanks.

  1. Built Python with Emscripten. Resulted in python.bc and libpython3.4.a (8.8MB)
  2. Built app with pfreeze. Resulted in Museum.c (53.8MB)
  3. emcc libpython3.4.a Museum.c -o museum.html
    Result: museum.html, museum.js (117kb), museum.wasm (9kb)

Obviously from the size, something went wrong. (Loading on the server just makes a blank small black window.)

Variations of step 3 made no difference:
1. emcc Museum.c libpython3.4.a -o museum.html (though the emscripten notes says order matters)
2. emcc Museum.o libpython3.4.a -o museum.html (after emcc of the c file by itself)
3. emcc Museum.c python.bc -o museum.html – error does not recognize file type python.bc, though I noted that you used .bc files in your build

Still a lot I don’t know about emscripten, but obviously something basic i am doing wrong.

(identical results in linux and OSX machines after fixing paths)

==================
commands used to build:
config emscripten for python (makes Makefile for Python) –
BASECFLAGS=-m32 LDFLAGS=-m32 emconfigure ./configure --without-threads --without-pymalloc --disable-shared --disable-ipv6 --disable-toolbox-glue

then make, of course

freeze museum –
python pfreeze.py -s -o Museum.c -l -i Maze.py

I do recommend using Python 3.8 instead of 3.4, since Python 3.8 fixes some undefined behaviour in the Python interpreter that doesn’t work in WebGL.

I will share the wrapper script I used around pfreeze back in the day which includes the compiler command-line that I used: freezify.py (6.9 KB)
It will need some fixes since it was originally created for my (patched) version of Python 2.7.

I forgot to mention that I also did 3.9, but there were extra edits to get it to work.

But I used 3.9 for freezify in all cases, that is what is installed on my systems.

Thanks for the freezify. I think the problem is in the freeze, though I thought I copied what you did (you put your script in that other thread). I will try again more carefully, I guess. I am still puzzled by the .bc file not working on emcc.

A big difference in freezify.py is that you have .bc files for all the individual python modules as well as for all your panda3d files. Did you emscripten compile each python module and each panda3d module separately? That would certainly make a difference. I don’t understand from the Emscripten documentation why that would be necessary, but I guess that is what I missed. Or are these other .bc files only needed because you are making an exe file?

If so, I have a lot more work to do.

The other thing I don’t understand is that this example seems to create an exe, which doesn’t help. Does it also create a c file on the way (the inline text header seems to be the start)?

I’ll keep working with it, I understand maybe 85% of what is in the file.

Is it possible that the version of emscripten that you used in 2015 used .bc files and emscripten now uses .o files instead? I have all the .o files.

Yes, that’s possible, it may be that the standard is now to use .o files instead of .bc. Alternatively, .a files (which are really just archives containing .o files) may be used.

I’m using the ability of FreezeTool to produce an executable. Or rather, FreezeTool thinks it’s producing an executable, when it’s actually telling emscripten to produce a .js file that can be embedded in the browser. Producing the .c file is an intermediary step in that process, but it should get you a .js as output.

Thanks. It seems to be going ok after all the edits. I changed the .bc to .o before I wrote that and it seemed ok. Just tracking down odd errors in the Python source like these:

“LONG_BIT definition appears wrong for platform (bad gcc/glibc config?).”
#error “LONG_BIT definition appears wrong for platform (bad gcc/glibc config?).”
(I ‘fixed’ this by just commenting out the test)

incompatible pointer types passing ‘char [12]’ to parameter of type ‘const wchar_t *’ (aka ‘const int *’) [-Wincompatible-pointer-types]
Py_SetProgramName(“emscripthon”);
^~~~~~~~~~~~~
/Users/davidmarques/prog/Python-3.9.1/Include/pylifecycle.h:38:51: note: passing argument to parameter here
PyAPI_FUNC(void) Py_SetProgramName(const wchar_t *);

But that is just a warning.

Also have these: (pandas again)
WARNING: Python cannot import extension modules under frozen Python packages; pandas._libs.window.indexers will be inaccessible. passing either -l to link in extension modules or use -x pandas to exclude the entire package.

When I did the command manually, I just put in -l

It looks like the script thinks I am on a Windows system, have to track that down.

The LONG_BIT definition can maybe just be adjusted manually.

freezify.py contains embedded code to invoke the Python interpreter that is written using the Python 2 API. The Python 3 API has some changes, the one in question being that Py_SetProgramName now accepts a wchar_t instead of char. However, you can just comment out the Py_SetProgramName call, since it’s not consequential.

If at any point you feel you are stuck, let me know and I’ll see if I can find some time to fix this up myself.

I suggest you try building something without pandas for now, just a simple sample program, to make sure you can get that working before moving on to a more complex program. I think that you would also need to build pandas from source in order to get it to work in the browser.

I am building simpler now, skipping pandas. But it is just warning, and pandas is not invoked until after the start.

I did comment out the Py_SetProgramName yesterday, saw that it did not make a difference. Just noted it.

[cmake a red herring, fixed it]

All builds, creates editor.js and editor.wasm (7.3MB) which is more like it. I am sure it won’t run, but at least I trust the build a bit more.

Now I have to go back and build all the .o files I need but skipped for now (all the undefined symbols for the python modules you listed, and a few others).

BTW, if you ever get back to this (I hope you can), the cmake throws a warning:
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a … suffix to tell
CMake that the project does not need compatibility with older versions.

Ok for now, but you would have to fix that in your code eventually.

I think I am stuck now. I don’t know how to get core.o and direct.o for panda3d. I suppose I have to build all of panda3d from source.

using:

python makepanda/makepanda.py --everything --installer

In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/chrono:827:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:64:9: error:
no member named ‘clock’ in the global namespace
using ::clock;
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:65:9: error:
no member named ‘difftime’ in the global namespace
using ::difftime;
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/…/include/c++/v1/ctime:66:9: error:
no member named ‘mktime’ in the global namespace; did you mean ‘mktemp’?
using ::mktime;
~~^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:675:7: note:
‘mktemp’ declared here
char *mktemp(char *);

And many more.

I am building on OSX instead of on native linux, but I don’t think that should make this difference. I can try linux tomorrow.

The python libraries are not as straightforward as I thought either. I get these undefined globals:

__posix_spawnx
init_collections
init_hashlib
init_io
init_libOpenALAudio
init_random
initbinascii
initcStringIO
initcore
initdirect
initmath
inittime
initunicodedata
task_manager_poll

I include all the modules built by emscripten:

“posixmodule”, “errnomodule”, “pwdmodule”, “_sre”, “_codecsmodule”, “_weakref”, “_functoolsmodule”, “_operator”, “_collectionsmodule”, “_abc”, “itertoolsmodule”, “atexitmodule”, “signalmodule”, “_stat”, “timemodule”, “_threadmodule”, “_localemodule”, “_iomodule”, “iobase”, “fileio”, “bytesio”, “bufferedio”, “textio”, “stringio”, “faulthandler”, “_tracemalloc”, “_peg_parser”, “symtablemodule”, “xxsubtype”

And that makes no difference. I’ll try variations of the emscripten build, but shooting in the dark.

Not really any progress. I rebuilt Python using 3.8.6, explicitly including every module built by emscripten in the emcc command in pyfreeze. Still 14 undefined symbols (I think 2 or 3 are panda3d – initcore and initdirect) and the program when it runs gives:

Fatal Python error: init_importlib: importlib install failed

Next I will try the simplest python program without panda3d.

Same result with a python module with a single statement in it, print("time is: "+str(20449357))

Same list of undefined globals, same error (importlib install failed) in the javascript console.

Obviously I am on the wrong track.

core.o/bc will be in the built/panda3d folder of the panda3d build. If you built Panda3D with the --static flag then they will be in the form of libpy.panda3d.core.a or something like that.

It’s a good idea to start with just getting the basic Python modules to work. Did you not build those modules when you built Python from source? What built objects do you have in your Python source directory?

The init_importlib error is something we’ll need to debug further. Try setting the Py_VerboseFlag at the beginning of your C code. I think it suggests that the Python interpreter failed to import the _frozen_importlib module, but that should have been frozen as part of the FreezeTool process.

That is what I am trying to do, get a simple python program to work without panda3. I added every .o module that was in the Python-3.8.6/Modules directory to the emcc command.

“_abc”, “_codecsmodule”, “_collectionsmodule”, “_functoolsmodule”, “_iomodule”, “_localemodule”, “_operator”, “_sre”, “_stat”, “_threadmodule”, “_tracemalloc”, “_weakref”, “atexitmodule”, “bufferedio”, “bytesio”, “config”, “errnomodule”, “faulthandler”, “fileio”, “gcmodule”, “getbuildinfo”, “getpath”, “hashtable”, “iobase”, “itertoolsmodule”, “main”, “posixmodule”, “pwdmodule”, “signalmodule”, “stringio”, “symtablemodule”, “textio”, “timemodule”, “xxsubtype”

The object files in Python-3.8.6/Python give duplicate definition errors, I assume because they are bundled into libpython3.8.a

There are no .o or .a files corresponding closely to the undefined globals.

I don’t have a built folder for panda3d because it errored out when I tried pytnon makepanda (see above). I am sure there are config issues, but I have not gone deeper for now, just trying to get a simple python program to build and run with emscripten. A long way to go.

Oh, but note that the C code in freezify will need further changes, because the module initialization syntax in Python changed. For example, what was inithashlib is now PyInit_hashlib. Some other modules may be removed or unnecessary, it may be good to comment out some of those lines and add them back in as you find they are needed.

If all the extension modules are linked in libpython3.8.a, then you should not need to pick up additional .o files.

I have been working on exactly that, assuming that your C code was out of date. In the process of trying to figure out all the new errors, not a simple 1:1, but will work through it.

Well, the simple answer is that commenting them all out reduces the missing globals to just 2:

__posix_spawnx, task_manager_poll

Result of running the html file is the same.

But I noted that now (after a rebuild of Python with no --without configure flags) I have two missing globals at the top, _winapi and pep517.

I’ll try going back to a different build of Python, but I don’t think it will make a difference. But it would not surprise me if the emscripten build of Python is a problem. I am using:

BASECFLAGS=-m32 LDFLAGS=-m32 emconfigure ./configure --without-threads --without-pymalloc --disable-shared --without-signal-module --disable-toolbox-glue --disable-ipv6

as recommended in the emscripten pages for OSX.

I don’t think __posix_spawnx is something we can provide, there may need to be another Python configuration flag set to disable that. However, I think that we may be able to ignore it until it becomes a problem.

task_manager_poll sounds like it might be a Panda thing? I’m not at my laptop right now but I’ll look later to see what I was using that for.

I don’t know why something is trying to import _winapi or pep517. Can you give me the exact error message that this is showing? Clearly, _winapi is not going to be available, but it’s not supposed to be.

Happens immediately:

$ python dmm_freezify.py
There are some missing modules: [’_winapi’, ‘pep517’]

There is a _winapi.c in the Python dist, and I was thinking of forcing that in the Makefile (but as you say, not a win system), but don’t know what I can do about pep517. I remember that was required in the requirements.txt builds.

Not sure what to try. Playing with the emscripten configure flags doesn’t seem to make any difference, but maybe I haven’t done the right combination.

BTW, the full java console error (I assume the map and MIME problems are irrelevant, but just for context) is:

A few notes in other forums seem to indicate a problem with freeze

cx-freeze problem on python3

another cx_freeze and python3.5

Oh, okay. Just ignore those _winapi and pep517 warnings, FreezeTool.py just picks up all imported modules, even conditional ones, that doesn’t necessarily indicate a problem.

The SystemError: unknown opcode seems like the error to actually look at. This might indicate a problem with the frozen bytecode somehow, hmm. Setting Py_VerboseFlag to 1 might shed additional light.