Go David go!
Almost ok, but still getting pzip.exe and flt2egg.exe crashing at the very end of the build
build launched with
1>**Number of tasks left 31 out of [T1] Compressing 497built/models/misc/fade_sphere.egg.pz 1> starting tasksbuilt/bin/flt2egg -ps keep -o built/models/misc/fade_sphere.egg dmodels/src/misc/fade_sphere.flt 1>Reading dmodels/src/misc/camera.fltReading dmodels/src/misc/fade.flt 1>Writing built/models/misc/camera.egg 1>Writing built/models/misc/fade.egg 1>built/bin/pzip built/models/misc/camera.egg 1>built/models/misc/camera.egg.pz 1>The following command returned a non-zero value: built/bin/flt2egg -ps keep -o built/models/misc/fade_sphere.egg dmodels/src/misc/fade_sphere.flt 1>Storing dependency cache. 1>The following command returned a non-zero value: built/bin/pzip built/models/misc/camera.egg 1>The following command returned a non-zero value: built/bin/flt2egg -ps keep -o built/models/misc/fade.egg dmodels/src/misc/fade.flt 1>Elapsed Time: 1 hours 9 min 1>Build process aborting. 1>Build terminated. 1>Build log was saved at "file://c:\Users\jc\Desktop\PANDA_BUILD_BOT\Source MAJ\panda3d\built\BuildLog.htm" 1>makepanda - 0 error(s), 235 warning(s) ========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
I get the same thing. If I keep running the build repeatedly it eventually makes it to then end, but the resulting build crashes.
I think I found it. It was a problem with our underlying ptmalloc2 scheme, which didn’t like being asked to provide 16-byte alignment for some reason. I fixed it by not using that scheme anymore; it turns out not be the performance advantage it used to be anyway.
Let me know if you’re still seeing these crashes. My own Windows build seems to be stable.
too bad, something new is popping up :
Whoops, sorry, there are lots of different build options, and I missed a case.
Please try it again.
btw. where is LINMATH_ALIGN defined?
edit: DONE! Thank you once more David !
In makepanda.py. It’s hardcoded on by default.
I was able to build successfully, but got a bit of a snag here:
from panda3d.core import * from direct.showbase.ShowBase import ShowBase class Game(ShowBase): def __init__(self): """Get the game ready to play.""" ShowBase.__init__(self) self.node = render self.model = self.loader.loadModel('smiley') self.model.reparentTo(self.node) shaderdata = Mat4(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) self.model.setShaderInput('shaderdata', shaderdata) game = Game() game.run()
Assertion failed: sizeof(mat(0, 0)) * mat.size() == pta.size() * sizeof(pta) at line 338 of c:\work\panda3d\built\include\shader.I Traceback (most recent call last): File "test.py", line 18, in <module> game = Game() File "test.py", line 16, in __init__ self.model.setShaderInput('shaderdata', shaderdata) AssertionError: sizeof(mat(0, 0)) * mat.size() == pta.size() * sizeof(pta) at line 338 of c:\work\panda3d\built\include\shader.I
Whoops. Clearly when I said “I think I’m done” back in post 2 of this thread, what I really meant was “I’ve committed everything and haven’t tested a damn thing.”
Please give it another try now.
Forgive us to have put you under pressure… but you know Panda’s groupies are like spoiled children waiting anxiously for Xmas morning!
As far as I believe now, everything is correct and working. Please let me know how your own experiments go.
First thing I notice is Panda is using up a lot more system memory. Running my game the frame rate was actually a bit lower, but then I remembered about clearing out the BAM cache. Unfortunately, it is now crashing with an out of memory message (presumably while generating BAM files):
Couldn't allocate memory page of size 131072: Not enough storage is available to process this command. This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information.
That is for the game client. The server does run and takes up about 1.2GB of RAM as opposed to 0.6GB previously.
Hmm, that’s strange. I would have expected a bit more memory usage, due to the 16-byte memory alignment requirements, but not double. Maybe the code itself is now much bigger due to the template explosion problem? This bears investigation.
My early experiments seem to indicate about 20-30% more memory required by ordinary Panda operations. I’m trying a compile now with the original alternative malloc scheme to see how much of this bloat is due to alignment requirements and how much is due to the change in malloc schemes.
Is that 30% additional requirement enough to push you over the edge on the client? How close were you running previously? How much does memory grow before your client dies?
Edit: another thought–are you perhaps comparing against the memory usage in 1.7.2? It’s also possible that the threading change with 1.8.0 is another major contributor to more memory consumption.
Would you be satisfied to use a Panda3D compiled with Win64, to allow it take advantage of more than 2GB of RAM?
I am comparing against my previous build from cvs, both of which I have disabled threading in.
I may not have waited long enough for my original memory measurement, the server seems to be using 0.9GB on the previous build now, so the increase to 1.2GB would be consistent with the 30% you are seeing. Actually now that I think about it that is a ton of memory for what is actually going on in the server so perhaps something for me to investigate there.
It seems to use a lot more memory when it has to cache EGG to BAM files and not release that memory. For example on the previous build (before eigen), first run of the client after clearing the BAM cache it is using almost 1.5GB, but after closing it and running again it is using only 0.8GB. It is likely that this is the real reason why it is hitting the 32 bit memory limit, and it’s just that this extra 30% on top of that has pushed it over the limit. Looks like the client at about 1.8GB when it crashes. If it was going to be taking up 1.5GB in the previous build, then that extra 30% would put it at about 2GB.
I can’t rely on a 64 bit build since end users may be running a 32 bit OS.
As a quick & dirty turn around you can try
editbin /LARGEADDRESSAWARE yourapp.exe
This should provide a true 3gb addressable space on x32 machines
EDIT: recommandation: do it merely as a post build command
It appears after some experimentation that this 30% memory bloat is due to the need to impose an external 16-byte alignment on top of the system malloc library. If we go back to an alternative malloc library that can provide 16-byte alignment natively, this waste goes away. I’ll run a few more tests to confirm this, then commit this change.
jean-claude’s advice is good, though. Also, you might consider distributing your game to end-users with your egg files already prebuilt to bams, to avoid this potential problem in general (the egg loader was never intended to be particularly memory-efficient; it was primarily designed for the developer, not the end user).
Yes the plan would be to eventually distribute as BAM files. I just didn’t want to maintain a 64-bit build for development in addition to the 32-bit for distribution. The large address tweak sounds like it would be a good solution for development though.
About the EGG loader, I just found it strange that the memory it uses sticks around even after the game is loaded up. Shouldn’t that memory it uses to convert to BAM be able to be released once it has finished loading?
I suppose I could have some pre-process that runs and loads up the EGG files to populate the BAM cache before running the game itself. I had assumed that is what was happening under the hood anyways, but maybe I’ve misunderstood.
just for my understanding
Assuming the use of alternative malloc library:
(1) what about /zp16 option in the compiler?
(2) can EIGEN_ALIGN16 then still be used as recommended in eigen3?
ie for instance typedef Eigen3::Map<Eigen3::Vector4f, Eigen3::Aligned> Vector4fMap;