I’m looking to start in on handling my level-loading in a separate thread, to allow me to show a loading-screen and perhaps an animation without worrying about the process of loading the level causing halts or other issues.
Now, I’m aware that we can load models asynchronously.
However, I have a few other bits of logic to run as part of level-loading, and so I have it in mind to have the whole process be run in a separate thread.
I have two questions, then:
First, if I start a new thread that runs some method, and want to load a model as part of that method, should I use the asynchronous form of model-loading, or the standard one? My guess is that I should use the standard one, but I’m not confident in that guess.
And second, am I correct in thinking that I should make a point of not attaching my models to the scene-root until the thread is complete?
You can run the whole thing in a separate thread, in which case you can use the synchronous form of loadModel, though there are various hazards associated with threading that you need to be aware of. Unless you’re really doing nothing else except having Panda render its things, you may need to set up your own synchronization primitives to protect two threads from accessing the same variable at the same time.
You can also decide to use coroutines, which make it a lot easier to write asynchronous loading code without having to deal with threading. They don’t run your own processing code in a separate thread per se, but any asynchronous calls to Panda (eg. threaded loader.loadModel) can be awaited to allow parallel execution. You can also periodically yield to other tasks (including the render task), or await other asynchronous constructs. And, you can still run your coroutine on a separate task chain if you decide to use threading after all.
async def load_my_level():
model1 = await loader.loadModel("model1", blocking=False)
model2 = await loader.loadModel("model2", blocking=False)
... perform processing on model...
# Add the coroutine to the task manager
task = taskMgr.add(load_my_level())
I’ve made a start on loading my level in a separate thread and… it’s not going very well right now. I fear that the process is somewhat complex, and was written sufficiently long ago now that I don’t recall everything that it does offhand. :/
(It largely works, but there are a few major issues–including an occasional crash–and I don’t seem to get useful debugging output from the thread.)
So perhaps I should try the coroutine approach–if model-loading is the main source of level-loading’s duration, perhaps it will prove enough.
Thank you for suggesting it! I’ll likely give it a shot and report back…
(I’ve never really gotten along with threading, as far as I recall. At least threaded level-saving proved reasonably easy, I believe!)
Hmm… Python doesn’t seem to be recognising the “await” keyword–when I try to prepend it to a call to “loadModel” I get an “invalid syntax” error. :/
Here is the full error:
/usr/bin/python3.6 "/home/thaumaturge/Documents/My Game Projects/Adventuring/GameCore.py"
Traceback (most recent call last):
File "/home/thaumaturge/Documents/My Game Projects/Adventuring/GameCore.py", line 66, in <module>
from Adventuring.MainMenu import *
File "/home/thaumaturge/Documents/My Game Projects/Adventuring/Adventuring/MainMenu.py", line 9, in <module>
from Adventuring.Level import Level
File "/home/thaumaturge/Documents/My Game Projects/Adventuring/Adventuring/Level.py", line 24, in <module>
from Adventuring.GameObject import *
File "/home/thaumaturge/Documents/My Game Projects/Adventuring/Adventuring/GameObject.py", line 231
self.actor = await loader.loadModel(actorFile, blocking = False)
^
SyntaxError: invalid syntax
Process finished with exit code 1
[edit]
Based on a little searching, it seems that the “await” keyword may only be valid within an “async” method–and not in any other method called by an “async” method. However, if I declare the method with which I’m encountering this issue to also be “async”, I nevertheless still get the syntax error… :/
[edit 2]
No, wait, I’m wrong! It was a different await-statement that prompted the error when I declared that method to be “async”!
All right, so I suppose that my question now is this: is it safe to have a handful of methods declared as “async” that may or may not be called in coroutines? I have a suspicion from what I’ve been reading that it’s not, that when not called via a coroutine the "await"s contained within will become problems…
[edit 3]
Actually, nevermind that. Looking further at my code, I don’t think that coroutines are likely to work for me–at least not without more experience with them. My feeling at the moment is that the best way forward right now might be to significantly rework how I handle level-loading so that it’s much friendlier to being put into a thread–either that or drop threaded level-loading… :/
Right, to make a method into a coroutine, you should use the async keyword. You can call any method from a coroutine, but you can only await inside a coroutine, and not inside any other method.
I’m still debating whether to do a significant overhaul of my level-loading process in order to make it amenable to threading, or to accept a static loading screen and drop threaded loading. We’ll see…
For now, I think that we can consider this thread closed, and thank you for your help!
I feel like if your level loading architecture was already suited for threading, then it should be just as easy (or easier) to do async stuff (or even just busy wait on promises).
Do you have a simple, high-level overview of how you handle loading?
One option may to do a “pre-load” where you async load all of the assets you want then your actual level loading code should pull from the cache when doing a load.
I think the pre-load would look something like (untested):
async def pre_load():
show_load_screen()
for asset in asset_list:
await loader.load_model(asset, blocking=False)
hide_load_screen()
load_done() # do something here (e.g., game state switch) to let you know you can move to level loading
taskMgr.add(pre_load)
I imagine that you’re right. However, I fear that the way that I’ve gone about my level-loading setup isn’t wonderfully suited for threading… :/
Part of the problem, admittedly, is that I coded most of it over a fairly long period, and at least some of it a fair while ago now, so I don’t remember offhand everything that it does.
What I do have offhand, both by memory and quick inspection, looks something like this:
If called for, create a player-core object (the thing that persists between levels)
Create an inventory-object, and add an always-available item to it; this involves loading a model
Create the collectible-view (which may involve loading models)
Create the lore-view (which may involve loading text-files and images)
Create a Bullet-world
Setup at least one card on which the off-screen rendering buffer is shown
Set up my portal-culling cell-manager
Create a level-player object
Create various bits of Bullet-collision stuff for the player
Create and set up the shadow-rendering buffers, textures, and cameras, as well as their shaders
Load the sky-model
Load the level itself
Apply the appropriate lighting shaders
Set various shader-inputs
Load various bits of miscellaneous data
Load its geometry
Extract Bullet collision geometry via BulletHelper
Search through that geometry for various tags, which may specify such things as whether a piece of collision geometry should be made into a GameObject for interaction, where the portal-culling cells and portals lie, how objects should be shaded, whether objects should cast shadows, what shader-inputs should be applied, and so on.
Load player-data
Load various objects
Load their actors and construct their Bullet-collision shapes
Run a single Bullet physics-update
Do some final miscellaneous setup.
But there are complications. In a few places there is more than one path through the above basic idea, depending on whether we are starting a new game, moving from one level to another, or loading a saved-game, I think. I honestly don’t remember why I originally did it this way! :/
Hmm… That might work, but could also be awkward given my current save-game format. I think that it would mean either hunting through the save-/level- file for any model-references, or significantly changing how I lay out my files.
Hmm… That might be worth trying. (I haven’t checked the timing, but I’m reasonably confident that the most significant delay comes from loading the (rather large) level-files, and possibly in processing them.
In fact, thinking back to that blog-post that you linked to, this might be well-served by that callback approach: the level-geometry-loading method loads the geometry, specifying a callback, and then ends. The callback then does the processing. That might be worth a try indeed.
Thank you very much! I feel silly for not having thought of that!
I’ll likely give that a shot (once I’m done with the sound-effect work that I’m doing right now) and report back!
I don’t think that I have a loop in which I’m loading a set of assets in that way (not without re-working how I load my game-objects, at least), but it’s interesting to know that it’s available.
All right, I’ve tried Moguri’s suggestion above, and it does indeed seem to work.
It’s not as simple as it might seem: it essentially comes down to the following, within my level-class:
Interrupt the restoration of the level-object
(To be clear, this happens after reading data from file; the file-reading isn’t interrupted)
Store the remainder of the level-object data
Let the pre-interruption part of the restoration go to the point of calling “loadGeometry” as far as loading the level-geometry, which is done asynchronously with a callback
The callback then:
Performs the rest of the logic that was in “loadGeometry”
Initiates the remainder of the level-restoration process, and
Informs the game-framework that level-loading is done
Changes to the game-framework itself seem to be minor.
A quick test seems to indicate that the above does work. Furthermore, a quick-and-dirty timing-test does seem to indicate that the loading of the level-model is easily the main source of delay.
However, I am worried that I’m missing a path or detail somewhere, or that something happening during loading might mess things up in an unforeseen way.
So, I’m currently debating whether to go with the above, or to stick with my current, static loading screen.
Whichever I choose, thank you to both of you who have posted in this topic. It’s appreciated!
