AsyncFuture returns before the real Future has completed

I’m trying to use loadModel’s callback optional parameter to asynchronously load 2 large models. Each loadModel call returns a direct.showbase.Loader.Loader._callback object. My code waits on this object using the result() method, which does return after a few seconds. However, the return is well before Loader calls the callback. After waiting for result(), there is nothing actionable for the program, because no observable future has completed.

Unless a program is interested in the internal timing of the Loader, there doesn’t seem to be any utility in waiting for result(). Synchronization between the initiating program and the callback function apparently requires other methods.

Is there something that I’m missing?
Would completion of the _callback future be more usefully defined to be the [observable and useful] completion of the callback function?

Code example:

from time import time
from direct.showbase.ShowBase import ShowBase, taskMgr
import direct.showbase.Loader as dsl
from panda3d.core import NodePath, Thread


class TstApp(ShowBase):
    def __init__(self):
        ShowBase.__init__(self)
        self.model1 = None
        self.model2 = None
        print(f"Thread.isThreadingSupported() is {Thread.isThreadingSupported()}")
        taskMgr.add(self.load_models, 'load')

    def load_models(self, task):
        self.t0 = time()
        print(f"t=0 calling loadModel(model1)")
        self.model1_future = self.loader.loadModel('model1.bam', callback=self.model1_callback)
        print(f"model1_future is {type(self.model1_future)}")
        print(f"t={time()-self.t0:4.3f} returned from loadModel('model1.bam')")
        print(f"t={time()-self.t0:4.3f} calling loadModel('model2.bam')")
        self.model2_future = self.loader.loadModel('model2.bam', callback=self.model2_callback)
        print(f"model2_future is {type(self.model2_future)}")
        print(f"{time()-self.t0:4.3f} returned from loadModel('model2.bam')")
        result:dsl.Loader._Callback = self.model1_future.result()
        print(f"t={time()-self.t0:4.3f} model1_future.result()={result}")
        print(f"code to process self.model1 would fail because self.model1={self.model1}")
        result:dsl.Loader._Callback= self.model2_future.result()
        print(f"t={time()-self.t0:4.3f} model2_future.result() {result}")
        print(f"code to process self.model2 would fail because self.model2={self.model2}")

    def model1_callback(self, np: NodePath):
        print(f"t={time()-self.t0:4.3f} model1_callback")
        self.model1 = np

    def model2_callback(self, np: NodePath):
        print(f"t={time()-self.t0:4.3f} model2_callback")
        self.model2 = np


tst = TstApp()
tst.run()

Results when I run the program:
Thread.isThreadingSupported() is True
t=0 calling loadModel(model1)
model1_future is <class ‘direct.showbase.Loader.Loader._Callback’>
t=0.001 returned from loadModel(‘model1.bam’)
t=0.001 calling loadModel(‘model2.bam’)
model2_future is <class ‘direct.showbase.Loader.Loader._Callback’>
0.001 returned from loadModel(‘model2.bam’)
t=1.454 model1_future.result()=None
code to process self.model1 would fail because self.model1=None
t=4.196 model2_future.result() None
code to process self.model2 would fail because self.model2=None
t=4.215 model1_callback
t=4.215 model2_callback

Yes, the callback is slower, for historical reasons (it still uses the event system, since that mechanism predated the existence of the new futures). It might be slightly faster to call future.addDoneCallback(...) instead of using the callback= argument, but the callback will still be slower than awaiting the future.

I am struggling to see the use case of combining the callback functionality with the future, though. Since you are using result() to (in essence) make the load synchronous, can’t you just call the callback right after result() returns if you want to control when it is precisely called?

I would discourage the use of result() to wait for the result. It should generally only be used in two circumstances:

  • If you expect that the future is already done or highly likely to be done, and you want to extract the result
  • If you are trying to wait for a future in a worker thread, and the thread has nothing else to do until the future is done so you want to put it to sleep

If you want to make this truly asynchronous, you should rewrite your code like so, which is how it is intended to be used (using async/await syntax, which unlike result() is non-blocking):

        taskMgr.add(self.load_models(), 'load')

    async def load_models(self):
        self.model1_future = self.loader.loadModel('model1.bam', blocking=False)
        self.model2_future = self.loader.loadModel('model2.bam', blocking=False)

        self.model1 = await self.model1_future
        self.model2 = await self.model2_future

Actually, you can await both futures at the same time, like so, although you won’t be able to start processing the first model yet until both are done:

    async def load_models(self):
        self.model1, self.model2 = await AsyncFuture.gather(self.model1_future, self.model2_future)

You could also simplify it down to this, since loadModel allows taking a list:

    async def load_models(self):
        self.model1, self.model2 = self.loader.loadModel(['model1.bam', 'model2.bam'], blocking=False)

You can also use async for, which does allow you to start processing each model as it’s loaded, though only in sequence:

    async def load_models(self):
        async for model in self.loader.loadModel(['model1.bam', 'model2.bam'], blocking=False):
            self.models.append(model)

If you want to ensure that you can start processing the result of the load as soon as the model in question is done, even if the second model finishes loading first, then you should either (1) use a separate coroutine invocation for each model, or (2) continue to use callbacks, but create your own futures that you gather and await in your callback to do the synchronization.

In any case, you may want to consider setting loader-num-threads 2 in Config.prc (the default is 1) to ensure that the two models can be loaded in parallel, otherwise they will still be loaded in sequence.