Lag with asynchronous model loading

Pipeline
1

Hello all!

I’m working on a GTA like geometry loading system where tiled data is loaded only when the user/camera is in its vicinity. The problem is I get tremendous lag every time a new geometry is read from my .bam files. I’ve implemented asynchronous geometry loading by specifying a callback function in the call to loader.loadModel(). The callback stashes the geometry in a dict which I later read and display with .reparentTo(). I’ve two collision events to this end: one to start the loading and the other to display the buffered geometry. Is this a sensible approach? I noticed that loading and immediately displaying the geometry results in even larger lags, hence the two-step approach.

I’m using Panda3D 1.8 dev build 20120107 on Ubuntu Oneiric. Thread.isThreadingSupported() returns True.

The .bam files are 50KB on average with an average of 9MB of .png textures.

The manual states that there are two different threading interfaces. Since I’m using a default build I guess I’m using SIMPLE_THREADS, right? The manual also states that SIMPLE_THREADS is the simple of the two. I then wonder whether SIMPLE_THREADS is capable of asynchronously loading files i.e. do I need to compile Panda with the more powerful threading interface? Is lag-less loading of geometry possible in Panda? rdb’s comment on ‘true threading’ in this thread makes me uncertain…

Any idea what might be causing the lag? Am I using loadModel() correctly or do I need to import other modules first e.g. Panda’s threaded file module (direct.stdpy.file)?

Thanks in advance for any insights!
Simeon

PS I’m building a proof-of-concept prototype for a memory limited mobile device, hence I want to be able to load and unload geometry on demand.

2

the 50kb bam should load instantly, the lag you get is most likely due to the 9 megs png textures. they are both, big in file size and require a lot of processing power to be read.

try reading the manual chapter about compressed textures. panda3d.org/manual/index.php … ompression
personally i found txo files to load incredibly fast. they do have some limitations but if you are ok with those, you can gain a lot.

3

I think by “lag” you mean that you are seeing a momentary drop in frame rate (a visible chug) when you are running the asynchronous loading.

It is true that simply loading a model should not impact your visible frame rate, so something may be wrong. In the 1.8 build, SIMPLE_THREADS is off by default, so you should have the full threading model. Have you looked at the chug in PStats to see where the time is being spent? Can you produce a simple example program that illustrates the chug so I can see the problem too?

Now, reparenting the model to render for the first time can indeed cause a visible chug, because all of the vertex buffers and textures have to be loaded to the graphics card, and this cannot be done in a background task. There are ways to trickle this content in, though, if this becomes a problem, but first let’s understand precisely what’s going on.

David

4

Hey guys, thanks for the quick replies!

Yes, by “lag” I mean a freeze in the frame rate lasting until the geometry has completed loading i.e. a complete stop in rendering.

Judging from PStats, the chug is located in the igLoop as App -> Showcode -> igLoop.

@David Would a video of the chug alongside some code help? I’ve my files scattered across several places and it will take me some time and effort to produce a stand alone app. I can still do that if you prefer.

EDIT: video

As stated in my previous post I’ve two collisions that trigger the loading and display of geometry. The loading is started after the first collision event as

    def collision_load(self, entry):
        tile_id = entry.getIntoNodePath().getTag('tile_id')
        print "Loading tile," tile_id

        self.loader.loadModel('data/%s.bam' % tile_id, callback=self.fill_buffer, extraArgs=[tile_id] )

        x = float(self.tiles[tile_id]['x']) - self.world_x
        y = float(self.tiles[tile_id]['y']) - self.world_y
        cs = CollisionSphere(x, y, 0, self.distance_show)
        cs_show = self.render.attachNewNode(CollisionNode('show'))
        cs_show.node().addSolid(cs)
        cs_show.setTag('tile_id', tile_id)

        next_tile = self.tiles_ordered[self.tiles_ordered.index(tile_id) + 1]
        x = float(self.tiles[next_tile]['x']) - self.world_x
        y = float(self.tiles[next_tile]['y']) - self.world_y
        cs = CollisionSphere(x, y, 0, self.distance_load)
        cs_load = self.render.attachNewNode(CollisionNode('load'))
        cs_load.node().addSolid(cs)
        cs_load.setTag('tile_id', next_tile)

So I start the async loading and create two new collisions spheres, one for the show event of the current tile and one for the load event of the next tile (BTW, I travel a predetermined path so I know the order of the tiles).

The fill_buffer method stashes the async load result in a dict as

    def fill_buffer(self, *args):
        model = args[0]
        tile_id = args[1]
        print "Loaded tile", tile_id
        
        if model != None:
            if tile_id in self.buffered_tiles:
                self.buffered_tiles[tile_id]['model'] = model
            else:
                self.buffered_tiles[tile_id] = tile_id
                self.buffered_tiles[tile_id] = {'model':model}

The second collision event marks a geometry as ready for display as

    def collision_show(self, entry):
        tile_id = entry.getIntoNodePath().getTag('tile_id')

        if tile_id in self.buffered_tiles:
            self.buffered_tiles[tile_id]['show'] = True
        else:
            self.buffered_tiles[tile_id] = {'show':True}

Finally, a task checks whether a model is loaded and ready for display and displays it

    def check_geom_buffer(self, task):
        for tile_id in self.buffered_tiles.keys():
            if len(self.buffered_tiles[tile_id]) == 2:
                tile = self.buffered_tiles[tile_id]['model']
                x = float(self.tiles[tile_id]['x']) - self.world_x
                y = float(self.tiles[tile_id]['y']) - self.world_y
                tile.setPos(x, y, 0)
                tile.reparentTo(self.render)
                del self.buffered_tiles[tile_id]

        return task.again

The chug starts at the print statement “Loading tile” and finishes at the print statement “Loaded tile” i.e. exactly during the loading.

@Thomas I’ll look into texture compression, thanks for the suggestion. I’ve a fair number of tiles and textures so compressing is a good idea. I was planning to read about textures later as I first wanted to see if async loading would be sufficient.

5

Hmm, are you sure you are actually running 1.8, and you’re not accidentally running 1.7.2 or some older version? The behavior you’re reporting sounds like what I’d expect from 1.7.2, but not from 1.8.

What happens if you do:

print PandaSystem.getVersionString()
print Thread.isThreadingSupported(), Thread.isSimpleThreads()

?

David

6

Yep, 1.8 it is.

>>> print PandaSystem.getVerstionString()
1.8.0
>>> print Thread.isThreadingSupported(), Thread.isSimpleThread()
True False
7

In that case, would you mind putting together a simple program that demonstrates the problem? That would make it a lot easier for me to track it down further. Thanks!

David

8

For anyone coming upon this thread via a search, this issue has been fixed, and the fix will be available in the upcoming release of Panda.

bugs.launchpad.net/panda3d/+bug/1019599