Threading freeze, attribute error...

Game_Starter · March 19, 2016, 9:13pm

Hi to all,

I read that one is not supposed to use python threads but panda3d’s own threading implementation. But there is a problem when I want to use threads along with queues in order to share information between threads. Basically, each time a new model is added, a thread is started for the group it would belong to [groups are formed by models of the same texture and materials]. This thread takes the new node, scans the existing clusters within the groups and then adds it to the nearest cluster. It then forms a new node by flattening all the nodes within the cluster this new node was just added to [so small static nodes that are close to each other get flattened to form one node].

I tried implementing this with python threads and it worked fine until tonight. Suddenly, my program just freezes whenever I try and add a new node into the game world. I could switch to Panda’s threading implementation, but I get this error:

....
  File "C:\Panda3D-1.10.0\direct\gui\DirectButton.py", line 103, in commandFunc
    apply(self['command'], self['extraArgs'])
  File "holo_viewer.py", line 19, in drop_item_to_world
    self.optimizer_object_func(game_obj.game_nodes)
  File "holo_viewer.py", line 23, in optimizer_object_func
    n_group.thread_starter_inst()
  File "holo_viewer.py", line 511, in thread_starter_inst
    self.thread_s.setDaemon(True)
  File "C:\Panda3D-1.10.0\direct\stdpy\threading.py", line 66, in setDaemon
    if self.is_alive():
  File "C:\Panda3D-1.10.0\direct\stdpy\threading.py", line 57, in is_alive
    return self.__thread.isStarted()
AttributeError: node_instance_creater instance has no attribute '_ThreadBase__th
read'

Doing this:


        global instance_queue
        self.thread_s=node_instance_creater(instance_queue)
        self.thread_s.setDaemon(True)
        self.thread_s.start()

Causes the error, the line that says “self.thread_s.setDaemon(True)” brings it all up.

This error only happens if I do this:

from direct.stdpy import threading

It doesn’t happen if I do this:

import threading

But of course if I use the second import, then the issue of the program suddenly freezing occurs. It only started freezing tonight, before that, it ran correctly.
Specifically, the part that causes the freezing is when I call flattenStrong() on thew newly formed root node:

...
print "adding to root_node..."
new_node.reparentTo(root_node)
print "added to root_node, commence flattening..."
root_node.flattenStrong()#<-this appears to be where the program freezes...
print "flattening complete!"#<-this line is never printed out to the console
...

I suppose I should use the panda import, but how would I solve that attribute error? I am using panda3d v 1.10.0, this one I think: [2015-12-31T20:08:11Z - Add normal/depth information to box-box test to make it work with the pusher].

Any help is welcome, thanks.

Game_Starter · March 20, 2016, 5:04pm

Okay, going through the manual, helped me solve the attribute error, by replacing this line:

from direct.stdpy import threading

with this one:

from direct.stdpy import threading2 as threading

So that means that I am using panda3d’s implementation of threading and should be safe. However, the freezing still occurs whenever I call ‘flattenStrong()’ within the other non-main threads. How to solve this?

I am calling ‘Thread.considerYield()’ from the other threads already:

def run(self):
          Thread.considerYield()
          while True:
            ....

The models being flattened within the other threads are procedurally generated. The program as I said was working fine, until last night when it started freezing; I hadn’t made any changes to it.

Game_Starter · March 22, 2016, 4:37pm

I’ve discovered that the reason it suddenly started freezing was because I was performing the exact same process, but with a different procedural model. So the process worked with other models, but not with this one. The only difference between this model and the other models is the size. This model is just a little bit bigger. Both this model and the others have no texture or materials. They do have normal data.

When I try to run the “flattenStrong()” method on this model within another thread, the program freezes. Similarly, when I run the method “writeBamFile(model_name)” on this particular model within another thread, the program freezes.
The program does not freeze whenever I call both aforementioned methods on this particular model within the same thread. So it appears to only freeze whenever I call these methods within another thread.

-The program does not freeze with other relatively smaller models, whether I call either method within the same main thread or within other spawned threads.

So that’s what a little testing has revealed. I’ve attached .bam files of:

The model which causes the program to freeze, the entire .bam file named “freezing_model_whole.bam”. This file is from the model generated within the main thread.
The model which causes the program to freeze, the ‘corrupted’ version of the .bam file, since when I call “writeBamFile(model_name)” on the model within another thread, the program only writes part of it, but like I said, it freezes at a certain point. The file is named “freezing_model_corrupt.bam”.
Lastly, I’ve attached a copy of the smaller model which is both properly flattened and written out to disk within both the main thread, and the other thread(s). It is named “non_freezing_model.bam”.

I really hope someone more adept can help me with this, since to me it’s rather strange…
test_models.zip (26.8 KB)

rdb · April 1, 2016, 8:46am

I’ll check in a fix for the threading AttributeError today.

I see there’s no Python code in test_models.zip. It would be nice to have a test case for reproduction.

Note that to do flattenStrong asynchronously, you can also use loader.asyncFlattenStrong(), which will use a C++ thread for the purpose.

Game_Starter · April 3, 2016, 8:54pm

Thanks, well I’m kind of away now, but I’ll see if I can do test case of it when I get back home. One thing I forgot to say is that, when I save the procedural model that is producing the freeze as a .bam file within the main thread, and then load it, and later flatten it in the other thread(s), there is no problem at all. The program runs fine without any freezing. So I suppose that this is somehow connected to only procedurally generated models.

Okay, I’ll look at loader.asyncFlattenStrong(), the existence of that method shows a great deal of foresight!

Game_Starter · April 16, 2016, 11:41am

Hi, sorry it took terribly long to respond and provide you with a test-case, but here is a little snippet that reproduces the error I am encountering. Just copy paste it and run it. When running, press 1 to see that flattening is okay in the main-thread. Close the program and run it again, but this time press 2 to see that flattening is not okay [the program freezes] in the sub-thread.

from direct.showbase.ShowBase import ShowBase
from direct.task import Task
from panda3d.core import *
from direct.gui.DirectGui import *
from direct.stdpy import threading2 as threading
from Queue import Queue

#thread class, where the flattening will be done:
class node_instance_creater(threading.Thread):
      def __init__(self, queue):
              threading.Thread.__init__(self)
              self.queue = queue
      
      def run(self):
          Thread.considerYield()
          while True:
            dat=self.queue.get()              
            #flatten it:
            print "COMMENCE FLATTENING: ",dat.getName()
            dat.flattenStrong()
            print "YOU WON'T SEE ME CAUSE THE PROGRAM HANGS :("
            self.queue.task_done()

#main class:
class freezer(ShowBase):
    def __init__(self):
      ShowBase.__init__(self)
      #administrative stuff...
      array = GeomVertexArrayFormat()
      array.addColumn(InternalName.make('vertex'), 3,Geom.NTFloat32, Geom.CPoint)
      array.addColumn(InternalName.make('texcoord'), 2,Geom.NTFloat32, Geom.CTexcoord)
      array.addColumn(InternalName.make('normal'), 3,Geom.NTFloat32, Geom.CNormal)
      array.addColumn(InternalName.make('color'), 4,Geom.NTFloat32, Geom.CColor)
      format = GeomVertexFormat()
      format.addArray(array)
      format = GeomVertexFormat.registerFormat(format)
      self.vdata = GeomVertexData('name', format, Geom.UHStatic)
      self.vertex = GeomVertexWriter(self.vdata, 'vertex')
      self.normal = GeomVertexWriter(self.vdata, 'normal')
      self.color = GeomVertexWriter(self.vdata, 'color')
      self.texcoord = GeomVertexWriter(self.vdata, 'texcoord')
      self.geom = Geom(self.vdata)
      #model vertex-data:
      vertex_data=[LPoint3f(12, 0, -40), LPoint3f(20, 0, -40), LPoint3f(12, 0, -36),
      LPoint3f(20, 0, -36), LPoint3f(12, 24, -40), LPoint3f(20, 24, -40), LPoint3f(12, 24, -36), LPoint3f(20, 24, -36), LPoint3f(-20, 0, -32), LPoint3f(-16, 0, -32),
      LPoint3f(-20, 0, -28), LPoint3f(-16, 0, -28), LPoint3f(-20, 24, -32), LPoint3f(-16, 24, -32), LPoint3f(-20, 24, -28), LPoint3f(-16, 24, -28), LPoint3f(16, 0, -24), LPoint3f(20, 0, -24), LPoint3f(16, 0, -20), LPoint3f(20, 0, -20), LPoint3f(16, 24, -24), LPoint3f(20, 24, -24), LPoint3f(16, 24, -20), LPoint3f(20, 24, -20), LPoint3f(-16, 0, -20), LPoint3f(4, 0, -20), LPoint3f(-16, 0, -16), LPoint3f(4, 0, -16), LPoint3f(-16, 24, -20), LPoint3f(4, 24, -20), LPoint3f(-16, 24, -16),
      LPoint3f(4, 24, -16), LPoint3f(-16, 0, -12), LPoint3f(-12, 0, -12), LPoint3f(-16, 0, -8), LPoint3f(-12, 0, -8), LPoint3f(-16, 24, -12), LPoint3f(-12, 24, -12),
      LPoint3f(-16, 24, -8), LPoint3f(-12, 24, -8), LPoint3f(-32, 0, -8), LPoint3f(-28, 0, -8), LPoint3f(-32, 0, -4), LPoint3f(-28, 0, -4), LPoint3f(-32, 24, -8), LPoint3f(-28, 24, -8), LPoint3f(-32, 24, -4), LPoint3f(-28, 24, -4), LPoint3f(-8,0, 8), LPoint3f(0, 0, 8), LPoint3f(-8, 0, 12), LPoint3f(0, 0, 12), LPoint3f(-8,24, 8), LPoint3f(0, 24, 8), LPoint3f(-8, 24, 12), LPoint3f(0, 24, 12),LPoint3f(-28, 0, -24), LPoint3f(-24, 0, -24), LPoint3f(-28, 0, -16),
      LPoint3f(-24, 0, -16), LPoint3f(-28, 24, -24), LPoint3f(-24, 24, -24), LPoint3f(-28, 24, -16), LPoint3f(-24, 24, -16), LPoint3f(-28, 0, -12), LPoint3f(-24, 0,-12), LPoint3f(-28, 0, 0), LPoint3f(-24, 0, 0), LPoint3f(-28, 24, -12), LPoint3f(-24, 24, -12), LPoint3f(-28, 24, 0), LPoint3f(-24, 24, 0), LPoint3f(-24, 0, -32), LPoint3f(-20, 0, -32), LPoint3f(-24, 0, -8), LPoint3f(-20, 0, -8), LPoint3f(-24, 24, -32), LPoint3f(-20, 24, -32), LPoint3f(-24, 24, -8), LPoint3f(-20, 24, -8), LPoint3f(-24, 0, -4), LPoint3f(-20, 0, -4), LPoint3f(-24, 0, 4), LPoint3f(-20, 0, 4), LPoint3f(-24, 24, -4), LPoint3f(-20, 24, -4), LPoint3f(-24, 24, 4), LPoint3f(-20, 24, 4), LPoint3f(-20, 0, -20), LPoint3f(-16, 0, -20), LPoint3f(-20,0, -8), LPoint3f(-16, 0, -8), LPoint3f(-20, 24, -20), LPoint3f(-16, 24, -20), LPoint3f(-20, 24, -8), LPoint3f(-16, 24, -8), LPoint3f(-20, 0, 0), LPoint3f(-16, 0, 0), LPoint3f(-20, 0, 20), LPoint3f(-16, 0, 20), LPoint3f(-20, 24, 0), LPoint3f(-16, 24, 0), LPoint3f(-20, 24, 20), LPoint3f(-16, 24, 20), LPoint3f(-16, 0, 4),
      LPoint3f(-12, 0, 4), LPoint3f(-16, 0, 12), LPoint3f(-12, 0, 12), LPoint3f(-16,24, 4), LPoint3f(-12, 24, 4), LPoint3f(-16, 24, 12), LPoint3f(-12, 24, 12), LPoint3f(-16, 0, 16), LPoint3f(-12, 0, 16), LPoint3f(-16, 0, 24), LPoint3f(-12, 0, 24), LPoint3f(-16, 24, 16), LPoint3f(-12, 24, 16), LPoint3f(-16, 24, 24), LPoint3f(-12, 24, 24), LPoint3f(-12, 0, -12), LPoint3f(-8, 0, -12), LPoint3f(-12, 0, 12), LPoint3f(-8, 0, 12), LPoint3f(-12, 24, -12), LPoint3f(-8, 24, -12), LPoint3f(-12, 24, 12), LPoint3f(-8, 24, 12), LPoint3f(-12, 0, 20), LPoint3f(-8, 0, 20), LPoint3f(-12, 0, 28), LPoint3f(-8, 0, 28), LPoint3f(-12, 24, 20), LPoint3f(-8, 24, 20), LPoint3f(-12, 24, 28), LPoint3f(-8, 24, 28), LPoint3f(-8, 0, 24), LPoint3f(-4, 0, 24), LPoint3f(-8, 0, 44), LPoint3f(-4, 0, 44), LPoint3f(-8, 24, 24), LPoint3f(-4, 24, 24), LPoint3f(-8, 24, 44), LPoint3f(-4, 24, 44), LPoint3f(-4, 0,36), LPoint3f(0, 0, 36), LPoint3f(-4, 0, 44), LPoint3f(0, 0, 44), LPoint3f(-4, 24, 36), LPoint3f(0, 24, 36), LPoint3f(-4, 24, 44), LPoint3f(0, 24, 44), LPoint3f(0, 0, 8), LPoint3f(4, 0, 8), LPoint3f(0, 0, 24), LPoint3f(4, 0, 24), LPoint3f(0, 24, 8), LPoint3f(4, 24, 8), LPoint3f(0, 24, 24), LPoint3f(4, 24, 24), LPoint3f(0, 0, 32), LPoint3f(4, 0, 32), LPoint3f(0, 0, 40), LPoint3f(4, 0, 40), LPoint3f(0, 24, 32), LPoint3f(4, 24, 32), LPoint3f(0, 24, 40), LPoint3f(4, 24, 40), LPoint3f(4, 0, -28), LPoint3f(8, 0, -28), LPoint3f(4, 0, -12), LPoint3f(8, 0, -12),
      LPoint3f(4, 24, -28), LPoint3f(8, 24, -28), LPoint3f(4, 24, -12), LPoint3f(8, 24, -12), LPoint3f(4, 0, 4), LPoint3f(8, 0, 4), LPoint3f(4, 0, 24), LPoint3f(8, 0,24), LPoint3f(4, 24, 4), LPoint3f(8, 24, 4), LPoint3f(4, 24, 24), LPoint3f(8, 24, 24), LPoint3f(4, 0, 28), LPoint3f(8, 0, 28), LPoint3f(4, 0, 36), LPoint3f(8,0, 36), LPoint3f(4, 24, 28), LPoint3f(8, 24, 28), LPoint3f(4, 24, 36), LPoint3f(8, 24, 36), LPoint3f(8, 0, -40), LPoint3f(12, 0, -40), LPoint3f(8, 0, -16), LPoint3f(12, 0, -16), LPoint3f(8, 24, -40), LPoint3f(12, 24, -40), LPoint3f(8, 24, -16), LPoint3f(12, 24, -16), LPoint3f(8, 0, 0), LPoint3f(12, 0, 0), LPoint3f(8, 0, 8), LPoint3f(12, 0, 8), LPoint3f(8, 24, 0), LPoint3f(12, 24, 0), LPoint3f(8, 24, 8), LPoint3f(12, 24, 8), LPoint3f(8, 0, 12), LPoint3f(12, 0, 12), LPoint3f(8,0, 32), LPoint3f(12, 0, 32), LPoint3f(8, 24, 12), LPoint3f(12, 24, 12), LPoint3f(8, 24, 32), LPoint3f(12, 24, 32), LPoint3f(12, 0, -28), LPoint3f(16, 0, -28),
      LPoint3f(12, 0, 4), LPoint3f(16, 0, 4), LPoint3f(12, 24, -28), LPoint3f(16, 24,-28), LPoint3f(12, 24, 4), LPoint3f(16, 24, 4), LPoint3f(12, 0, 12), LPoint3f(16, 0, 12), LPoint3f(12, 0, 20), LPoint3f(16, 0, 20), LPoint3f(12, 24, 12), LPoint3f(16, 24, 12), LPoint3f(12, 24, 20), LPoint3f(16, 24, 20), LPoint3f(16, 0, -8),
      LPoint3f(20, 0, -8), LPoint3f(16, 0, 0), LPoint3f(20, 0, 0), LPoint3f(16, 24, -8), LPoint3f(20, 24, -8), LPoint3f(16, 24, 0), LPoint3f(20, 24, 0), LPoint3f(20,0, -40), LPoint3f(24, 0, -40), LPoint3f(20, 0, -20), LPoint3f(24, 0, -20), LPoint3f(20, 24, -40), LPoint3f(24, 24, -40), LPoint3f(20, 24, -20), LPoint3f(24, 24, -20)]
      #screenText:
      Instructions = OnscreenText(text="Press 1 to flatten in main thread.\n\nPress 2 to flatten in other thread.",
                           style=1, fg=(0, 0, 0, 1), pos=(0.06, -0.08),
                           align=TextNode.ALeft, scale=.05,
                           parent=base.a2dTopLeft)
      #some light:
      dlight = DirectionalLight('dlight')
      lens=PerspectiveLens()
      dlight.setLens(lens)
      dlight.setColor(VBase4(0.4,0.2,0.1, 1))
      dlnp = render.attachNewNode(dlight)
      dlnp.setHpr(0, -60, 0)
      render.setLight(dlnp)
      alight = AmbientLight('alight')
      alight.setColor(VBase4(0.15,0.15,0.15, 1))
      alnp = render.attachNewNode(alight)
      render.setLight(alnp)
      pntLight = PointLight("point")
      pntLight.setColor(Vec4(0.4,0.2,0.1, 1.0))
      pntNode = render.attachNewNode(pntLight)
      render.setLight(pntNode)
      #move the camera back a bit:
      base.cam.setPos(0,-200,0)
      #generate the data:
      counter=0
      global instance_queue
      instance_queue=Queue(maxsize=0)      
      for v_p in range(0,len(vertex_data),8):
          prim = GeomTriangles(Geom.UHStatic)
          #the top and bottom:
          self.draw_faces(prim,[vertex_data[v_p+2],vertex_data[v_p+3],vertex_data[v_p+6],vertex_data[v_p+7]],counter)
          self.draw_faces(prim,[vertex_data[v_p],vertex_data[v_p+1],vertex_data[v_p+4],vertex_data[v_p+5]],counter+4)
          #the back and front:
          self.draw_faces(prim,[vertex_data[v_p],vertex_data[v_p+1],vertex_data[v_p+2],vertex_data[v_p+3]],counter+8)
          self.draw_faces(prim,[vertex_data[v_p+4],vertex_data[v_p+5],vertex_data[v_p+6],vertex_data[v_p+7]],counter+12)
          #the left and right:
          self.draw_faces(prim,[vertex_data[v_p+4],vertex_data[v_p],vertex_data[v_p+6],vertex_data[v_p+2]],counter+16)
          self.draw_faces(prim,[vertex_data[v_p+5],vertex_data[v_p+1],vertex_data[v_p+7],vertex_data[v_p+3]],counter+20)
          self.geom.addPrimitive(prim)
          counter+=24
      node = GeomNode("Wild GeoDude Appeared!")
      node.addGeom(self.geom)

      self.nodePathaqq = render.attachNewNode(node)
      self.nodePathaqq.setTransparency(TransparencyAttrib.MAlpha,1)
      self.nodePathaqq.setTwoSided(True)                    
      #start the other thread:
      self.thread_starter_inst()
      #the accepts:
      self.accept('1',self.flattenModel,[0])
      self.accept('2',self.flattenModel,[1])
    
    def myNormalize(self,myVec):
       myVec.normalize()
       return myVec
    
    def draw_faces(self,prim_dat,array_sent,numbr):
            #1.vertex:
            self.vertex.addData3f(array_sent[0].x,array_sent[0].y, array_sent[0].z)
            self.normal.addData3f(self.myNormalize(Vec3(2*array_sent[0].x-1,2*array_sent[0].y-1,2*array_sent[0].z-1)))
            self.color.addData4f(1, 1, 1, 1)
            self.texcoord.addData2f(1, 0)
            
            #2.vertex:
            self.vertex.addData3f(array_sent[1].x,array_sent[1].y, array_sent[1].z)
            self.normal.addData3f(self.myNormalize(Vec3(2*array_sent[1].x-1,2*array_sent[1].y-1,2*array_sent[1].z-1)))
            self.color.addData4f(1, 1, 1, 1)
            self.texcoord.addData2f(1, 1)
            
            #3.vertex:
            self.vertex.addData3f(array_sent[2].x,array_sent[2].y, array_sent[2].z)
            self.normal.addData3f(self.myNormalize(Vec3(2*array_sent[2].x-1,2*array_sent[2].y-1,2*array_sent[2].z-1)))
            self.color.addData4f(1, 1, 1, 1)
            self.texcoord.addData2f(0, 1)
            
            #4.vertex:
            self.vertex.addData3f(array_sent[3].x,array_sent[3].y, array_sent[3].z)
            self.normal.addData3f(self.myNormalize(Vec3(2*array_sent[3].x-1,2*array_sent[3].y-1,2*array_sent[3].z-1)))
            self.color.addData4f(1, 1, 1, 1)
            self.texcoord.addData2f(0, 0)        
            
            #add to primitive:
            one=numbr
            two=numbr+1
            three=numbr+2
            four=numbr+3
            prim_dat.addVertices(one, two, three)
            prim_dat.addVertices(two, three, four)
    
    #this method starts the other thread:
    def thread_starter_inst(self):
        global instance_queue
        self.thread_s=node_instance_creater(instance_queue)
        self.thread_s.setDaemon(True)
        self.thread_s.start()
    
    #this method does the flattening, either in the main
    #thread, or the other thread.
    def flattenModel(self,Fmode):
        if(Fmode==0):
            #flatten in the main thread, so just do it!!:
            print "FLATTENING IN MAIN THREAD: "
            self.nodePathaqq.flattenStrong()
            print "MODEL FLATTENED!"
        elif(Fmode==1):
            #flatten in the other thread, so add it to the queue:
            global instance_queue
            instance_queue.put(self.nodePathaqq)
            instance_queue.join()
        
runner=freezer()
runner.run()

If indentation is a bother, then I also attached a copy of the code with this post. I doubt that it’s my code causing the freeze, but if it is, please point it out.

Thanks.
freeze_test_case.zip (2.77 KB)