Offsetting indices in primitives using "offsetvertices"

Hi to all,
This question might be a bit tricky, but I’ll try and shorten it, so, I found it painfully slow to renumber the indices using a GeomVertexReader in a primitive that references vertices whenever I delete said vertices. Using this method, all works well, but it gets really slow for models with thousands of vertices:

...
primitiveArrayModder=GeomVertexWriter(vertexArrayData,0)
primitiveArrayReader=GeomVertexReader(vertexArrayData,0)
while not primitiveArrayReader.isAtEnd():
      oldRowIndex=primitiveArrayReader.getData1i()
      primitiveArrayModder.setData1i(newRowIndex)
...

The reader as I’m using it is slow whenever I’m dealing with a large model. So I opted to use the “primitive.offsetVertices(int offset, int begin_row, int end_row)” method, but it seems to have no effect. Here is how I am using it:

        #bufferDat->[] these are indices from the vertex list that are to be deleted.
        delPrimz=[]
        kounter=0
        #mark any row that references a deleted vertex for deletion:
        while not primitiveArrayReader.isAtEnd():
             rowIndex=primitiveArrayReader.getData1i()
             if(rowIndex in bufferDat):
                 delPrimz.append(kounter)
             kounter+=1
        #delete the row:
        for start in sorted(delPrimz, reverse=True):
            vertexArrayHandleP.setSubdata(start*lengthP, lengthP, '')
        rNumz2=vertexArrayData.getNumRows()
        #remove vertex-specific data:
        oldRowIndices=range(vdata.getNumRows())
        numVertReduce=0
        for start in sorted(bufferDat, reverse=True):
            vertexArrayHandle.setSubdata(start*length, length, '')
            numVertReduce-=1
            del oldRowIndices[start]
        #renumber the indices...
        if(len(delPrimz)>0):
            lRNum=rNumz2-1
            lenRem=len(delPrimz)
            beginUpdateRow=min(delPrimz)
            if(beginUpdateRow<=lRNum):
                prim.offsetVertices(numVertReduce, beginUpdateRow, lRNum)
                

Using it like that yields no changes at all. So what would the proper way to use the “offsetVertices()” method be so that I can quickly renumber the indices in the vertex list that a primitive references? As a test case, I wrote this fully functional script below:

class run_me(ShowBase):
    def __init__(self):
        ShowBase.__init__(self)
        #make the columns and formats:
        array = GeomVertexArrayFormat()
        array.addColumn(InternalName.make('vertex'), 3,Geom.NTFloat32, Geom.CPoint)
        array.addColumn(InternalName.make('texcoord'), 2,Geom.NTFloat32, Geom.CTexcoord)
        array.addColumn(InternalName.make('normal'), 3,Geom.NTFloat32, Geom.CNormal)
        array.addColumn(InternalName.make('color'), 4,Geom.NTFloat32, Geom.CColor)
        format = GeomVertexFormat()
        format.addArray(array)
        format = GeomVertexFormat.registerFormat(format)
        node = GeomNode("blockTestGeom")
        #the writers and geom and primitive:
        vdata = GeomVertexData('VertexData', format, Geom.UHStatic)
        vertex = GeomVertexWriter(vdata, 'vertex')
        normal = GeomVertexWriter(vdata, 'normal')
        color = GeomVertexWriter(vdata, 'color')
        texcoord = GeomVertexWriter(vdata, 'texcoord')
        tileGeom=Geom(vdata)
        tileGeom.setBoundsType (3)
        prim = GeomTriangles(Geom.UHStatic)
        counter=0
        self.blockDimensions=Point3(4,4,4)
        self.activeColor=Point4(1,1,1,1)
        #okay, now, to draw the faces:
        self.masterTestPointArray=[]
        self.activePointCloud=[]
        self.activePointCloud.extend([Point3(0,0,0),Point3(4,0,0),Point3(8,0,0)])
        #generate exemption list for both:
        exemptFaces1=self.returnExemptCubeFaces(Point3(0,0,0))
        exemptFaces2=self.returnExemptCubeFaces(Point3(4,0,0))
        exemptFaces3=self.returnExemptCubeFaces(Point3(8,0,0))
        self.generateCube(Point3(0,0,0),self.blockDimensions,exemptFaces1)
        self.generateCube(Point3(4,0,0),self.blockDimensions,exemptFaces2)
        self.generateCube(Point3(8,0,0),self.blockDimensions,exemptFaces3)
        for currentArray in self.masterTestPointArray:
            self.currentDrawArray=currentArray
            self.drawAFace(vertex,normal,color,texcoord,prim,counter)
            counter+=4
        prim.closePrimitive()
        tileGeom.addPrimitive(prim)
        node.addGeom(tileGeom)
        gotProcGeom = render.attachNewNode(node)
        #accept for deleting the middle-cube:
        self.accept("d", self.deleteCubeAtThisPoint,extraArgs=[Point3(4,0,0),gotProcGeom,"d"])
        self.accept("e", self.deleteCubeAtThisPoint,extraArgs=[Point3(4,0,0),gotProcGeom,"e"])
    
    def returnExemptCubeFaces(self,sentPoint):
        exemptFaces=[]
        
        leftPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        rightPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        frontPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        backPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        topPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        bottomPoint=Point3(sentPoint.x,sentPoint.y,sentPoint.z)
        
        leftPoint.x-=self.blockDimensions.x
        rightPoint.x+=self.blockDimensions.x
        frontPoint.y-=self.blockDimensions.y
        backPoint.y+=self.blockDimensions.y
        topPoint.z+=self.blockDimensions.z
        bottomPoint.z-=self.blockDimensions.z
        if(leftPoint in self.activePointCloud):
            #exempt the left point:
            exemptFaces.append(3)
        if(rightPoint in self.activePointCloud):
            #exempt the right point:
            exemptFaces.append(4)        
        if(frontPoint in self.activePointCloud):
            #exempt the front point:
            exemptFaces.append(1)
        if(backPoint in self.activePointCloud):
            #exempt the back point:
            exemptFaces.append(2)
        if(topPoint in self.activePointCloud):
            #exempt the top point:
            exemptFaces.append(5)
        if(bottomPoint in self.activePointCloud):
            #exempt the bottom point:
            exemptFaces.append(6)
        
        return exemptFaces
        
    def generateCube(self,originPoint,cubeDimension,exemptFaces):
        for i in range(1,7,1):
            if (i not in exemptFaces):
                #draw it:
                gotArray=self.returnProperFaceCoordinates(originPoint,cubeDimension,i)
                self.masterTestPointArray.append(gotArray)
    
    def drawAFace(self,*args):
        #structure is: 
        #0->positional data.
        #1->normal data.
        #2->color data.
        #3->uv data.
        #4->primitive.
        #5->current starting index for primitive.
        vertex=args[0]
        normal=args[1]
        color=args[2]
        texcoord=args[3]
        prim_dat=args[4]
        numbr=args[5]
        for specificPoint in self.currentDrawArray:
            vertex.addData3f(specificPoint.x,specificPoint.y,specificPoint.z)
            normal.addData3f(self.myNormalize(Vec3(2*specificPoint.x-1,2*specificPoint.y-1,2*specificPoint.z-1)))
            color.addData4f(self.activeColor.x, self.activeColor.y, self.activeColor.z, self.activeColor.w)
            texcoord.addData2f(1, 1)
        prim_dat.addVertices(numbr, numbr+1, numbr+2)
        prim_dat.addVertices(numbr, numbr+2, numbr+3)
    
    
    def myNormalize(self,myVec):
       myVec.normalize()
       return myVec
        
    def returnProperFaceCoordinates(self,*args):
        #0->(xOrigin,yOrigin,zOrigin)
        #1->(xDimension,yDimension,zDimension)
        #2->side to draw: 1,2,3,4,5,6: front,back,left,right,top,bottom
        originPoint=args[0]
        dimensionData=args[1]
        sideToDraw=args[2]
        if(sideToDraw==1):
            #drawing the front:
            point1=Point3(originPoint.x,originPoint.y,originPoint.z)
            point2=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z)
            point3=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z+dimensionData.z)
            point4=Point3(originPoint.x,originPoint.y,originPoint.z+dimensionData.z)
        elif(sideToDraw==2):
            #drawing the back:
            point1=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z)
            point2=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z)
            point3=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
            point4=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
        elif(sideToDraw==3):
            #drawing the left:
            point1=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z)
            point2=Point3(originPoint.x,originPoint.y,originPoint.z)
            point3=Point3(originPoint.x,originPoint.y,originPoint.z+dimensionData.z)
            point4=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
        elif(sideToDraw==4):
            #drawing the right:
            point1=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z)
            point2=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z)
            point3=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
            point4=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z+dimensionData.z)
        elif(sideToDraw==5):
            #drawing the top:
            point1=Point3(originPoint.x,originPoint.y,originPoint.z+dimensionData.z)
            point2=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z+dimensionData.z)
            point3=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
            point4=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z+dimensionData.z)
        elif(sideToDraw==6):
            #drawing the bottom:
            point1=Point3(originPoint.x,originPoint.y+dimensionData.y,originPoint.z)
            point2=Point3(originPoint.x+dimensionData.x,originPoint.y+dimensionData.y,originPoint.z)
            point3=Point3(originPoint.x+dimensionData.x,originPoint.y,originPoint.z)
            point4=Point3(originPoint.x,originPoint.y,originPoint.z)
        return [point1,point2,point3,point4]
        
    #the delete process:
    def deleteCubeAtThisPoint(self,sentPoint,sentNp,keyPresser):
        self.removeFaceFromExistingNodePath(sentPoint,1,sentNp,keyPresser)
        self.removeFaceFromExistingNodePath(sentPoint,2,sentNp,keyPresser)
        self.removeFaceFromExistingNodePath(sentPoint,3,sentNp,keyPresser)
        self.removeFaceFromExistingNodePath(sentPoint,4,sentNp,keyPresser)
        self.removeFaceFromExistingNodePath(sentPoint,5,sentNp,keyPresser)
        self.removeFaceFromExistingNodePath(sentPoint,6,sentNp,keyPresser)
    
    
    def removeFaceFromExistingNodePath(self,sentPoint,faceSide,sentNp,keyPresser):
        gotFaceArrayPoints=self.returnProperFaceCoordinates(sentPoint,self.blockDimensions,faceSide)
        terrainGeomNode=sentNp.node()
        specificGeom=terrainGeomNode.modifyGeom(0)

        vdata = specificGeom.modifyVertexData()
        vertexReader = GeomVertexReader(vdata, 'vertex')
        prim=specificGeom.modifyPrimitive(0)
        
        vertexArrayData=prim.modifyVertices()
        vertexArrayHandleP=vertexArrayData.modifyHandle()
        vertexArrayFormatP=vertexArrayHandleP.getArrayFormat()
        lengthP=vertexArrayFormatP.getStride()
        primitiveArrayReader=GeomVertexReader(vertexArrayData,0)
        
        vertexArray=vdata.modifyArray(0)
        vertexArrayHandle=vertexArray.modifyHandle()
        vertexArrayFormat=vertexArrayHandle.getArrayFormat()
        length=vertexArrayFormat.getStride()
        
        bufferDat=[]
        gotIt=prim.getVertexList()
        minVal=min(gotIt)
        maxVal=max(gotIt)
        for indxx in xrange(minVal,maxVal,4):
            vi=indxx
            vii=indxx+1
            viii=indxx+2
            viv=indxx+3
            
            vertexReader.setRow(vi)
            v1 = vertexReader.getData3f()

            vertexReader.setRow(vii)
            v2 = vertexReader.getData3f()

            vertexReader.setRow(viii)
            v3 = vertexReader.getData3f()

            vertexReader.setRow(viv)
            v4 = vertexReader.getData3f()
            faceArrayTest=[v1,v2,v3,v4]
            foundPoints=0
            for singlePoint in faceArrayTest:
                if(singlePoint in gotFaceArrayPoints):
                    foundPoints+=1
            if(foundPoints==4):
                bufferDat.extend([indxx,indxx+1,indxx+2,indxx+3])
                break
        
        #now, remove the relevant vertices from the primitive:
        delPrimz=[]
        kounter=0
        while not primitiveArrayReader.isAtEnd():
             rowIndex=primitiveArrayReader.getData1i()
             if(rowIndex in bufferDat):
                 delPrimz.append(kounter)
             kounter+=1
        for start in sorted(delPrimz, reverse=True):
            vertexArrayHandleP.setSubdata(start*lengthP, lengthP, '')
        rNumz2=vertexArrayData.getNumRows()
        #remove vertex-specific data:
        oldRowIndices=range(vdata.getNumRows())
        numVertReduce=0
        for start in sorted(bufferDat, reverse=True):
            vertexArrayHandle.setSubdata(start*length, length, '')
            numVertReduce-=1
            del oldRowIndices[start]
        #renumber the indices...
        if(len(delPrimz)>0):
            if(keyPresser=="d"):
                lRNum=rNumz2-1
                lenRem=len(delPrimz)
                beginUpdateRow=min(delPrimz)
                if(beginUpdateRow<=lRNum):
                    print "OFFSET-DATA: ",numVertReduce,beginUpdateRow, lRNum,maxVal
                    prim.offsetVertices(numVertReduce, beginUpdateRow, lRNum)
                    print "Change: ",max(prim.getVertexList())
                    gotTest=vertexArrayHandleP.getSubdata(beginUpdateRow*lengthP, lengthP)
                    print "TEST: ",gotTest
            else:
                self.renumberVertInPrims(oldRowIndices,vertexArrayData)
        
    def renumberVertInPrims(self,oldRowIndices,vertexArrayData):
        #go through the primitives in this geom:
        primitiveArrayModder=GeomVertexWriter(vertexArrayData,0)
        primitiveArrayReader=GeomVertexReader(vertexArrayData,0)
        while not primitiveArrayReader.isAtEnd():
            oldRowIndex=primitiveArrayReader.getData1i()
            newRowIndex=oldRowIndices.index(oldRowIndex)
            primitiveArrayModder.setData1i(newRowIndex)
        
w = run_me()
w.run()

Just copy-paste to run. Press “d” to delete the central cube where the renumbering occurs via the “offsetvertices” method, or “e” to renumber it using the primitiveArrayReader.
Another thing, what type of string does

setSubData(int start, int size, VectorUchar data)

expect? It can be used to outright delete primitive or vertex data by setting the string to nothing, but what if one wanted to change the data instead of deleting it? What type of content would “VectorUchar” be?
Thanks in advance and forgive the long question, I hope it makes sense, if any part is unclear, just tell me and I’ll try and clarify what I meant. :smile:

I can’t get your code to run in Python 3 at all. After fixing all the Python 2-isms, I get this when I press “d”:

OFFSET-DATA:  -4 30 77 55
Change:  55
TEST:  b'\x14\x00'
OFFSET-DATA:  -4 30 71 55
Change:  55
TEST:  b'\x14\x00'
OFFSET-DATA:  -4 30 65 55
Change:  55
TEST:  b'\x14\x00'
OFFSET-DATA:  -4 30 59 55
Change:  55
TEST:  b'\x14\x00'
:gobj(error): GeomTriangles references vertices up to 55, but GeomVertexData has only 40 rows!
Assertion failed: reader.check_valid(data_reader) at line 1864 of panda/src/gobj/geom.cxx

Glancing at your code, it is not quite obvious to me that you’re making an apples to apples comparison, though. offsetVertices adds a constant offset to every index in the given range, but renumberVertsInPrims does not obviously appear to be doing the same thing. If I were able to run it, I might print out the values of oldRowIndices to see if that is correct.

VectorUchar accepts a bytes object or a bytes-compatible object. However, I do not recommend using it. I recommend instead using copySubdataFrom, which is a faster way to do the same thing, and it accepts any bytes-compatible object or object supporting the buffer protocol.

Are you aware that Python 2 support will end soon? Do you have plans to upgrade?

It seems to be a very small mistake; lRNum=rNumz2-1 in removeFaceFromExistingNodePath should just be lRNum=rNumz2:

#renumber the indices...
if(len(delPrimz)>0):
    if(keyPresser=="d"):
        lRNum=rNumz2

It seems to work fine with that change :slight_smile: .

Okay, thanks, but what would be the content of the bytes object/object supporting the buffer protocol? For example, using this version of the copySubdataFrom:

copySubdataFrom (int to_start, int to_size, object buffer)

I assume that to_start and to_size would be the range of the rows in the primitive I would want to affect, and the buffer would be the new content I want that range to have, like to overwrite it. So if I had entries in the row referencing vertices 1 to 10 consecutively and I wanted to make them now reference 20 to 30, how would I do that using that particular version of the mentioned method? Could you give a brief example if it’s not too much to ask?

Yes I am and not only that, I’m also aware that I’m using a slightly older version of panda3d that doesn’t reflect some changes you’ve made to the newer version. I do have plans to upgrade to both python 3 and the most recent version of panda3d, but I will cross that bridge when I get to it.

Thank you! That indeed did fix everything, though the reason I was subtracting one from that variable was because I thought that version of the method required the last parameter to be the last index of the row and not the number of rows contained…Like:

rNumz2=vertexArrayData.getNumRows()
lRNum=rNumz2-1
prim.offsetVertices(numVertReduce, beginUpdateRow, lRNum)

So it is the number of rows and not the last index number that is sent, in that case?

It’s probably meant to work like the arguments of the range(start, stop) function, where stop is not included in the resulting sequence (i.e. 1 bigger than the biggest integer in that sequence). This is not clearly indicated in the API docs, though, and I do recall having fallen victim to the same confusion in the past.

Alright, another subtle thing to remember, thanks as always. :grin:

So after a bit of research, I was able to use copySubdataFrom to update the value of the indices that the primitive references:

...
gotTest=vertexArrayHandleP.getSubdata(beginUpdateRow*lengthP, lengthP*lRNum)
newReplaceStr=''
unpackedBigg=struct.unpack('B'*len(gotTest), gotTest)
for strEntry in gotTest:
    unpackedGot=struct.unpack('B', strEntry)
    mutableVal=unpackedGot[0]
    if(unpackedGot[0]!=0):
        mutableVal+=numVertReduce
    newReplaceStr+=struct.pack('B',mutableVal)
vertexArrayHandleP.copySubdataFrom (beginUpdateRow*lengthP, lengthP*lRNum, newReplaceStr)
print unpackedBigg
...

That indeed did work, except, I kept getting zeros between the values I actually wanted to manipulate:

(24, 0, 25, 0, 26, 0, 24, 0, 26, 0, 27, 0, 28, 0, 29, 0, 30, 0, 28, 0, 30, 0, 31
, 0, 32, 0, 33, 0, 34, 0, 32, 0, 34, 0, 35, 0, 36, 0, 37, 0, 38, 0, 36, 0, 38, 0
, 39, 0, 40, 0, 41, 0, 42, 0, 40, 0, 42, 0, 43, 0)

I don’t know if that’s part of the formatting used to store index data, or am I requesting it in the wrong way?

Lastly, will you be supporting 32 bit systems from an end-user perspective for much longer? I mean when shipping a product to a consumer, will their machine have to be a 64 bit one?
(I plan to upgrade everything this coming weekend, I guess you can’t fight change forever…:sweat_smile:).
Thanks as always.

You are using the B type, which indicates a number in the range 0-255 stored in a single byte. However, the indices are by default stored as numbers in the range 0-65534, as an unsigned short. You should use H instead, which should produce the correct result.

Instead of struct, if you want to manipulate the indices directly, it may be more efficient to use array.array('H'). In fact, instead of modifyHandle(), it may be even faster to use Python’s memoryview interface to allow Python to directly manipulate Panda memory (though you need to be sure that the size of the array is set correctly before manipulating it):

view = memoryview(vertexArrayData)
view[0] = 24
view[1] = 25

Great to hear! We have no plans to drop support for 32-bit systems, except on macOS.