Altering Geometry by Memoryview: Geometry Not Appearing?

[Thaumaturge]

[edit 2]
I’ve made some changes to this process, as described in the edits below. For the sake of clarity, then, I’ve edited the below description of my approach to better match what I have at time of this edit.
[/edit 2]

So, as per the conclusion of this thread, I’m attempting to use a memoryview in order to alter geometry on a per-frame basis. (Having switched from the use of stored GeomVertexWriters, which I’m told is unwise.)

After some trial and error I’ve managed to get to the point at which the code runs–but for some reason I’m not seeing any geometry. (Where, please note, the geometry was visible when using GeomVertexWriters.)

I would seem to be doing something wrong–but I don’t know quite what. (Especially as I’m rather new to the use of memoryviews.)

In short, my approach is as follows:

During initialisation:

• Create a GeomVertexFormat and a GeomVertexData
  • My choice of format is “v3n3t2”
• Set the number of rows in the latter to reflect the number of vertices
  • In this case, that amounts to (12) * (number of data-points used), for a naive definition of four triangles per data-point.
• Create a memoryview for the GeomVertexData, casting it to “B” and then to “f”
• Create my primitive (a GeomTriangles)
• Set the appropriate indices in the primitive
• Set up a GeomNode and NodePath using that primitive

Then, during updates:

• Create an array, with format “f”, initialised with an empty list
  • i.e. Like so: array.array("f", [])
• Iterate over the data that defines the vertices
  • For each data-point:
    • Perform various calculations
      • (Noting that these should be unchanged since the version that worked when using GeomVertexWriter)
    • Add to the array via “fromlist”
      • Specifically, adding the following four triangles:

           dataList.fromlist(
               [
               # Tri 1
               vert1.x, vert1.y, vert1.z,
               normal1.x, normal1.y, normal1.z,
               uv1.x, uv2.y,
        
               # Tri 2
               # As with Tri 1, but with its own data
        
               # And so on for Tris 3 and 4...
               ]
           )
        

• At the end, update the memoryview like so:

      self.memoryviewObject[:len(dataList)] = dataList
  

And that’s basically it.

Does anyone see where I might be going wrong…? Is there a format issue with the array? Or the order in which I’m adding data? Or something else…?

Note that I’m not seeing junk vertices–I’m just seeing… nothing.

[edit]
Okay, I’ve realised one mistake, at least:

I had the number of rows being set to “(number of elements in the format) * (number of vertices) * (total number of numeric values across the format)”.

However, I now think that a row already incorporates the number of numeric values and the number of elements in the format–the former of which includes the latter, for that matter. This then just leaves the number of vertices.

Further, I was miscalculating the number of vertices.

The result is now, essentially: (number of data-points) * (number of vertices per data-point)

(The latter being 12 in my case; I could reduce this by re-using vertices, but for now at least separate vertices makes for simpler logic.)

However, having corrected this, I believe, I’m still seeing no geometry… :/

[edit 2]
Oh, and I’ve removed the use of the GeomVertexWriters; I’m just leaving the data as default until it’s modified in the update code.

[Thaumaturge]

(Separate post as it may include the answer–pending confirmation that this is the right way to do this. As such, a separate post might make it easier for people to find said answer–whether to use it themselves or to refute it.)

Okay, I think that I may have found the solution!

It seems that one isn’t expected to just make a memoryview and then keep it.

Instead, one makes a new memoryview each update!

Thus, the approach ends up looking something like this:

During initialisation:

• Create a GeomVertexFormat and a GeomVertexData
• Set the number of rows in the latter to reflect the number of vertices
• Create the primitive (in this case a GeomTriangles)
• Set up a GeomNode and NodePath using that primitive

Then, during updates:

• Create an array, with format “f”, initialised with an empty list
  • i.e. Like so: array.array("f", [])
• Iterate over the data that defines the vertices
  • For each data-point:
    • Perform various calculations
    • Add to the array via “fromlist”
• At the end, update the memoryview like so:
  self.memoryviewObject[:len(dataList)] = dataList

Does that seem right…?

[edit]
Just to check: Should I be cleaning up my memoryviews in some way…?

[rdb]

I only had time to skim your posts, I will revisit this thread later.

Just like GeomVertexWriter, a memoryview is created when you want to modify a geom. Panda will record that the geometry is changed at the moment the memoryview is created. So, if you do not create a new memoryview, Panda will not know something has changed.

If you’re going to use a memoryview, there seems to be little point in first putting your data in a list, then an array.array, then copying that into the memoryview. You can write the data directly into the array, directly into the memoryview, or directly copy from the list into the memoryview.

Do a print() of the GeomVertexArrayData after writing. It’ll tell you what data is actually in there, and you can see what went wrong.

[Thaumaturge]

@rdb

If you’re going to use a memoryview, there seems to be little point in first putting your data in a list, then an array.array, then copying that into the memoryview.

It does seem a little convoluted! But at the time it seemed like the most reasonable way that I saw…

@rdb

You can write the data directly into the array, directly into the memoryview …

I suppose–but that would be (if I’m not much mistaken) 96 separate additions to the array/memoryview per data-point. (That is: there are four triangles, each with three vertices, each of which has eight floats–three location-values, three normal-values, and two uv-values. 4 * 3 * 8 = 96.)

I think that I guessed that adding all 96 within a single list would be more efficient than having 96 separate additions.

@rdb

… or directly copy from the list into the memoryview.

I had the impression that it wouldn’t accept that, I’ll admit. (Which I think was the whole reason that I used an “array” at all–a Python structure that I’ve never really used before, that I recall, generally sticking with built-in lists.)

And indeed, trying it, I get the following error:

TypeError: a bytes-like object is required, not 'list'

That from a simple assignment, something like the below:

dataAsList = []

# Fill in the list with float-values

myMemoryView[:len(dataAsList)] = dataAsList

[rdb]

@Thaumaturge

I think that I guessed that adding all 96 within a single list would be more efficient than having 96 separate additions.

But now you’re paying the cost for adding them to the list and then for copying from the list to the array and then for copying from the array to the memoryview, whereas you could’ve been adding them to the memoryview (or to the array) directly and skipped one of those costly operations. Unless the list is pre-computed at another time, and you’re dynamically copying in sections of pre-computed data? In which case, storing them pre-computed as bits of array.array and copying the data from there in blocks would still be more efficient.

@Thaumaturge

And indeed, trying it, I get the following error:

Yes, you’re right, you need to convert the list to an array. Since array has an append and such, maybe you can just start with an array and fill that instead of starting with a list.

[Thaumaturge]

@rdb

But now you’re paying the cost for adding them to the list …

Well, not entirely, I would think, as I’m constructing the list in-place. Essentially something like this:

myList = [
 x, y, z,
normalX, normalY, normalZ,
u, v,

# And so on for eleven more such sets of values...
]

(Save that I’m not actually placing the list into a named variable, but rather just creating it within the call to “array.fromlist”.)

However, thinking about this again, and going back to the documentation, I see that doing so " is equivalent to for x in list: a.append(x)".

As such, I presumably am getting hit for all 96 additions to the array, even so…

That said, I do wonder: is a call to a function (in this case “array.append”) more efficient within a for-loop, or is it the same as doing it with individual calls…?

@rdb

Unless the list is pre-computed at another time, and you’re dynamically copying in sections of pre-computed data?

No, it’s all computed on the fly. Somewhat naively, but well enough so far, I think.

@rdb

Yes, you’re right, you need to convert the list to an array.

So, just to clarify, when above you mention copying from a list to a memoryview, do you mean on an individual-element level–as in something like this: myMemoryView[i] = myList[i]?

@rdb

Since array has an append and such, maybe you can just start with an array and fill that instead of starting with a list.

Perhaps you are right!

I will say that it won’t be as neatly and clearly formatted, since I’ll presumably have to add each component individually, which is a downside.

That is, I’d be going from something like this:

myArray.fromList(
    [
        # Tri 1
        x1, y1, z1,
        normX1, normY1, normZ1
        u1, v1,

        # Tri 2
        # And so on...
    ]
)

To something like this:

# Tri 1
myArray.append(x1)
myArray.append(y1)
myArray.append(z1)

myArray.append(normX1)
myArray.append(normY1)
myArray.append(normZ1)

myArray.append(u1)
myArray.append(v1)

# Tri 2
# And so on...

Which isn’t terrible, but also isn’t as neat as the former, I feel.

[rdb]

@Thaumaturge

Save that I’m not actually placing the list into a named variable

That in itself doesn’t make a difference, for what it’s worth.

@Thaumaturge

As such, I presumably am getting hit for all 96 additions to the array, even so…

No, what you’re doing is better, since it knows ahead of time how many elements are going to be inserted. But at that point, it’s even better to use a tuple instead of a list, though it won’t matter a whole lot.

@Thaumaturge

So, just to clarify, when above you mention copying from a list to a memoryview, do you mean on an individual-element level–as in something like this: myMemoryView[i] = myList[i]?

I was mistaken and assumed that there would be a way to copy a whole range at once, but now that it turns out there isn’t, I’m not sure if this would be more efficient. I doubt it now.

@Thaumaturge

Which isn’t terrible, but also isn’t as neat as the former, I feel.

I don’t think that’d be more efficient. I think you can keep doing as you have been doing, initializing your data into the array using a list (you can try a tuple for a slight improvement) and then copying the array into the memoryview.

[Thaumaturge]

Aah, I see–okay, that’s encouraging then, thank you!

All right, then, I think that–aside from switching to a tuple, because either way it’s an easy change to make–I’m likely happy with my approach, then!

Given that, I’m thinking that it might be worth my writing a small program for the “Code Snippets” sub-forum, and maybe submitting a pull request to the manual, in order to aid others who might be looking to do something similar.

[edit]
Update: I’ve just tried switching from using a list to using a tuple–i.e. (effectively) from “myArray.fromlist([ ... ])” to “myArray.extend( ( ...) )”, because it seems that “fromlist” only accepts lists. And it appears that there isn’t a lot of difference in performance, at least for my purposes–but that, if anything, the list-based approach might be slightly faster!