Improve performance of molecular editor with many atoms (meshes)

There is no error, it runs well but slow. I think the problem is there are too many Geom objects (I’m just guessing because I don’t know the inside of the Panda3D).

I don’t think Python is the bottleneck. There are two pieces of evidence.

  1. My OpenGL program (fast run with 50,000 meshes) is also written in Python with PyOpenGL.
  2. flattenStrong() also gives a high frame rate.

It’s just a guess, but try turning off the shadow caster. Also, thanks for the interesting post!

# Add shadow caster.
directionalLight.setShadowCaster(True, 1024, 1024)
directionalLight.getLens().setFilmSize(1000, 1000)
directionalLight.getLens().setNearFar(-500, 500)
#directionalLight.showFrustum()

1 Like

Perhaps you’d better have your own lightweight class for storing states. Then you can send this data to the shader. In panda, classes suffer from redundant logic.

1 Like

Hi, welcome to the community! :slight_smile:

If you are comfortable with using a development build from the master branch of Panda3D, it includes support for automatic hardware instancing via the InstancedNode class. It does require a custom shader, but I’m happy to provide additional instructions.

An alternative, far more efficient method would be to just fake it. Instead of thousands of spheres, you would use a single Geom and GeomPoints with many points and configure them to render as point sprites. Using a shader, you could shade them as though they were spheres by calculating the normal vector for lighting based on the distance to the center of the card. This method may allow you to have hundreds of thousands of “spheres” with very low performance cost.

I am curious, though, about the specific method you used to implement it in OpenGL, and I may be able to advise you how to do the same thing with Panda3D.

Unfortunately, the shadow caster or filters did not affect performance. Thanks!

I think it works for me because I’m not use textures (or different RenderState) for the molecular structure.
I tested your code at

(but with tens of thousands of cubes) and I could see it working well.
I think based on this code, I need to develop a class that can handle shapes such as spheres and capsules easily (for the component of molecules).

Thank you for the advanced recommendation!

Hi rdb, nice to see you.

I think all your suggestions can be applied to my problem.
But I’m not an expert, so I don’t think it’s a good idea to use low-level API from the beginning (error-prone and hard to extend). For example, if I add a capsule shape (for bonds in a molecule), I will probably have to modify the shader.
I would like to use the excellent basic functions provided by Panda3D as possible.

My OpenGL program is very primitive. You can guess what I did with the GLSL code below. v_offset is the atomic positions.

#version 330 core

layout(location = 0) in vec3 v_position;
layout(location = 1) in vec3 v_uv;
layout(location = 2) in vec3 v_normal;
layout(location = 3) in vec3 v_offset;

out struct F {
    vec3 normal;
    vec3 pos_world;
} f;

uniform mat4 PVM;
uniform mat4 model;

void main()
{
    gl_Position = PVM * vec4(v_position+v_offset, 1.0);
    f.pos_world = vec3(model * vec4(v_position, 1.0)); 
    f.normal = v_normal;
}
glDrawElementsInstanced(
    GL_TRIANGLES,
    len(self.indices),
    GL_UNSIGNED_INT,
    None,
    len(self.offsets),
)

Thank you for your kind reply!

I wrote a new code that is based on your recommendation. Now I can draw more than 100,000 spheres without frame drop. Thank you!

2 Likes

Ah, that’s wonderful! Glad I was able to help :slight_smile: .

Can you send the code snippet?

Sure, here is the full script.

import numpy as np

from panda3d.core import *
from direct.filter.CommonFilters import CommonFilters
from direct.showbase.ShowBase import ShowBase

N_SPHERES = 15000

load_prc_file_data("", """
    win-size 1600 900
    window-title High performance spheres
    framebuffer-multisample 1
    multisamples 4
""")

base = ShowBase()
render = base.render
render.setAntialias(AntialiasAttrib.MMultisample)

# Original code from http://www.songho.ca/opengl/gl_sphere.html.
def make_sphere(stack_counts, sector_counts, radius=1):
    sector_step = 2 * np.pi / sector_counts
    stack_step = np.pi / stack_counts

    vertices = []
    normals = []
    tex_coords = []
    for i in range(stack_counts + 1):
        stack_angle = np.pi / 2 - i * stack_step
        xy = radius * np.cos(stack_angle)
        z = radius * np.sin(stack_angle)

        for j in range(sector_counts + 1):
            sector_angle = j * sector_step

            x = xy * np.cos(sector_angle)
            y = xy * np.sin(sector_angle)

            vertices += [x, y, z]
            normals += [x/radius, y/radius, z/radius]

            tex_coords += [j / sector_counts, i / stack_counts]

    # Make indices.
    indices = []
    for i in range(stack_counts):
        k1 = i * (sector_counts + 1)
        k2 = k1 + sector_counts + 1
        for j in range(sector_counts):
            if i != 0:
                indices += [k1, k2, k1 + 1]
            if i != (stack_counts - 1):
                indices += [k1 + 1, k2, k2 + 1]
            k1 += 1
            k2 += 1

    vertices_data = np.array(vertices, dtype=np.float32).reshape(-1, 3)
    normals_data = np.array(normals, dtype=np.float32).reshape(-1, 3)
    uvs_data = np.array(tex_coords, dtype=np.float32).reshape(-1, 2)
    indices = np.array(indices, dtype=np.uint32)

    data = {
        "vertices": vertices_data,
        "normals": normals_data,
        "texcoords": uvs_data,
        "indices": indices,
    }

    return data

# Generate custom sphere.
data = make_sphere(stack_counts=20, sector_counts=20, radius=1)

array_format = GeomVertexArrayFormat()

array_format.addColumn("vertex", 3, Geom.NTFloat32, Geom.CPoint)
array_format.addColumn("normal", 3, Geom.NTFloat32, Geom.CNormal)
array_format.addColumn("texcoord", 2, Geom.NTFloat32, Geom.CTexcoord)
array_format.addColumn("color", 4, Geom.NTUint8, Geom.CColor)

vertex_format = GeomVertexFormat(array_format)
vertex_format = GeomVertexFormat.registerFormat(vertex_format)

vertex_data = GeomVertexData("sphere", vertex_format, Geom.UHStatic)
n_verts = len(data["vertices"])
n_rows = N_SPHERES * n_verts

vertex_data.uncleanSetNumRows(n_rows)
data_array = vertex_data.modify_array(0)
memview = memoryview(data_array).cast("B").cast("f")

# Set indices for primitive.
n_inds = len(data["indices"])
primitive = GeomTriangles(Geom.UHStatic)
primitive.setIndexType(Geom.NTUint32)
tris_array = primitive.modify_vertices()
tris_array.uncleanSetNumRows(N_SPHERES * n_inds)
prim_memview = memoryview(tris_array).cast("B").cast("I")

# 3 + 3 + 2 + 1.
strides = 9

for i in range(N_SPHERES):
    color = [
        np.random.randint(0, 255),
        np.random.randint(0, 255),
        np.random.randint(0, 255),
        255
    ]

    # Convert 4 np.uint8 values to 1 np.float32 value.
    color_data = np.tile(
        np.array(color, dtype=np.uint8),
        reps=(n_verts, 1),
    ).view(np.float32)

    scale = np.random.uniform(0.1, 1)
    pos = np.random.uniform(-10, 10, size=(1, 3)).astype(np.float32)

    all_data = np.concatenate([
            scale*data["vertices"]+pos,
            data["normals"],
            data["texcoords"],
            color_data
        ],
        axis=1,
    ).copy()

    data_memview = memoryview(all_data).cast("B").cast("f")
    memview[i*n_verts*strides : (i+1)*n_verts*strides] = data_memview

    # Apply some transformation.
    #scale_mat = Mat4.scaleMat(1, 0.7, 2.0)
    #vertex_data.transformVertices(scale_mat)

    # Set indices for primitive.
    idx = (data["indices"] + i*n_verts).astype(np.uint32)
    data_memview = memoryview(idx).cast("B").cast("I")
    prim_memview[i*n_inds : (i+1)*n_inds] = data_memview

geom = Geom(vertex_data)
geom.addPrimitive(primitive)

node = GeomNode("gnode")
node.addGeom(geom)

node_path = render.attachNewNode(node)
node_path.setPos(0, -100, 100)
node_path.setScale(100)

myMaterial = Material()
myMaterial.setShininess(125.0)
myMaterial.setSpecular((0.8, 0.8, 0.8, 1))
node_path.setMaterial(myMaterial)

# Position the camera.
base.trackball.node().setPos(0, 4000, -100)
base.camLens.setFov(70)

# Create Ambient Light
ambientLight = AmbientLight('ambientLight')
ambientLight.setColor((0.7, 0.7, 0.7, 1))
ambientLightNP = render.attachNewNode(ambientLight)
render.setLight(ambientLightNP)

# Directional light 02
directionalLight = DirectionalLight('directionalLight')
directionalLight.setColor((1, 1, 1, 1))

directionalLightNP = render.attachNewNode(directionalLight)
# This light is facing forwards, away from the camera.
directionalLightNP.setHpr(50, -30, 50)
render.setLight(directionalLightNP)

# Add shadow caster.
directionalLight.setShadowCaster(True, 1024, 1024)
directionalLight.getLens().setFilmSize(1000, 1000)
directionalLight.getLens().setNearFar(-500, 500)

render.setShaderAuto()

# Apply SSAO.
base.camLens.setNearFar(100, -500)
filters = CommonFilters(base.win, base.cam)
filters.setAmbientOcclusion(numsamples=16, strength=0.01, radius=0.015, amount=1)

base.run()
1 Like

When i try to run the code I get the following error

Traceback (most recent call last):
  File "spheres.py", line 90, in <module>
    memview = memoryview(data_array).cast("B").cast("f")
AttributeError: 'memoryview' object has no attribute 'cast'

i am using panda3d 1.10.6
and python 3.8.5

I’m not sure what is the problem. The official python document shows that the memoryview object has member function cast().

https://docs.python.org/3.8/library/stdtypes.html?highlight=memoryview#memoryview

Are you sure that you’re running Python 3.8? It looks like older versions of Python may not have that method, and I wonder whether you’re not perhaps inadvertently using such a version.

To test this, what output is printed if you add the following to the script and run it?

import sys
print (sys.version)

the problem is… i’m an idiot.
i do this all the time. i ran it with python on the command line and not python3

then when i check my version of python, i run python3. lol.

it works ! sorry for the noise.

i really have to find a way to python default to python3. i guess i should set up an alias.

1 Like

Did you install python from its default page? If you did, you will get an application called IDLE. You can code in it and it runs the code according to the version you have installed

i’m running on linux.
it’s not hard, you type “python” for python2 and “python3” for python3.

i constantly leave off the 3…

the other way to handle this is put

#!/usr/bin/python3

as the first line of the file and make the script executable. However at this point i need python to default to python3 because i very rarely use python2 any more.

You can make a program in python called executor.py.
The code will be:

import os

pyfile = "path/to/file.py"
os.system("python3 " + pyfile)

Then, you can run this with python because os was supported in older versions.

You can also do:

import sys
import os

os.system("python3 " + sys.argv[1])

And in the command line, type: python executor.py path/to/file.py

Thanks for the code! Unfortunately, The screen is loading but the atoms aren’t

I find PyCharm to work rather well, myself–if you want an IDE, I can recommend that one, I believe.

Honestly, it’s a pretty easy mistake to make: it’s easy to forget that extra “3” when “python” alone is the intuitive command to enter, I daresay, and furthermore one that would have been appropriate if you previously used it with 2.7.