By using taskMgr.add(update_environment) I am able to update the position of actors dynamically inside the function update_environment(). Additionally, every time I am inside update_environment(), I extract the image (as a numpy array) of the current window of the ShowBase environment (I basically get the same image I see in the ShowBase window). To do this I initialize a texture object
to obtain my image (I skipped some of the post-processing steps).
Now to the question: Is there a way to obtain the information about which pixels of the image a occupied by which actor. So additionally to the RGB image I would also like to get information about pixels occupied by actor1 and actor2. I need this to solve a classification task where every actor as well as the empty space belongs to a different class.
You could render the scene a second time, with the scenery rendered in black and the Actors rendered in different colours, solid and unshaded. The resulting image could then be queried, with pixel-colours indicating specific Actors.
What @Thaumaturge suggested could be accomplished by making use of the tag state key system.
Basically, a second camera would render the actors in a flat color using alternative render states associated with specific tags that are set on the actors.
When you move the camera around, you should see a smiley and a frowney model, textured and lit in the main display region, but with flat colors in a secondary, small display region at the bottom right hand corner of the window.
Instead of this secondary display region (which I only use to quickly show what is being rendered), you will likely need to set up an off-screen texture buffer for that second camera to render into.