I have a script that randomly places and orients a 3D model within a scene and then saves it to an image file. I am now trying to draw a 2D bounding box that appears to perfectly contain the model, wherever it may appear on the screen. I’ve read through this thread, but none of the solutions presented there seem to do quite what I’m looking for. I don’t think getTightBounds() is relevant at all given that with rotations in all three axes, a box that bounds the model in 2D can’t be extrapolated from these 3D bounding points.
I’ve thought about reading and projecting every single vertex of the model into 2D space and analyzing these coordinates for a min and max but that seems cumbersome (especially given the complexity of my model) and I would assume there is a simpler way to do this. Any pointers?
EDIT: Another, perhaps more simple question would be what function I should use to project a given point in 3-space to get it’s relative/pixel coordinates on the 2D screen plane. Is the project() function what I’m looking for? How does one implement it?
I believe what you are looking for is the equation of a projection, which you can find here:
To add to this, you only need to project the points that contain your model, not every single vertex. And this I believe should be easy as the size of an object can be calculated by this:
min, max = model.getTightBounds()
size = max-min
Then you can imagine it as the points of a cube that you rotate and position relative to where your mesh is located and oriented. At last, projecting the 8 vertices to the 2d space of your plane, you can then draw a rectangle that contains all 8 points (by comparing the distances to find the largest x and y distances).
Thanks for your reply. So there is no way to have Panda simply give me the projections, I have to redo the matrix math myself?
I’m aware of the the TightBounds classes, but drawing a box around the projections of those vertices will create a larger-than-necessary box in 2D. If there’s no better way to do it, this is likely close to what I’ll end up doing, I had just hoped for a cleaner method.
I do have a few other ideas that could be easier… But it depends on what you want.
If you just care about drawing a box and not having the coordinates, you can create a cube of LineSegs using the size of your model with TightBounds, where it is positioned appropriately, and then you apply a billboard effect to that cube object that surrounds your model. Because of how the billboard effect works, the orientation of the cube relative to the camera should not change making it appear as a box of 2 dimensions. However this cube will have depth, so after playing around with this, you can find which surface is projected at front by default and simply not draw any of the other 5 faces.
Now, as for the idea proposed in my earlier post, unlike the one I am describing now it should resize appropriately as you can imagine that in 3 dimensions orientation changes appropriately, and when converting to two dimensions the shape of this square you will end up drawing will change shape depending on not just where your model is positioned but how it is rotated relative to the camera (and of course its absolute size).
One more idea that I wouldn’t exactly recommend but may suit your needs is actually creating 4 vertical planes from the view of the camera (two horizontal and two vertical), shifting them towards the center of the screen until they collide with your object. Then using the 4 collision points it should be doable to draw a box that surrounds your object.
This is actually already used to create selection boxes as you can see from this post:
There may of course be an easier way of doing what you want, perhaps using existing API, however I am not exactly aware of anything like this, but since I started using Panda3D a bit more than a month ago, I am not very familiar with its API.
I’m pretty confident that the “project” method that you mentioned earlier does in fact do this. You give it a point in 3D space, and it fills a second point with the result.
Hmm… If the result of getTightBounds isn’t sufficiently accurate, and iterating through vertices is problematic, you could perhaps use a rendering approach: render to an offscreen buffer with your object (or objects) drawn in specific colours, and then search for the leftmost, topmost, rightmost, and bottommost pixels of the specified colour to get a (more or less=) pixel-accurate set of bounds.
Actually, there is a way straightforward way to do this in Panda; Panda allows you to pass an arbitrary node to the underlying function it uses to calculate bounds. It will then transform the vertices to that node’s coordinate space before calculating their bounds. So you can get the bounds in an arbitrary reference frame, by simply putting a desired transformation on a node and passing it as argument.
I’ll make some sample code and post it on this thread.
However, I’ve needed to make a change to the Panda3D source to allow this, since the bounds calculation currently assumes that only an affine transformation is used: (EDIT: the patch is no longer needed as of Panda3D 1.10.4+)
The way it works is that a node is created under the camera to hold the camera’s projection matrix. Then we simply ask Panda to get the bounds relative to that node. Here is the code, which will not work in Panda3D 1.10.3 without the above patch applied:
from panda3d.core import *
from direct.showbase.ShowBase import ShowBase
base = ShowBase()
base.trackball.node().set_pos(0, 100, 0)
model = loader.load_model("panda.egg")
model.reparent_to(base.render)
# Add a dummy node to the camera to hold the projection matrix
proj_dummy = base.cam.attach_new_node("proj-dummy")
# Set up a node in 2-D space to hold the drawn lines
line_node = GeomNode("lines")
line_path = base.render2d.attach_new_node(line_node)
def draw_box(task):
# Copy the projection matrix to the dummy
proj_mat = base.cam.node().get_lens().get_projection_mat_inv()
proj_dummy.set_transform(TransformState.make_mat(proj_mat))
# Calculate the box in projected (2D) space and draw lines in the 2D graph
min, max = model.get_tight_bounds(proj_dummy)
segs = LineSegs()
segs.move_to(min[0], 0, min[1])
segs.draw_to(min[0], 0, max[1])
segs.draw_to(max[0], 0, max[1])
segs.draw_to(max[0], 0, min[1])
segs.draw_to(min[0], 0, min[1])
line_node.remove_all_geoms()
segs.create(line_node)
return task.cont
base.taskMgr.add(draw_box, "draw-box")
base.run()
At this point, you have several options:
Compile Panda from source with the patch applied (or I can get you a development build)
Wait until I make a new 1.10.4 release with the patch included
Make your own code to iterate over the vertices and calculate the bounds yourself. If you go this route, I’m happy to let you know how you can let Panda do the matrix math for you.
I know that this has been solved already, but since we spoke of the Project() method earlier, for the sake of completion here is I believe an example of how one can use it (which would mean there is no need for applying the projection equation that I described in my first post).
Thanks for much for your help everyone! I would like to try to implement the solution that @rdb provided. Would you mind walking me through recompiling Panda to include that patch?
The color-seeking idea that @Thaumaturge came up with is super clever and maybe something I will look into if I can’t get the developer build up and running easily.
In the meantime, I will read through the thread that @Okal linked to see if there’s anything I can apply from there in the short term.
Okay, which platform are you on? If you are on Windows, unfortunately the buildbots are currently offline, so I don’t have an automated build. Building from source requires getting the latest check-out from GitHub (grab the release/1.10.x branch if you want a stable build), and following the instructions down on this page:
Awesome. I had to play with pip a little bit to get the rebuild to work but I got it and now I have your code sample running on my machine. Next step: implement it!
worked perfectly (y) Great work. I am saving this as a 2D picture with the bounding box. and Then convert the min,max values to the pixel values. but when I draw the Box from the pixel values using OPenCV the bounding box is flipped vertically.
