CollisionRay not working Properly? (Picking 3D objects)

[Daniel_Sanchez]

Greetings,

I am getting really desperate to solve this issue and could really use some help. I have read several past solutions but I simply cannot get object selection to work.

Goal:

My goal is to be able to select or pick a 3D object using the mouse. As simple as it sounds, I have had no success implementing the recommended CollisionRay technique shown in:
https://docs.panda3d.org/1.10/python/programming/collision-detection/clicking-on-3d-objects .
Every time I click on any of the pickable objects no collision is registered.

I have also looked at similar problems like: Adding pickable objects during runtime , but I haven’t found a solution.

Here is the relevant code

        def cursorWatch(mouseWasDown,objcmlist,task):
            if base.mouseWatcherNode.hasMouse():
                if base.mouseWatcherNode.isButtonDown(MouseButton.one()):
                    if not mouseWasDown[0]:
                        mouseWasDown[0] = True

                        myTraverser = CollisionTraverser()
                        myHandler = CollisionHandlerQueue()

                        pickerNode = CollisionNode('mouseRay')
                        pickerNP = base.camera.attachNewNode(pickerNode)
                        pickerNode.setFromCollideMask(GeomNode.getDefaultCollideMask())
                        pickerRay = CollisionRay()
                        pickerNode.addSolid(pickerRay)
                        myTraverser.addCollider(pickerNP, myHandler)

                        mpos = base.mouseWatcherNode.getMouse()
                        pickerRay.setFromLens(base.camNode, mpos.getX(), mpos.getY())
                        oocnp = NodePath("collision nodepath")

                        for obstaclecm in objcmlist:
                            obstaclecm.copycdnpTo(oocnp)

                        myTraverser.traverse(oocnp)
                        # Assume for simplicity's sake that myHandler is a CollisionHandlerQueue.
                        if myHandler.getNumEntries() > 0:
                            print("Collision Registered")
                            # Do stuff...

                else:
                    mouseWasDown[0] = False
            return task.again

        taskMgr.doMethodLater(0.05, cursorWatch, "cursorWatch",
                              extraArgs=[self.mouseWasDown,self.objcmList],
                              appendTask=True)

In my code, I implemented a Task and task manager to monitor the state of the mouse so, whenever the user clicks we check if the mouse is over one of the objects on a list of pickable objects.

objcmlist is a python list filled with one or more CollisionModel objects. CollisionModel is a custom made class that lets me handle the collision nodes, nodepath and other necessary elements for loading a .stl file as a 3D object and handle possible collisions.

The method “copycdnpTo(oocnp)” basically lets me attach the associated object’s collision Node (self.__cdcn) to the oocnp NodePath as shown in the code:

    def copycdnpTo(self, nodepath):
        """
        return a nodepath including the cdcn,
        the returned nodepath is attached to the given one

        :param nodepath: parent np
        :return:
        """
        returnnp = nodepath.attachNewNode(copy.deepcopy(self.__cdcn))
        returnnp.setMat(self.__objnp.getMat())
        return returnnp

The object is surrounded by a collisionBox generated by the code:

    #self.__objnp is a nodePath associated to the loaded object
    self.__cdcn = self.__gencdboxcn(self.__objnp, radius = expand_radius)
    def __gencdboxcn(self, pandanp, name='boxcd', radius=15.0):
        """

        :param obstacle:
        :return:

        """

        cnp = CollisionNode(name)
        bottomLeft, topRight = pandanp.getTightBounds()
        center = (bottomLeft + topRight) / 2.0
        # enlarge the bounding box
        bottomLeft -= (bottomLeft - center).normalize() * radius
        topRight += (topRight - center).normalize() * radius
        cbn = CollisionBox(bottomLeft, topRight)
        cnp.addSolid(cbn)
        return cnp

Now, the collision box is correctly surrounding the objects as you can see in the following picture:

But every time I tried to click on the object boxes no collision is detected whatsoever. I started suspecting that the CollisionRay might be guilty so I decided to print out the Ray Data. So I added a print statement so every time I click on the screen the code does a print(pickerRay) and I got the following rays:

ray, o (-0.283897 1 0.119706), d (-14217.6 50080.2 5994.89)

ray, o (0.0566176 1 -0.125368), d (2835.42 50080.2 -6278.44)

ray, o (-0.317059 1 0.139118), d (-15878.4 50080.2 6967.04)

ray, o (0.129412 1 -0.118088), d (6480.97 50080.2 -5913.88)

…

Now, I am not exactly sure how the CollisionRay is supposed to work, but I find it really odd that the origin coordinates ALWAYS include a “1” in the, I assume Y-axis, If I understand correctly the CollisionRay is an infinite straight line that connects points “o” and “d” right? however, I tried showing that line in my worldbase and it just doesnt make sense:

Do you have any idea what could be the problem? Did I explain myself correctly?

Thanks for your time and reading this,

Hope you can help me.

[Thaumaturge]

Hmm… I’m not sure.

This is a long shot, but what happens if you change the following:
pickerNP = base.camera.attachNewNode(pickerNode)
to this:
pickerNP = base.cam.attachNewNode(pickerNode)

@Daniel_Sanchez

The method “copycdnpTo(oocnp)” basically lets me attach the associated object’s collision Node (self.__cdcn) to the oocnp NodePath as shown in the code:
…

Hmm… I don’t know whether setting the matrix like this will take into account any parent transformations. If the original object is a child of a NodePath that has a transformation, perhaps your copied collision-objects aren’t ending up where expected.

What happens if, instead of copying your colliders, you have your collision-traverser just traverse render? (Or whatever root-node your scene uses.)

Something like this:

                        mpos = base.mouseWatcherNode.getMouse()
                        pickerRay.setFromLens(base.camNode, mpos.getX(), mpos.getY())

                        myTraverser.traverse(render)
                        # Assume for simplicity's sake that myHandler is a CollisionHandlerQueue.
                        if myHandler.getNumEntries() > 0:
                            print("Collision Registered")

@Daniel_Sanchez

If I understand correctly the CollisionRay is an infinite straight line that connects points “o” and “d” right?

I think that “d” is a direction, not a point through which the ray passes. That is, the ray starts at point “o”, and extends in direction “d”. I may be mistaken, however!

As to that origin-point, where is your camera located? I would expect the origin to correspond to position of the camera, since you’re attaching it to the camera and setting it to extend along the camera’s view.

By the way, I notice that you’re creating your ray, traverser, and queue every time that your “cursorWatch” task reaches the relevant piece of code–potentially once per frame under certain conditions. It might be more efficient to create those objects once, and then simply use them in the task.

[Daniel_Sanchez]

Hey, thanks for the quick reply.

A combination of your recommendations actually solved the problem! Thank you very much! It was driving mad.

I changed the line to:

pickerNP = base.cam.attachNewNode(pickerNode)

And I changed the traverser to traverse render

myTraverser.traverse(base.render)

I had tried before to traverse render without success, I also tried to first change the line to pickerNP = base.cam.attachNewNode(pickerNode) without success. But doing both things seems to make the trick. May I ask what is the difference between “cam” and “camera”? I am also not sure why traversing oocnp does not register a collision; if I understood correctly, the traverser should just check for collisions with oocnp children? With the current solution it registers collisions with all elements on the world and not just my list of elements ( Though this is not a big problem, and I will make use of Tags to differentiate objects that I want to pick from other static objects, hope it works.)

Also, just for reference if somebody else has this problem I will answer the question:

@Thaumaturge

Hmm… I don’t know whether setting the matrix like this will take into account any parent transformations. If the original object is a child of a NodePath that has a transformation, perhaps your copied collision-objects aren’t ending up where expected.

I am pretty sure the collision objects are placed in the correct position, the nodes only parent is render and they dont get modified by other things, but thanks for pointing it out.

@Thaumaturge

think that “d” is a direction, not a point through which the ray passes. That is, the ray starts at point “o”, and extends in direction “d”. I may be mistaken, however!

As to that origin-point, where is your camera located? I would expect the origin to correspond to position of the camera, since you’re attaching it to the camera and setting it to extend along the camera’s view.

I see, thanks for the info. Do you know if it is possible to get a line function off of the Ray?

I actually re-checked my code and I think the problem had to do with me using cam and not camera when setting up the “base” class. I figured pickerNP = base.camera.attachNewNode(pickerNode) was the ONLY option.

@Thaumaturge

By the way, I notice that you’re creating your ray, traverser, and queue every time that your “cursorWatch” task reaches the relevant piece of code–potentially once per frame under certain conditions. It might be more efficient to create those objects once, and then simply use them in the task.

Thank you, you are very kind, I had actually moved the code inside of the Task to ease my life trying to find the cause of my problem. I will move it out, as suggested.

Again, thanks for your help!

[Thaumaturge]

@Daniel_Sanchez

Hey, thanks for the quick reply.

It’s my pleasure, and I’m glad that you found a solution to the issue!

@Daniel_Sanchez

May I ask what is the difference between “cam” and “camera”?

I wasn’t sure myself, to be honest, but a quick forum-search turned up an answer, starting in the third paragraph of the below-linked post:

Difference between base.cam & base.camera

I don’t know about your frame rate change; it’s likely just changing because you’re changing the view and seeing different objects. If it’s dropping from 60 to 30, you probably have video sync on, which makes it difficult to come to meaningful conclusions about frame rate. Turn it off with: sync-video 0 in your Config.prc file, and you’ll probably see the frame rate drop from 65 to 55 instead or something more reasonable. As to the difference between base.cam and base.camera, they exist as a…

@Daniel_Sanchez

I am also not sure why traversing oocnp does not register a collision; if I understood correctly, the traverser should just check for collisions with oocnp children?

I still suspect that there’s some problem with the way that you were copying those elements, something that was resulting in the collision objects either not being at their intended locations, or not having their intended shapes. Perhaps it was the matrix-assignment that I pointed out earlier. Perhaps collision-objects don’t respond well to “deepcopy”. Perhaps it’s something else. I’m not sure, I’m afraid.

Ah, wait! I think that I may see the problem: Your ray wasn’t a child (directly or indirectly) of “oocnp”, and nor did it have a common parent with “oocnp”. In short, the ray and the collision-objects were on separate scene-graphs, and thus didn’t interact.

@Daniel_Sanchez

With the current solution it registers collisions with all elements on the world and not just my list of elements

Part of this, I think, is that you’re calling “pickerNode.setFromCollideMask(GeomNode.getDefaultCollideMask())”. That line causes the ray to collide with anything that has GeomNode’s default collide-mask–which is to say, any (unmodified) GeomNode. It thus causes the ray to collide with things like visible geometry.

If you remove that line, your ray should collide only with collision objects, which may fit better with what you seem to be doing.

And if you want to collide with only a sub-set of your collision-objects, perhaps more reliable than "deepcopy"ing would be to temporarily reparent your objects to a separate NodePath–one that’s nevertheless attached to render, directly or indirectly–using “wrtReparentTo” in order to preserve the relative transformations of the objects. Once the traversal has been done, you could then use wrtReparentTo in order to transfer the objects back to their proper parentage.

@Daniel_Sanchez

Do you know if it is possible to get a line function off of the Ray?

Hm… I don’t know offhand, I’m afraid. You could dig through the API, however!

[Daniel_Sanchez]

Awesome!

Thanks for the explanation and suggestions! I will try to implement them between today and tomorrow. Would it be ok to keep this thread alive a little longer if I want to ask questions regarding the “wrtReparentTo” part or should I start a new thread if I have other problems? (I will obviously check for possible solutions on this forum and google beforehand)

I will also check the API.

Again, thank you very much!

[Thaumaturge]

I think that for a new problem, a new thread might be better.

It means that another user looking for an answer to a similar problem is more likely to find any answers given, as they’re not hidden in a thread that seems unrelated.

And again, the help is my pleasure.

[Daniel_Sanchez]

You are right.

Thank you, I wish you a nice day!

[Thaumaturge]

I hope for a pleasant day for you, too!

[Epihaius]

@Daniel_Sanchez

Now, I am not exactly sure how the CollisionRay is supposed to work, but I find it really odd that the origin coordinates ALWAYS include a “1” in the, I assume Y-axis, If I understand correctly the CollisionRay is an infinite straight line that connects points “o” and “d” right?

Due to the call to set_from_lens, the origin point o is placed on the near plane of the lens. Therefore, those coordinates are given in camera space, with the y-coordinate always equal to the near distance .

[Daniel_Sanchez]

I see, I assumed it would be world coordinates. That explains a lot.

Thanks for the explanation!

[www20602]

I guess you can use the bulletRayHit method in envrionment.bulletcdhelper.

def bulletRayHit(pfrom, pto, geom):
    """
    find the nearest collision point between vec(pto-pfrom) and the mesh of nodepath
    :param pfrom: starting point of the ray, Point3
    :param pto: ending point of the ray, Point3
    :param geom: meshmodel, a panda3d datatype
    :return: None or Point3
    """
    bulletworld = BulletWorld()
    facetmesh = BulletTriangleMesh()
    facetmesh.addGeom(geom)
    facetmeshnode = BulletRigidBodyNode('facet')
    bullettmshape = BulletTriangleMeshShape(facetmesh, dynamic=True)
    bullettmshape.setMargin(1e-6)
    facetmeshnode.addShape(bullettmshape)
    bulletworld.attach(facetmeshnode)
    result = bulletworld.rayTestClosest(pfrom, pto)
    if result.hasHit():
        return result.getHitPos()
    else:
        return None

[itsafeature]

Let me add to this since this thread keeps popping up on google.

@Epihaius

Due to the call to set_from_lens, the origin point o is placed on the near plane of the lens. Therefore, those coordinates are given in camera space, with the y-coordinate always equal to the near distance .

Like Epihaius said, the ray’s origin is given relative to the camera. But the direction vector is given relative as well! So the whole ray is given relative to the camera. This is because set_from_lens is supposed to be called with the ray as child to the lens object it is called with! Always make sure to set your ray as a child to the camera if you plan on using this function.

Now to your ray visualization problem: Actually, you can always just call ray_node_path.show() on the NodePath to draw a ray automatically. However, this will ONLY work if your ray is NOT child of the camera you are using to look at the scene. If the ray IS child of your scene camera, it will always be culled, since it sits directly in the near plane. Also, you can’t see it from the side when you pan around since it will move with the camera!

To also visualize rays that are children of the scene camera, use this code:

class Application(ShowBase):

    ....

    def visualize_ray(self, ray: CollisionRay, ray_node_path: NodePath):
        # Uncomment this if there should only be one line at a time
        # if self.ray_visual:
        #     self.ray_visual.remove_node()
        #     self.ray_visual = None
        lines = LineSegs()
        lines.set_color(1, 0, 1, 1) # pink
        lines.set_thickness(3)
        # get the ray position relative to root (= render)
        lines.move_to(ray_node_path.get_pos(self.render))
        # to the same point as before, add the ray direction relative to the root (times 10 for longer line, usually long enough)
        lines.draw_to(ray_node_path.get_pos(self.render) + self.render.getRelativeVector(ray_node_path, ray.get_direction() * 10))
        node = lines.create()
        self.ray_visual = self.render.attach_new_node(node)

This will programatically generate a line at the right coordinates in world space and parent it to the render root, so you can move the camera and see it.
We can also let panda3d do all the heavy lifting and use its built in method for reparenting while keeping world coordinates the same:

def visualize_ray2(self, ray: CollisionRay):
    lines = LineSegs()
    # now we use relative coordinates
    lines.set_color(1, 0, 1, 1)
    lines.draw_to(ray.getOrigin() + ray.getDirection())
    ray_visual = self.cam.attach_new_node(lines.create())
    ray_visual.wrtReparentTo(self.render) # this reparents while keeping the original transform consistent

And finally, a working example application with visualized rays, where rays that hit the cube are drawn yellow, rays that dont hit are drawn pink. Rays are casted for picking on left click. You may need to rotate the scene to see the rays, the camera controls are default ones from panda3d.

EXAMPLE IMAGE

EXAMPLE APP

from direct.showbase.ShowBase import ShowBase
from panda3d.core import NodePath, CollisionTraverser, CollisionHandlerQueue, CollisionNode, BitMask32, CollisionRay, \
    LineSegs


class Application(ShowBase):
    def __init__(self):
        super().__init__()
        self.ray_visual: NodePath = None
        self.pickline: NodePath  = None

        # all objects that should collide with the picker ray (as well as the ray itself) must have this mask
        my_collision_mask = BitMask32(0x10)
        # with tags, you can filter amongst the collided objects and also easily get higher nodes, if you only
        # match a child node (for example some geometry) during collision
        # Node [tag = "collision_tag"]
        # |--Node
        # |--Node  <- suppose this got returned by collision, but we can get the parent as next highest tagged node
        self.my_collision_tag = "collision_tag"


        # ---- collision setup ----
        picker_node = CollisionNode('mouseRay')
        picker_node.setFromCollideMask(my_collision_mask)
        self.picker_ray = CollisionRay()
        picker_node.add_solid(self.picker_ray)
        # IMPORTANT!!! only works when ray is child of cam
        self.picker_np = self.cam.attach_new_node(picker_node)
        self.picker_np.show() # will have no effect since the ray is child of the camera

        self.picker = CollisionTraverser()
        self.picker_queue = CollisionHandlerQueue()
        self.picker.add_collider(self.picker_np, self.picker_queue)

        # ---- cube to check collision against ----
        cube = self.loader.loadModel("models/box")
        cube.reparentTo(self.render)
        # IMPORTANT!!! else it wont match because masks dont match
        cube.setCollideMask(my_collision_mask)
        # IMPORTANT!!! set collision tag if you want to differentiate between collided objects and especially if you
        # match pure geometry, so you can get the parent node you wnat
        cube.set_tag(self.my_collision_tag, "")

        # Event bindings
        self.accept("mouse1", self.on_left_click)


    def visualize_ray(self, ray: CollisionRay, ray_node_path: NodePath, color=(1, 0, 1, 1)):
        # Add this if there should only be one line at a time
        # if self.ray_visual:
        #     self.ray_visual.remove_node()
        #     self.ray_visual = None
        lines = LineSegs()
        lines.set_color(color) # default is pink
        lines.set_thickness(3)
        # get the ray position relative to root (= render)
        lines.move_to(ray_node_path.get_pos(self.render))
        # to the same point as before, add the ray direction relative to the root (times 10 for longer line, usually long enough)
        lines.draw_to(ray_node_path.get_pos(self.render) + self.render.getRelativeVector(ray_node_path, ray.get_direction() * 10))
        node = lines.create()
        self.ray_visual = self.render.attach_new_node(node)

    def visualize_ray2(self, ray: CollisionRay):
        lines = LineSegs()
        # now we use relative coordinates
        lines.set_color(1, 0, 1, 1)
        lines.draw_to(ray.getOrigin() + ray.getDirection())
        ray_visual = self.cam.attach_new_node(lines.create())
        ray_visual.wrtReparentTo(self.render) # this reparents while keeping the original transform consistent


    def on_left_click(self):
        print("left-click")

        if self.mouseWatcherNode.hasMouse():  # check that mouse is not outside screen.
            mouse_pos = self.mouseWatcherNode.getMouse()

        # Start ray from camera and point to the screen coordinates of the mouse.
        # resulting ray only works RELATIVE to the camera!
        self.picker_ray.set_from_lens(self.camNode, mouse_pos.getX(), mouse_pos.getY())

        picked_obj = None
        self.picker.traverse(self.render)
        if self.picker_queue.get_num_entries() == 0:
            print("No collision.")
        else:
            self.picker_queue.sort_entries()
            picked_obj = self.picker_queue.get_entry(0).get_into_node_path().find_net_tag(self.my_collision_tag)
            if not picked_obj.is_empty():
                print(f"Picked: {picked_obj}")
            else:
                print("Nothing with required tag found")

        if picked_obj:
            self.visualize_ray(self.picker_ray, self.picker_np, (1, 1, 0, 1))  # make a yellow ray because we picked sth
        else:
            self.visualize_ray(self.picker_ray, self.picker_np)  # default ray visualization



if __name__ == '__main__':
    app = Application()
    app.run()