Method signatures requiring optional parameters

The Pythonic method_name signatures for some methods are not the same as the camelCase signatures with respect to optional parameters. I’m using the PyCharm IDE for developing with panda3d 1.10.9.
PyCharm is happy with method calls node.addChild(path) and node.setTag(‘picTag’, ‘me’), but produces missing parameter warnings for the Pythonic versions node.add_child(path) and node.set_tag(‘picTag’, ‘me’).
Similarly, for path.detachNode() vs path.detach_node(), and path.removeNode() vs path.remove_node().
I expected Filename.fromOsSpecific(‘file.txt’), and Filename.from_os_specific(‘file.txt’) to each be correct. However, Filename.fromOsSpecific and Filename.from_os_specific have different signatures. So, PyCharm produces different warnings for each.

The documentation defines NodePath.hide() and NodePath.show() as having no parameters, except for self. However, the imports define hide and show as e.g. def hide(self, const_NodePath_self: Any) → None:. So, the IDE produces missing parameter warnings.

For Intervals, E.g. LerpHprInterval and LerpPosInterval, the documentation indicates that the finish, loop, pause, and start methods have no required parameters. Yet, PyCharm notices that their parameters are all required, thus issues warnings.

E.g. for the following correct (though not useful) code, the IDE correctly notes a pile of “Parameter ‘<parameter_name>’ unfilled” Problems.

from direct.interval.LerpInterval import LerpHprInterval, LerpPosInterval
from panda3d.core import Filename, ModelNode, NodePath, Point3
node = ModelNode()
path = NodePath(node)
node.addChild(path)
node.add_child(path)
node.setTag('pickTag', 'me')
node.set_tag('pickTag', 'me')
path.removeNode()
path.remove_node()
path.detachNode()
path.detach_node()
path.hide()
path.show()
hpr = LerpHprInterval(path, 8.0, hpr=(8*360.0, 0.0, 0.0), startHpr=(0.0, 0.0, 0.0))
hpr.start()
hpr.pause()
hpr.loop()
hpr.finish()
pos = LerpPosInterval(path, 10, Point3(10, 10, 10), startPos=Point3(0, 0, 0))
pos.start()
pos.pause()
pos.loop()
pos.finish()
fn1 = Filename.fromOsSpecific('file.txt')
fn2 = Filename.from_os_specific('file.txt')

The Pythonic method signatures seem to often not be equivalent to the camelCase method signatures. Here’s a sampling from warning messages revealing methods for which the camelCase and Pythonic signatures are different:
NodePath.attach_new_node, get_relative_point, reparent_to, set_pos, set_scale
Lens.get_fov, Lens.get_hfov, Lens,get_vfov, Lens.get_lens.mat, Lens.get_near, Lens.get_far, Lens.get_aspect_ratio,
TextNode.calc_width
Camera.get_lens
CollisionNode.add_child, add_collider, add_solid, set_from_collide_mask, set_into_collide_mask
CollisionRay.set_direction, set_origin
CollisionHandlerQueue.sort_entries, get_entry, get_num_entries
AsyncFuture.set_result
HTTPClient.add_direct_host, set_verify_ssl, make_channel, set_connect_timeout, set_http_timeout, set_idle_timeout, set_username, base64_encode
HTTPChannel.get_status_code, is_connection_ready, is_valid, begin_get_document, open_read_body
StreamReader.extract_bytes, make_channel, send_extra_header
BamFile.open_write, get_writer
GeomVertexFormat.

Additional methods for which optional parameters are not optional in their signatures:
PSClient.connect()
panda3d.core.LineSegs.LineSegs.lines.create()
panda3d.core.TextNode.TextNode.generate()
NodePath.node()
Collision.traverse
HTTPChannel.run()
PMNIImage.read
Texture.load
BamFile.close()

The methods are identical, so that’s quite strange, and perhaps a bug in PyCharm.

As an example, the signatures that I see in panda3d.code.NodePath.NodePath are slightly different. removeNode() is what I expect from the documentation; remove_node() is quite different:

    def removeNode(self, *args, **kwargs): # real signature unknown
        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
        /**
         * Disconnects the referenced node from the scene graph.  This will also
         * delete the node if there are no other pointers to it.
         *
         * Normally, this should be called only when you are really done with the
         * node.  If you want to remove a node from the scene graph but keep it around
         * for later, you should probably use detach_node() instead.
         *
         * In practice, the only difference between remove_node() and detach_node() is
         * that remove_node() also resets the NodePath to empty, which will cause the
         * node to be deleted immediately if there are no other references.  On the
         * other hand, detach_node() leaves the NodePath referencing the node, which
         * will keep at least one reference to the node for as long as the NodePath
         * exists.
         */
        """
        pass

    def remove_node(self, const_NodePath_self, Thread_current_thread): # real signature unknown; restored from __doc__
        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
        /**
         * Disconnects the referenced node from the scene graph.  This will also
         * delete the node if there are no other pointers to it.
         *
         * Normally, this should be called only when you are really done with the
         * node.  If you want to remove a node from the scene graph but keep it around
         * for later, you should probably use detach_node() instead.
         *
         * In practice, the only difference between remove_node() and detach_node() is
         * that remove_node() also resets the NodePath to empty, which will cause the
         * node to be deleted immediately if there are no other references.  On the
         * other hand, detach_node() leaves the NodePath referencing the node, which
         * will keep at least one reference to the node for as long as the NodePath
         * exists.
         */
        """
        pass

The code that I see for remove_node() in

What’s probably happening is that PyCharm notices the remove_node signature in the docstring of the remove_node method and uses it to interpret the parameter name. But it won’t use it for removeNode since the is different there.

I’m not going to add two versions of the docstring because that’d be a waste, but I could perhaps list both camelCase and under_score methods in the same docstring, and it might pick up both.

And what is probably happening is that for things like hide() and show() there are multiple C++ overloads accepting a different number of parameters, and PyCharm is picking the wrong one.

It would be good to find out what PyCharm expects the method signatures to look like. Or maybe we should make the method signatures such that PyCharm doesn’t interpret them in any way.

In panda3d.core.NodePath.NodePath.py, the docstrings for removeNode and remove_node are nearly identical. What’s different is their signatures in the Python code:

    def removeNode(self, *args, **kwargs): # real signature unknown
        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
...
        """
        pass

    def remove_node(self, const_NodePath_self, Thread_current_thread): # real signature unknown; restored from __doc__
        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
        /**
         * Disconnects the referenced node from the scene graph.  This will also
         * delete the node if there are no other pointers to it.
         *
         * Normally, this should be called only when you are really done with the
         * node.  If you want to remove a node from the scene graph but keep it around
         * for later, you should probably use detach_node() instead.
         *
         * In practice, the only difference between remove_node() and detach_node() is
         * that remove_node() also resets the NodePath to empty, which will cause the
         * node to be deleted immediately if there are no other references.  On the
         * other hand, detach_node() leaves the NodePath referencing the node, which
         * will keep at least one reference to the node for as long as the NodePath
         * exists.
         */
        """
        pass

As would be expected from the above signatures, PyCharm is happy with nodepath.removeNode() because this matches removeNode’s “def removeNode(self, *args, **kwargs):” signature.

However, nodepath.remove_node() produces a warning simply because this does not match remove_node()'s “def remove_node(self, const_NodePath_self, Thread_current_thread):” signature.

PyCharm no longer complains if I replace remove_node’s signature line with removeNode’s signature line in the panda3d.core.NodePath.NodePath.py file.

Python methods defined in C have no notion of “signature”; PyCharm is guessing what the signature is based on the docstring. In the case of removeNode, it fails to guess what the signature is (and defaults to the most liberal possible signature, meaning it doesn’t really check arguments), and in the case of remove_node, it guesses incorrectly.

This isn’t really a Panda issue, but I am happy to consider how we can change the docstring to either make PyCharm’s guessing better or to make it consistently fail.

Yes, the actual methods written in C have expectations for the parameters they receive and return, but do not have a Python signature.

I’ve never extended or embedded Python with/into C, and have no knowledge of any automated pipeline that you might use to ensure compatibility between headers and macros in your C code and the Python modules that you provide for import into a Python program. I just see the results of pip installing panda3d into a local site-packages/panda3d directory as the file core.sp38-win_amd64.pyd

core.cp38-win_amd64.pyd is a Windows DLL. I’m not familiar with these, so the following is a SWAG. It includes code portions as well as header-like code sufficient to define the signatures. In this file, I see a section of header -like text that defines the interface for each class. Specifically, within the section for class NodePath, I see the following snippet for remove_node:

remove_node(const NodePath self, Thread current_thread)

/**
 * Disconnects the referenced node from the scene graph.  This will also
 * delete the node if there are no other pointers to it.
 *
 * Normally, this should be called only when you are really done with the
 * node.  If you want to remove a node from the scene graph but keep it around
 * for later, you should probably use detach_node() instead.
 *
 * In practice, the only difference between remove_node() and detach_node() is
 * that remove_node() also resets the NodePath to empty, which will cause the
 * node to be deleted immediately if there are no other references.  On the
 * other hand, detach_node() leaves the NodePath referencing the node, which
 * will keep at least one reference to the node for as long as the NodePath
 * exists.
 */      C++ Interface:

I also see the text “remove_node” interspersed in binary portions of the .pyd, E.g. the following which might actually be the source of the signature that it understands:

        NodePath.remove_node    Arguments must match:
remove_node(const NodePath self, Thread current_thread)

I see only one occurrence of the text “removeNode”, in a text portion containing pairs of names “… remove_node removeNode detach_node detachNode reverse_ls reverseLs get_net_state getNetState”. My SWAG is that this section is defining entry point aliases.

I.e. it appears that remove_node, and other pythonic names, have defined signatures in the .pyd file, but the equivalent camelCase names do not – only some type of aliasing to the pythonic names. The above C-like signature for remove_node is presumably correct from the C side of the interface, but is not the correct Python interface.

PyCharm parses and reverse engineers the .pyd into a Python signature for each class. E.g. it understands the above snippet from the NodePath class as:

    def remove_node(self, const_NodePath_self, Thread_current_thread): # real signature unknown; restored from __doc__

        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
        /**
         * Disconnects the referenced node from the scene graph.  This will also
         * delete the node if there are no other pointers to it.
         *
         * Normally, this should be called only when you are really done with the
         * node.  If you want to remove a node from the scene graph but keep it around
         * for later, you should probably use detach_node() instead.
         *
         * In practice, the only difference between remove_node() and detach_node() is
         * that remove_node() also resets the NodePath to empty, which will cause the
         * node to be deleted immediately if there are no other references.  On the
         * other hand, detach_node() leaves the NodePath referencing the node, which
         * will keep at least one reference to the node for as long as the NodePath
         * exists.
         */
        """
        pass

Although there is no explicit C-like header for removeNode, PyCharm apparently combines the entry-point aliasing to remove_node with the documentation for remove_node. Is cannot be sure about the interface for this name, so it supplies a fits-everything interface. I.e. the signatures for remove_node and removeNode are different because there is no signature for removeNode. This yields the following understanding as it expresses in Python:

    def removeNode(self, *args, **kwargs): # real signature unknown
        """
        C++ Interface:
        remove_node(const NodePath self, Thread current_thread)
        
        /**
         * Disconnects the referenced node from the scene graph.  This will also
         * delete the node if there are no other pointers to it.
         *
         * Normally, this should be called only when you are really done with the
         * node.  If you want to remove a node from the scene graph but keep it around
         * for later, you should probably use detach_node() instead.
         *
         * In practice, the only difference between remove_node() and detach_node() is
         * that remove_node() also resets the NodePath to empty, which will cause the
         * node to be deleted immediately if there are no other references.  On the
         * other hand, detach_node() leaves the NodePath referencing the node, which
         * will keep at least one reference to the node for as long as the NodePath
         * exists.
         */
        """
        pass

The PyCharm IDE uses the above signatures to support type and parameter checking, as well as providing the developer with a Python code equivalent to the interface and documentation included in the DLL.

In the case of methods like hide() and show(). The pyd. includes

hide(const NodePath self)
hide(const NodePath self, BitMask camera_mask)

/**
 * Makes the referenced node (and the entire subgraph below this node)
 * invisible to all cameras.  It remains part of the scene graph, its bounding
 * volume still contributes to its parent's bounding volume, and it will still
 * be involved in collision tests.
 *
 * To undo this, call show().
 */

/**
 * Makes the referenced node invisible just to the cameras whose camera_mask
 * shares the indicated bits.
 *
 * This will also hide any nodes below this node in the scene graph, including
 * those nodes for which show() has been called, but it will not hide
 * descendent nodes for which show_through() has been called.
 */             C++ Interface:

and

   NodePath.hide   hide() takes 1 or 2 arguments (%d given)

PyCharm understands the signature to be:

    def hide(self, const_NodePath_self): # real signature unknown; restored from __doc__
        """
        C++ Interface:
        hide(const NodePath self)
        hide(const NodePath self, BitMask camera_mask)
        
        /**
         * Makes the referenced node (and the entire subgraph below this node)
         * invisible to all cameras.  It remains part of the scene graph, its bounding
         * volume still contributes to its parent's bounding volume, and it will still
         * be involved in collision tests.
         *
         * To undo this, call show().
         */
        
        /**
         * Makes the referenced node invisible just to the cameras whose camera_mask
         * shares the indicated bits.
         *
         * This will also hide any nodes below this node in the scene graph, including
         * those nodes for which show() has been called, but it will not hide
         * descendent nodes for which show_through() has been called.
         */
        """
        pass

I.e. for remove_node and similar, PyCharm seems to be correctly understanding their (unfortunately incorrect) signatures in the pyd. However, for show and hide, PyCharm is not prepared to integrate the meaning of the multiple signatures and/or “hide() takes 1 or 2 arguments ($d given)”

There is no such thing as a “signature”, each method has code that looks at the arguments and figures out which C++ overload to call, or generates an error otherwise. The “C-like header” you’re looking at is the docstring, ie. NodePath.remove_node.__doc__, which is the same as NodePath.removeNode.__doc__. Panda’s Python binding generator extracts the signatures and comment blocks from the C++ source and puts them in the __doc__ field. (The other string in the DLL you saw is the “Arguments must match” error message, but that is impossible to extract without actually calling the method with wrong arguments.) So I’m certain that PyCharm is string-parsing remove_node.__doc__ and decided that there is something that looks like a signature, so it’s using it to determine the arguments. Only, it’s not very successful, because it thinks the argument name is const_NodePath_self, and has no idea that the second argument is optional.

Since the docstring only contains remove_node and not removeNode, PyCharm’s parsing logic fails to pick up the signature in the case of removeNode, so it gives up trying to determine the argument list and that’s why it’s not checking what you’re passing in.

Thanks.
I now see that:

  • NodePath.abc_def.__doc__ is identical to NodePath.abcDef.__doc__. My inspection of the binary DLL file found only one string for each pair, which is apparently referenced by both.
  • PyCharm tried to guess each signature from its docstring, but came up short. Apparently,
    • It rejected a docstring 1st line that didn’t match the method name as probably not a signature.
    • When it recognized the 1st line as probably a signature, it didn’t completely understand the C syntax.
    • There’s no general way to express overloaded signatures in Python. Trying to get value from the C signature in the 2nd line is pointless.

My objective was to eliminate the many lines of artifact “Problems”, to concentrate on the few real problems that the IDE identifies. One workaround is to use camelCase method names to get no checking.

I’m curious what the IDE would do with a real Python signature in the 1st line of the docstring, but distributing docstrings with Python signatures, including optional parameters, seems lot a lot of work.

thanks