I’ve heard some good things about doxygen to generate documentation, so I’ve ran some tests - it’s pretty nice. Nice thing is, it parses Panda’s Python code flawlessly without having to actually import the interfaces (which is a terrible nuisance.) However, it’s very strict about C++ interfaces. It requires you to put docstrings in the header files, in a specific format, and most importantly, it didn’t chew a lot of Panda3D’s macros, especially it choked on PointerTo.
So I ended up writing a small tool that imported Panda’s C++ modules, looped through it’s interfaces, and used the interrogate docstring to grab the function prototype and doc. Then, it outputted all the function prototypes and their docstrings into a huge hpp file.
Then, I ran doxygen and fed it with all panda’s *.py and the new hpp file.
Here are two snapshots of the result. pro-rsoft.com/screens/doxygen-test-PandaNode.png (C++ class) pro-rsoft.com/screens/doxygen-te … edNode.png (Python class)
Crazy, huh? But it works!
What do you guys think? Is this idea worth continuing, should we ditch the current documentation in favor of doxygen?
I especially like the Python handling; it looks like they’ve spent a lot of work getting that right. The C++ handling seems only a little better than what we’ve already got, but still it’s better.
Well, the advantage here is that we have more control over the C++ side by using dict and doc rather than interrogatedb. Plus, overloaded methods will be easily possible as well.
Yeah, it certainly will, that’s the main point of it. As for the layout, I could easily adjust doxygen to match the current API reference layout.
The current doc system is kinda in embarrassing shape. While I like the class/method/function layout, the fact that it dups the comments for overloaded methods takes some getting used to.
if that would make it possible to learn about getPos(node) and setPos(node,pos) and those other overloaded methods that I had no idea existed, it would be sweet.
Hm, seems that interrogate lacks two things for this to work well:
It doesn’t put the return value of the function in the docstring.
It doesn’t show the documentation per prototype, but instead it shows all the overloads, and then all docstrings. So it’s very tricky to extract the doc on a per-prototype basis.
David, do you think these two things could be fixed easily?
Actually, those two points are issues only with the deprecated interrogate_function_comment() interface. The new interface is interrogate_wrapper_comment(), which will return the docstring for each overload, instead of all overloads bundled together in one big string.
For the return value, and indeed for the complete prototype, you should use interrogate_wrapper_return_type() and interrogate_wrapper_parameter_type()/interrogate_wrapper_parameter_name(). Use interrogate_type_scoped_name() to translate a given type to a printable string, and thus construct the Python-friendly function prototype string.
I think you’ve misunderstood me - I’m not using the interrogatedb interface, but I’m reading out the doc values from the classes directly, these are made by InterfaceMakerPythonNative.
Do you think I should be using the interrogatedb interface instead? Does it really support everything we’ll need?
(Also, makepanda currently doesn’t expose it - should we expose interrogate_interface to Python or should I just use ctypes (just as easy) ?
Oh, hmm. If you’re not already using the interrogatedb interface, it might be more trouble than it’s worth to switch to using it. But maybe not–it just means you need have available the .in files that interrogate generates. If you do have those files, then it is as simple as calling the appropriate C-style functions given in interrogate_interface.h, and ctypes is a fine way to do that.
The biggest advantage to using the interrogatedb interface is that it can give you a lot more specific information than what gets shoved into the docstring.
On the other hand, if it turns out to be a nuisance to use that interface, I think it probably won’t be too hard to fix the generated docstring to alternate comments and prototypes, and to include the return value. It probably should do that anyway.
Interfacing to interrogate_interface via ctypes is probably a much cleaner solution than parsing doc. But don’t we then end up with the same problems that we used to have? The old gendocs.py parses the .in files directly, and it had quite some limitations (remember this thread?), or doesn’t that apply to interrogate_interface?
EDIT: Oh, I just found this thread. I guess it’ll work to use that then. Would it make sense to expose these interfaces to Python, or should I just stick to ctypes (just as easy)?
Hmm, we used to have this file itself be automatically processed with a special interrogate command line, and thus exposed to Python. For instance, you can see the (legacy) code in direct/src/ffi/FFIInterrogateDatabase.py making direct calls to this module. I’ll see if I can figure out what happened to that.
If it’s too much effort, I can use ctypes as well (pretty straightforward as it’s a C interface and there are no complex types/structures), or just write the documentation tool in C(++).
Well, I didn’t find the old file, but it was easy to generate a new file. I just used interrogate on itself, and wrote the file interrogate_wrapped.cxx, which if added to the source list should expose all of the interrogate_interface.h functions when libdtoolconfig.dll is imported as a module. I figure there’s no harm in committing this generated file back to the source, since interrogate_interface.h changes so rarely (no need to instrument that file at each build; we’ll just re-create and re-commit that one file if we ever need it to change).
To use it, you have to use interrogate_add_search_directory() to add the path(s) to the .in files, and then you have to actually import the code you want to instrument, for instance by importing libpandaexpress and so on. Then you can start by interrogate_number_of_global_types() / interrogate_get_global_type(n), and work from there.
Oh, son of a gun. I just found the old code when I compiled in Windows and got a multiply-defined symbol. It’s already there, has been all along, in pydtool.cxx; you should have it already when you import libdtoolconfig.
Ah, thanks!
Importing libp3dtoolconfig fails because there is no init function for that module. Makepanda does compile pydtool into libp3dtoolconfig, though. Should I let makepanda link it into libpandaexpress instead, or is there some kind of file that contains the python wrappers for the libp3dtoolconfig module?
Hmm, that darn dll renaming. Python requires the dll name to be part of the init function. This module is initialized correctly for me, in dtool/metalibs/dtoolconfig/pydtool.cxx, which exports the function initlibdtoolconfig(), not initlibp3dtoolconfig().
I guess we’ll need an ifdef to switch that to initlibp3dtoolconfig() in the makepanda case. Or maybe it’s time for us to rename the dll’s in ppremake too. But I’d much rather do this in about four months, when we’ve gotten our online games moved to the new plugin system.
Okay, I’ve just made that fix - it now checks for a MAKEPANDA define. We could remove this hack later, when we change ppremake to the new system.
PS. Do you think it’s worth changing panda3d’s commenting style to one accepted by doxygen? That would make generating a reference for the C++ side so much easier and cleaner, and also allow us to put advanced format stuff, todo items, bug numbers, etc. in it.
Just a suggestion though. I don’t know if it’s such a good idea to alter a long-established coding convention.
