Cygwin build error: ppremake wont actually run

[hughperkins]

Cygwin build error: ppremake wont actually run

Did the following:

• Downloaded source from download page

• Installed Cygwin

• No configure in ppremake directory, so followed the instructions in BUILDINGFROMCVS.txt (not the exact filename, but you know which one I mean?)

• Did:

./configure
make
make install

Now, if I do ./ppremake I get an access violation 0xc00000005 error.

This occurs even before main runs, since if I modify main as follows:

main(int argc, char *argv[]) {
cout << “start” << endl;
string progname = argv[0];
extern char *optarg;

… then “start” never actually appears on the console (or anywhere).

A normal helloworld runs just fine, ie:

#include
using namespace std;

int main()
{
cout << “blah” << endl;
return 0;
}

g++ -c test.o test.cpp
g++ -o test test.o

The helloworld runs just fine

Any ideas? Cygwin is whatever the current Cygwin version is on the website (downloaded it yesterday); the sourecode is whatever the sourcecode is in the 33~MB .zip file on the download page (again, downloaded it yesterday).

Hugh

[hughperkins]

It gets stranger:

leaving in the cout line at the start of main, ie:

...
int
main(int argc, char *argv[]) {
	cout << "start" << endl;
  string progname = argv[0];
  extern char *optarg;
...

If I comment out the following lines, then the program runs to completion , but if I uncomment them, access violation 5

 /*
case 'p':
  platform = optarg;
  break;

case 'c':
  ppremake_config = optarg;
  got_ppremake_config = true;
  break;

case 's':
  sed_command = optarg;
  got_sed_command = true;
  break;
*/

The wierd thing is: these lines shouldnt even run right? Certainly not before the cout ought to be executed

Some Cygwin wierdness going on?

Hugh

[hughperkins]

Presumably the issue is with the optarg variable, which is an extern.

Perhaps a link with this message during the link? :

Info: resolving _optarg by linking to __imp__optarg (auto-import)

?

Hugh

[hughperkins]

Ok, I added the following line to ppremake.cxx, at line 9

char *optarg;

and now ppremake seems to run ok. Not sure if it will correctly parse commandline arguments or not though?

Hugh

[hughperkins]

Ok, so ppremake seems to run with no errors now.

Next question: so, if I’m running cygwin, why does it complain that cl.exe is not found?

Ok, so I like cl.exe, it runs much faster than cygwin g++, so I add it to the path.

• make now provides the error message that /DEBUG is an invalid option for link

Presumably the gcc link is being called instead of msvc link.

-> rename /usr/bin/link to /usr/bin/_link

Make works better, but now complains cant find msvcrt.lib

… which is normal since I have the Microsoft Visual C++ Toolkit 2003, which contains the full .net compiler, linker and statically linked runtime libraries, but not the dynamically linked runtime libraries, ie msvcrt.

But my question is: why is make trying to use msvc when we are running under Cygwin?

Hugh

[hughperkins]

Ok, read the instructions; seems that msvc compiler is used anyway.

Added #define LINK_ALL_STATIC 1 to config.pp

And changed line 56 of dtool\pptempl\compilerSettings.pp to read:

  #define LIBBER link /lib

(was #define LIBBER lib)

Cant help thinking: perhaps it could be useful to create a wiki (eg mediawiki.org, works nicely, looks good) for Panda? That way, this information above for example could be easily added into platform-specific build instructions?

Note: build not complete yet, but close-ish.

Hugh

[hughperkins]

3 hours later (!), added the following line to Config.pp:

#define LINK_FORCE_STATIC_RELEASE_C_RUNTIME 1

dtool builds!

A wiki could be great for putting this information?

Hugh

[hughperkins]

4 hours later:

Ok, panda built, but…

I had to add the following line to Config.pp, because the make was looking for directx 8 header, and Microsoft only supplies DirectX9:

#define HAVE_DX

So, my config.pp looked finally like:

#define LINK_ALL_STATIC 1
#define LINK_FORCE_STATIC_RELEASE_C_RUNTIME 1
#define HAVE_DX
#define HAVE_PYTHON /cygdrive/f/dev/panda3d-1.0.2-cyg/thirdparty/win-python

-> This built ok, but when I try to run it, I get:

Administrator@A29 /cygdrive/f/dev/panda3d-1.0.2-cyg/panda
$ pview
Known pipe types:
(all display modules loaded.)
No graphics pipe is available!
Your Config.prc file must name at least one valid panda display
library via load-display or aux-display.

So, my Config.prc file contains the following:

load-display pandagl

It’s at:

/cygwin/usr/local/panda/etc/Config.prc (Cygwin notation)
F:\cygwin\usr\local\panda\etc\config.prc (Windows notation)

I tried specifying the path to this file by doing:

export PRC_DIR=/usr/local/panda/etc

but the errormessage still displays.

I tried mistyping pandagl in the config file, to see if that gives an errormessage, and it didnt, and I saw youve modifed the finding-the-config-file algorithm recently, so Im thinking that maybe panda is simply not reading the config file at all?

Hugh

[hughperkins]

Additional information:

If I try running pview from Windows, I get the same error message.

However, if I try running pview from Windows, defining PRC_DIR explicitly, I get a new errormessage (more worrying actually):

F:\cygwin\usr\local\panda>bin\pview
Error! Cannot change back to /usr/local/panda
:display: loading display module: libpandagl.dll
:display: Unable to load: Module not found
Known pipe types:
(all display modules loaded.)
No graphics pipe is available!
Your Config.prc file must name at least one valid panda display
library via load-display or aux-display.

I searched my harddrive for libpandagl.dll, and there werent any (except for the one in the prebuilt binaries from the prebuilt binary download).

Not sure where to go from here really?

Hugh

[hughperkins]

Hmmm, and now I get the same error on Cygwin, doing:

export PRC_DIR=/f/cygwin/usr/local/panda/etc

So it seems initially the problem was one of locating Config.prc, but there is a more serious problem now, with the build?

Hugh

[hughperkins]

Additional info:

deleting pview.exe and remaking shows that pview.exe is linked with libpandagl.lib , so presumably the appropriate symbols are available? Maybe there is somethign that wants to load the dll anyway?

Hugh

[hughperkins]

Additional information:

tried removing #define LINK_ALL_STATIC 1 from Config.pp, leaving:

#define LINK_FORCE_STATIC_RELEASE_C_RUNTIME 1
#define HAVE_DX
#define HAVE_PYTHON /cygdrive/f/dev/panda3d-1.0.2-cyg/thirdparty/win-python

redid ppremake on dtool, remade dtool
redid ppremake on panda, make clean, make

Following error message on the build:

Administrator@A29 /cygdrive/f/dev/panda3d-1.0.2-cyg/panda/metalibs/pandaexpress
$ make
link /nologo /DLL  /MAP /MAPINFO:EXPORTS /MAPINFO:LINES /fixed:no /incremental:n
o /stack:4194304 /NODEFAULTLIB:LIBCI.LIB /NODEFAULTLIB:MSVCRTD.LIB /OPT:REF  /OU
T:"Opt3-Cygwin\libpandaexpress.dll" Opt3-Cygwin/pandaexpress_pandaexpress.obj Op
t3-Cygwin/libpandaexpress_module.obj ../../src/downloader/Opt3-Cygwin/downloader
_downloader_composite1.obj ../../src/downloader/Opt3-Cygwin/downloader_downloade
r_composite2.obj ../../src/downloader/Opt3-Cygwin/libdownloader_igate.obj ../../
src/express/Opt3-Cygwin/express_express_composite1.obj ../../src/express/Opt3-Cy
gwin/express_express_composite2.obj ../../src/express/Opt3-Cygwin/libexpress_iga
te.obj ../../src/pandabase/Opt3-Cygwin/pandabase_pandabase.obj /LIBPATH:"f:\cygw
in\usr\local\panda\lib" libdtoolconfig.lib libdtool.lib advapi32.lib ws2_32.lib
libcmt.lib libcpmt.lib
   Creating library Opt3-Cygwin\libpandaexpress.lib and object Opt3-Cygwin\libpa
ndaexpress.exp
pandabase_pandabase.obj : error LNK2001: unresolved external symbol "__declspec(
dllimport) void * (__cdecl* global_operator_new)(unsigned int)" (__imp_?global_o
perator_new@@3P6APAXI@ZA)
libdownloader_igate.obj : error LNK2001: unresolved external symbol "__declspec(
dllimport) void * (__cdecl* global_operator_new)(unsigned int)" (__imp_?global_o
perator_new@@3P6APAXI@ZA)
.... (this repeats for a while, with other symbols )...

I’ve seen this before, but I never quite figured out how to solve it. I assume it’s looking for the new and delete functions from the C++ runtime library, but not sure why it cant find them in libcpmt.lib?

Hugh

[drwr]

Wow, you sure know your way around C++ and the Microsoft build environment! I’m impressed!

I can answer your few remaining confusions.

(1) You need to set the environment variable PANDA_ROOT to f:/cygwin to allow Panda to find your files (without having to prefix everything with /f/cygwin). This is because Panda does not link with the Cygwin libraries and has no idea where the Cygwin root directory is unless you tell it.

(2) The most recent build error is because you didn’t make clean in dtool after you changed from lib’s to dll’s (and so the symbols _global_operator_new and _global_operator_delete, which are Panda symbols defined in dtool, aren’t properly exported).

Other points you have already figured out:

(3) There appears to be a bug in recent versions of Cygwin that cause users of getopt() to crash on startup; you found a fine workaround, although I’m not sure if ppremake can read the arguments to the command-line options that require them, like -p or -c with that workaround. But those options are rarely used, so no problem.

(4) We only use Cygwin for a build environment, and we only use the Cygwin compiler to bootstrap the build environment by compiling ppremake. After that it’s all VC++. We once tried to use the Cygwin compiler to build Panda itself–it would be better if we could support it, since it’s free, and it seems reasonable that we could, since Panda compiles just fine under gcc on Linux. In fact, gcc seems to do a better job of optimizing, in many cases, than VC++. But the last time we tried porting to Cygwin’s gcc, we discovered that it lacked sufficient DLL support for Panda. This was some time ago; it’s possible things have changed since then, but we haven’t had the time to revisit it.

(5) When you use LINK_ALL_STATIC, the libraries are all compiled into one big executable. This includes libpandagl, which is normally built as a separate dll (and must be loaded via load-display). The intended design is for libpandagl to get compiled into the executable in this case and properly initialized so that it gets added to the internal table of available display devices, which obviates the need to load it explicitly via load-display. It sounds like this is not quite working. It has been years since we’ve built with LINK_ALL_STATIC enabled, though, so frankly I’m amazed it worked as well as it did.

Of course, building with LINK_ALL_STATIC makes it impossible to use Panda with Python, unless you also want to recompile Python to link it statically with the Panda library.

David

[system]

Ok, that’s great! I’ll clean dtool and rebuild and see how that goes.

(3) There appears to be a bug in recent versions of Cygwin that cause users of getopt() to crash on startup; you found a fine workaround, although I’m not sure if ppremake can read the arguments to the command-line options that require them, like -p or -c with that workaround. But those options are rarely used, so no problem.

Just so you know, ppremake does seem to accept the commandline arguments ok. I tested just now for example -h and -V, and they both worked (-h displays the help page, -V displays 1.18). Maybe #ifdef the optarg declaration into ppremake somewhere?

We once tried to use the Cygwin compiler to build Panda itself–it would be better if we could support it, since it’s free

Did you know you can get msvc for free too? microsoft.com/downloads/deta … layLang=en

My impression is that cl seems to run much much faster than gcc-in-cygwin, though that could possibly be a cygwin issue rather than a gcc issue. A not-very-rigorous benchmarking gave the impression that cl-in-cmd ran around 2-5 times faster than gcc-in-cygwin

So, msvc for build, and Cygwin for configure could be a great combination of cross-platform configuration and build speed optimization.

Hugh

[drwr]

My prediction is that things like “ppremake -h” will work, since this does not use the optarg variable (-h has no arguments); but things like “ppremake -c myConfig.pp” will fail (the optarg variable, which should contain “myConfig.pp”, will instead be empty).

I didn’t know about the VC++ Toolkit; that does help things a bit. I am interested to learn how well it works out for you. The idealist would argue that although it is free in the monetary sense, it is not Free in the Richard Stallman sense, and so is still less than perfect. I won’t make that argument, though.

My experience with gcc, VC++, and Cygwin is this: gcc tends to run faster than VC++ in its non-optimizing mode, and slower than VC++ in its optimizing mode. But in its optimizing mode, it generates smaller, faster code than VC++ does in its optimizing mode. (Although VC++ 7.1 is starting to catch up.) Also, all programs that are heavily I/O and process dependent, such as compilers, tend to run slower under Cygwin than their native counterparts, so I suspect Cygwin/gcc will suffer that penalty as well. But it still might be a preferable compiler if it generates tighter code. In any case, it’s a bit of an academic argument at this point, since we don’t support Cygwin/gcc.

David

[hughperkins]

Ok, that makes sense.

I forgot to say before btw, I’m impressed by how quickly you reply to questions, and the attention to detail.

Just for info, the build is working now! In the end, what I found is that there are instructions on the internet for obtaining msvcrt.lib and msvcprt.lib, and so the build can be pretty standard.

There’s a summary of the tweaks to build Panda on Windows using the Toolkit and Cygwin here:

manageddreams.com/osmpwiki/index … le=Panda3D

Hugh

[hughperkins]

Random thought: perhaps it would be possible to use gnumake natively from Windows rather than via Cygwin? So, the ppremake would run in Cygwin, but the build would run from Windows.

Not sure if it actually builds quicker, but at least the machine remains 100% useable whilst the build is going on in the background (Cygwin basically kills my machine…).

The makefiles generated by ppremake seem to work almost perfectly with gnumake. The only thing to watch is that there is a directory structure \usr\local\panda in the current partition.

Hugh

Edit: there’s a precompiled gnumake.exe at: manageddreams.com/gnumake.exe

[drwr]

Great, thanks for your feedback! Are you going to keep the wiki up for the foreseeable future? If so, we can just leave a link to it for now, rather than copying your advice into our build document (I think our plan is to have our own wiki eventually).

Does the VC++ Toolkit include nmake.exe? That is a Microsoft-provided make utility that ppremake can use in lieu of GNU make, and it does run a lot faster on Windows.

The OSMP looks very interesting! I’m excited to see how it develops. One suggestion I might provide: although you can integrate your FractalSpline library with Panda by setting a callback and then having the FractalSpline library issue OpenGL commands, it might be a more effective integration to have your library instead generate the Panda representations of the OpenGL primitives it would have eventually rendered. This may be a little more work to integrate, because it means you have to write more code (you’ll generate data structures instead of issuing OpenGL calls), but at the end of the day it gives Panda more control to optimize the primitives along with the rest of the scene, and it also incidentally provides DirectX support for free (there are some recent graphics cards that support DirectX much better than OpenGL, sadly).

There are several examples of this kind of integration in the current Panda codebase; for instance panda/src/parametrics/ropeNode.h, which generates a number of possible representations of a parametric spline. There’s also sheetNode.h, which generates a parametric mesh, and panda/src/text/textNode.h, which generates on-the-fly text primitives.

Another suggestion is to get a CVS client and check out the latest Panda source code directly from the SourceForge repository. This is especially important if you might be considering my suggestion to generate Panda primitives, since I am in the process of rewriting the internal primitive structure right now to make them more general and easier to construct, and this new code is only available in the CVS repository at the moment.

David

[hughperkins]

Some replies, and a critical blocking question at the end:

My prediction is that things like “ppremake -h” will work, since this does not use the optarg variable (-h has no arguments); but things like “ppremake -c myConfig.pp” will fail (the optarg variable, which should contain “myConfig.pp”, will instead be empty).

Yes, thats correct; and premake -D pp.dep fails too.

A more correct workaround seems to be simply to edit config.h and undefine HAVE_GNU_OPT.

Are you going to keep the wiki up for the foreseeable future?

Yes, should stay up for a while.

Does the VC++ Toolkit include nmake.exe?

It doesnt although I did manage to find a copy on the Internet.

it might be a more effective integration to have your library instead generate the Panda representations of the OpenGL primitives it would have eventually rendered.

Ok, that makes sense, and sounds like a good idea for a few reasons.

There are several examples of this kind of integration in the current Panda codebase; for instance panda/src/parametrics/ropeNode.h, which generates a number of possible representations of a parametric spline. There’s also sheetNode.h, which generates a parametric mesh, and panda/src/text/textNode.h, which generates on-the-fly text primitives.

Ok, thats really good info; I’m looking into these.

Another suggestion is to get a CVS client and check out the latest Panda source code directly from the SourceForge repository.

Ok. Question: what would it take to become a developer with write-access to the repository?

Lastly, the critical blocking question: how can I set up a unit testing framework for a new parametric node? Just so you know, linking libpanda.dll takes around 5-10 minutes on my machine, and renders the machine essentially inoperable during that time!

I was thinking maybe just link the new node with pview.exe, but it seems that this gives many errors of references already defined in libpanda.dll?

What would be your recommendations for this? Ideally, it should be possible to compile and link a test node in under 10 seconds, certainly less than 30.

What do you think?

Hugh

[drwr]

There are several examples of this kind of integration in the current Panda codebase; for instance panda/src/parametrics/ropeNode.h, which generates a number of possible representations of a parametric spline. There’s also sheetNode.h, which generates a parametric mesh, and panda/src/text/textNode.h, which generates on-the-fly text primitives.

Ok, thats really good info; I’m looking into these.

If you get the latest from CVS, you’ll see the code to generate primitives falls into two cases, selected by use_qpgeom. Study the case in which use_qpgeom is true, and ignore the other case. And put “use-qpgeom 1” (note the hyphen) in your Config.prc file.

During this transition period of rewriting the Geom implementation, we have both the original and the new implementations in the codebase, with “use_qpgeom” to select which one you want to use. When I am satisfied with the new code, which will be soon, I will remove the other cases, remove the use_qpgeom variable, and globally remove the qp prefix.

Question: what would it take to become a developer with write-access to the repository?

You’d just have to persuade me that you were serious about making contributions to Panda, that you were in accord with our vision for Panda, and that it would be easier for me to let you commit things directly into the repository than to review and merge your contributions via a patch file. Oh, and that you had read panda/src/doc/coding-style.txt.

Lastly, the critical blocking question: how can I set up a unit testing framework for a new parametric node? Just so you know, linking libpanda.dll takes around 5-10 minutes on my machine, and renders the machine essentially inoperable during that time!

The absolute simplest answer is to do your development on Linux, if that’s an option for you. On Linux, each directory compiles into its own .so, so there is no mammoth link at the libpanda.so stage. On Windows, it’s more complicated for fiddly reasons having to do with the way DLL’s interoperate (poorly) with templates.

Still, you can develop on Windows too, especially if you’re more comfortable there. There is no reason why you can’t link your new code in with pview, or with any other standalone application. You probably shouldn’t remove code from libpanda, just create new classes and link them in with pview. (It is possible to move an existing class out of libpanda.dll, but it’s not intuitive how to do this. I can show you how if you decide this is something you need to do.) In your new classes, you won’t use the EXPCL_PANDA prefix on the class definition, since that is used to tell the compiler which DLL the class is compiled into. Just omit the EXPCL_ prefix altogether it if is compiled directly in pview.

David