Compile SDK (Ubuntu 14.04 Beta)

JornS · March 9, 2014, 11:21am

Hello everyone,

I’m working on a build for Trusty, both 32 and 64 bit. Now I’m stuck with the 64bit build. At first, makepanda can’t find libtiff - even if tiff.h is there. But more important is an issue with FFMPEG:

buildbot@buildbot-VirtualBox:~/panda3d-1.8.1$ python makepanda/makepanda.py --everything --no-gles --no-gles2 --threads 3
WARNING: Could not locate pkg-config package eigen3, excluding from build
Generating library cache...
WARNING: Could not locate thirdparty package artoolkit, excluding from build
WARNING: Could not locate thirdparty package fftw, excluding from build
WARNING: Could not locate thirdparty package tiff, excluding from build
WARNING: Could not locate thirdparty package vrpn, excluding from build
Generating dependencies...
[T1] Building Interrogate database built/pandac/input/libp3text.in
 [T2] Building C++ object built/tmp/p3movies_composite1.o
[T3] Building C++ object built/tmp/p3movies_composite2.o
In file included from panda/src/movies/p3movies_composite2.cxx:3:0:
panda/src/movies/ffmpegVirtualFile.cxx: In member function ‘bool FfmpegVirtualFile::open_vfs(const Filename&)’:
panda/src/movies/ffmpegVirtualFile.cxx:123:68: error: ‘av_open_input_file’ was not declared in this scope
     av_open_input_file(&_format_context, url.c_str(), NULL, 0, NULL);
                                                                    ^
panda/src/movies/ffmpegVirtualFile.cxx: In member function ‘bool FfmpegVirtualFile::open_subfile(const SubfileInfo&)’:
panda/src/movies/ffmpegVirtualFile.cxx:179:68: error: ‘av_open_input_file’ was not declared in this scope
     av_open_input_file(&_format_context, url.c_str(), NULL, 0, NULL);
                                                                    ^
panda/src/movies/ffmpegVirtualFile.cxx: In member function ‘void FfmpegVirtualFile::close()’:
panda/src/movies/ffmpegVirtualFile.cxx:197:5: warning: ‘void av_close_input_file(AVFormatContext*)’ is deprecated (declared at /usr/include/libavformat/avformat.h:1397) [-Wdeprecated-declarations]
     av_close_input_file(_format_context);
     ^
panda/src/movies/ffmpegVirtualFile.cxx:197:40: warning: ‘void av_close_input_file(AVFormatContext*)’ is deprecated (declared at /usr/include/libavformat/avformat.h:1397) [-Wdeprecated-declarations]
     av_close_input_file(_format_context);
                                        ^
panda/src/movies/ffmpegVirtualFile.cxx: In static member function ‘static void FfmpegVirtualFile::register_protocol()’:
panda/src/movies/ffmpegVirtualFile.cxx:231:10: error: ‘URLProtocol’ does not name a type
   static URLProtocol protocol;
          ^
panda/src/movies/ffmpegVirtualFile.cxx:232:3: error: ‘protocol’ was not declared in this scope
   protocol.name = "pandavfs";
   ^
panda/src/movies/ffmpegVirtualFile.cxx:249:52: error: ‘av_register_protocol2’ was not declared in this scope
   av_register_protocol2(&protocol, sizeof(protocol));
                                                    ^
panda/src/movies/ffmpegVirtualFile.cxx: In static member function ‘static int FfmpegVirtualFile::pandavfs_open(URLContext*, const char*, int)’:
panda/src/movies/ffmpegVirtualFile.cxx:271:4: error: invalid use of incomplete type ‘struct URLContext’
   h->priv_data = self;
    ^
In file included from panda/src/movies/ffmpegAudioCursor.h:26:0,
                 from panda/src/movies/ffmpegAudio.cxx:19,
                 from panda/src/movies/p3movies_composite2.cxx:1:
panda/src/movies/ffmpegVirtualFile.h:31:8: error: forward declaration of ‘struct URLContext’
 struct URLContext;
        ^
In file included from panda/src/movies/p3movies_composite2.cxx:3:0:
panda/src/movies/ffmpegVirtualFile.cxx: In static member function ‘static int FfmpegVirtualFile::pandavfs_read(URLContext*, unsigned char*, int)’:
panda/src/movies/ffmpegVirtualFile.cxx:282:52: error: invalid use of incomplete type ‘struct URLContext’
   FfmpegVirtualFile *self = (FfmpegVirtualFile *)(h->priv_data);
                                                    ^
In file included from panda/src/movies/ffmpegAudioCursor.h:26:0,
                 from panda/src/movies/ffmpegAudio.cxx:19,
                 from panda/src/movies/p3movies_composite2.cxx:1:
panda/src/movies/ffmpegVirtualFile.h:31:8: error: forward declaration of ‘struct URLContext’
 struct URLContext;
        ^
In file included from panda/src/movies/p3movies_composite2.cxx:3:0:
panda/src/movies/ffmpegVirtualFile.cxx: In static member function ‘static int64_t FfmpegVirtualFile::pandavfs_seek(URLContext*, int64_t, int)’:
panda/src/movies/ffmpegVirtualFile.cxx:324:52: error: invalid use of incomplete type ‘struct URLContext’
   FfmpegVirtualFile *self = (FfmpegVirtualFile *)(h->priv_data);
                                                    ^
In file included from panda/src/movies/ffmpegAudioCursor.h:26:0,
                 from panda/src/movies/ffmpegAudio.cxx:19,
                 from panda/src/movies/p3movies_composite2.cxx:1:
panda/src/movies/ffmpegVirtualFile.h:31:8: error: forward declaration of ‘struct URLContext’
 struct URLContext;
        ^
In file included from panda/src/movies/p3movies_composite2.cxx:3:0:
panda/src/movies/ffmpegVirtualFile.cxx: In static member function ‘static int FfmpegVirtualFile::pandavfs_close(URLContext*)’:
panda/src/movies/ffmpegVirtualFile.cxx:365:4: error: invalid use of incomplete type ‘struct URLContext’
   h->priv_data = 0;
    ^
In file included from panda/src/movies/ffmpegAudioCursor.h:26:0,
                 from panda/src/movies/ffmpegAudio.cxx:19,
                 from panda/src/movies/p3movies_composite2.cxx:1:
panda/src/movies/ffmpegVirtualFile.h:31:8: error: forward declaration of ‘struct URLContext’
 struct URLContext;
        ^
In file included from panda/src/movies/p3movies_composite2.cxx:4:0:
panda/src/movies/ffmpegAudioCursor.cxx: In constructor ‘FfmpegAudioCursor::FfmpegAudioCursor(FfmpegAudio*)’:
panda/src/movies/ffmpegAudioCursor.cxx:55:38: error: ‘av_find_stream_info’ was not declared in this scope
   if (av_find_stream_info(_format_ctx)<0) {
                                      ^
panda/src/movies/ffmpegAudioCursor.cxx:81:42: error: ‘avcodec_open’ was not declared in this scope
   if(avcodec_open(_audio_ctx, pAudioCodec)<0) {
                                          ^
panda/src/movies/ffmpegAudioCursor.cxx: In member function ‘bool FfmpegAudioCursor::reload_buffer()’:
panda/src/movies/ffmpegAudioCursor.cxx:218:17: warning: ‘int avcodec_decode_audio3(AVCodecContext*, int16_t*, int*, AVPacket*)’ is deprecated (declared at /usr/include/libavcodec/avcodec.h:3608) [-Wdeprecated-declarations]
       int len = avcodec_decode_audio3(_audio_ctx, _buffer, &bufsize, &pkt);
                 ^
panda/src/movies/ffmpegAudioCursor.cxx:218:74: warning: ‘int avcodec_decode_audio3(AVCodecContext*, int16_t*, int*, AVPacket*)’ is deprecated (declared at /usr/include/libavcodec/avcodec.h:3608) [-Wdeprecated-declarations]
       int len = avcodec_decode_audio3(_audio_ctx, _buffer, &bufsize, &pkt);
                                                                          ^
panda/src/movies/ffmpegAudioCursor.cxx: In member function ‘virtual void FfmpegAudioCursor::seek(double)’:
panda/src/movies/ffmpegAudioCursor.cxx:266:42: error: ‘avcodec_open’ was not declared in this scope
   if(avcodec_open(_audio_ctx, pAudioCodec)<0) {
                                          ^
In file included from panda/src/movies/p3movies_composite2.cxx:5:0:
panda/src/movies/ffmpegVideoCursor.cxx: In member function ‘bool FfmpegVideoCursor::open_stream()’:
panda/src/movies/ffmpegVideoCursor.cxx:512:38: error: ‘av_find_stream_info’ was not declared in this scope
   if (av_find_stream_info(_format_ctx) < 0) {
                                      ^
panda/src/movies/ffmpegVideoCursor.cxx:544:43: error: ‘avcodec_open’ was not declared in this scope
   if (avcodec_open(_video_ctx, pVideoCodec) < 0) {
                                           ^
The following command returned a non-zero value: g++ -ftemplate-depth-30 -fPIC -c -o built/tmp/p3movies_composite2.o -Ibuilt/tmp -Ibuilt/include -I/usr/include/python2.7 -DMAKEPANDA= -Ipanda/src/movies -pthread -msse2 -O2 -DBUILDING_PANDA panda/src/movies/p3movies_composite2.cxx
Storing dependency cache.
Elapsed Time: 5 sec
Build process aborting.
Build terminated.

I hope someone has a solution for this. Have a nice day!

rdb · March 9, 2014, 11:23am

1.8.1 was not compatible with more recent versions of ffmpeg. The current CVS version of Panda is, though.

Sorry, you’ll either need to backport the ffmpeg changes from the CVS trunk (which is laborious) or start compiling the latest CVS build.

About the tiff error: please give me the first part of the output with the --verbose flag added.

JornS · March 9, 2014, 12:27pm

Hello Reinier and thank you for your response!

Regarding ffmpeg, that’s odd - but not the end of the world

Well, the tiff issue:

WARNING: Could not locate pkg-config package eigen3, excluding from build
Generating library cache...
Couldn't find library libAR
Couldn't find header file AR/ar.h
WARNING: Could not locate thirdparty package artoolkit, excluding from build
Couldn't find library libfftw
Couldn't find library librfftw
Couldn't find header file fftw.h
Couldn't find header file rfftw.h
WARNING: Could not locate thirdparty package fftw, excluding from build
Couldn't find header file tiff.h
WARNING: Could not locate thirdparty package tiff, excluding from build
Couldn't find library libvrpn
Couldn't find library libquat
Couldn't find header file quat.h
Couldn't find header file vrpn/vrpn_Types.h
WARNING: Could not locate thirdparty package vrpn, excluding from build

But it exists:

buildbot@buildbot-VirtualBox:/usr/include/x86_64-linux-gnu$ ls | grep tiff
tiffconf.h
tiff.h
tiffio.h
tiffio.hxx
tiffvers.h

EDIT: are there any more things broken now in 1.8.1 with more recent libs beside Bullet and FFMPEG?

rdb · March 9, 2014, 1:44pm

I don’t know if there are other things broken. Probably nothing as big as ffmpeg (the ffmpeg developers really like to break their API often).

As for tiff, it seems that libtiff doesn’t come with a convenient pkg-config .pc file, so makepanda resorts to trying to find it by itself, and it doesn’t search in /usr/include/x86_64-linux-gnu. Hmm, a mess. A workaround is to copy the tiff header files to somewhere that makepanda does find them, such as /usr/local/include. For header files, this won’t affect the compatibility of the completed .deb with other systems.

Thanks for making the Ubuntu builds! The community greatly appreciates your efforts.

JornS · March 9, 2014, 5:02pm

Okay I will try it without FFMPEG and look what works. Personally, I don’t really need videos.

That sounds good and explains a lot.

Thank you too for your help. I think that is some kind of a win-win situation: the less people ask you about builds for Ubuntu x.y, the more you can concentrate on the hard work

unixfan · March 9, 2014, 7:01pm

Here’s what I did on my systems (12.04/13.04 respectively)

sudo apt-get purge libavutil-dev sudo apt-get -f install (in case you end up with unmet dependencies and it refuses to purge itself)
sudo apt-get purge libavutil-dev (in case you needed to fix the install first, otherwise move on to steps below) sudo apt-get install yasm
wget [ffmpeg.org/releases/ffmpeg-2.1.3.tar.gz](http://www.ffmpeg.org/releases/ffmpeg-2.1.3.tar.gz) tar xvfz ffmpeg-2.1.3.tar.gz
CFLAGS=-fPIC ./configure --disable-static --enable-shared make
$ sudo make install

That solved the FFMPEG issue for me.

Canonical currently considers switching back to FFMPEG, so this should cease to be an issue soon.

rdb · March 10, 2014, 10:05am

It is also used for decoding audio under OpenAL; it is unfortunate that 1.8.1 relies on ffmpeg for audio. That’s why I’ve recently added support for loading wavs and oggs through libvorbis to the 1.9 trunk, but this doesn’t help in the short term. I guess it’d be good if 1.9 gets released sooner than later.

JornS · March 10, 2014, 12:28pm

+1!

koudelka · April 4, 2014, 3:50pm

I have compiled current CVS tree (Panda3D 1.9 SDK) for ubuntu 14.04 64bit.

The deb package is available from my Google drive in case anyone wants it (sometimes “dpgk -i” is needed to install. Bug in 14.04?). goo.gl/ZXL2NF
If there are any issues, please report to me
This is the build status report.

Makepanda Final Status Report
Makepanda: Regular build
('Makepanda: Compiler:', 'GCC')
('Makepanda: Optimize:', 3)
('Makepanda: Keep Pkg:', 'PYTHON DIRECT GL GLES GLES2 TINYDISPLAY NVIDIACG EGL EIGEN OPENAL VORBIS FFMPEG SWSCALE ODE BULLET PANDAPHYSICS ZLIB PNG JPEG TIFF FREETYPE OPENSSL OPENCV VISION NPAPI GTK2 WX FLTK OSMESA X11 XF86DGA XRANDR XCURSOR PANDATOOL PVIEW DEPLOYTOOLS SKEL PANDAFX PANDAPARTICLESYSTEM CONTRIB SSE2 NEON ')
('Makepanda: Omit Pkg:', 'DX8 DX9 FMODEX SWRESAMPLE PHYSX SPEEDTREE SQUISH MAYA6 MAYA65 MAYA7 MAYA8 MAYA85 MAYA2008 MAYA2009 MAYA2010 MAYA2011 MAYA2012 MAYA2013 MAYA20135 MAYA2014 MAX6 MAX7 MAX8 MAX9 MAX2009 MAX2010 MAX2011 MAX2012 MAX2013 MAX2014 FCOLLADA VRPN FFTW ARTOOLKIT DIRECTCAM AWESOMIUM ROCKET CARBON COCOA TOUCHINPUT ')
Makepanda: Don't generate API reference manual
Makepanda: Version ID: 1.9.0

There is one issue I have not managed to resolve yet. I am unable to make it work together with fftw-dev avaliable in ubuntu repo. Maybe someone has some ideas about this?

//usr/lib/librfftw.so.2: undefined reference to `fftw_make_node_real2hc'
//usr/lib/librfftw.so.2: undefined reference to `fftw_complete_twiddle'
//usr/lib/librfftw.so.2: undefined reference to `fftw_gettimeofday_time_diff'
//usr/lib/librfftw.so.2: undefined reference to `fftw_wisdom_add'
//usr/lib/librfftw.so.2: undefined reference to `fftw_malloc'
//usr/lib/librfftw.so.2: undefined reference to `fftw_use_plan'
//usr/lib/librfftw.so.2: undefined reference to `fftw_make_node_rgeneric'
//usr/lib/librfftw.so.2: undefined reference to `fftw_destroy_table'
//usr/lib/librfftw.so.2: undefined reference to `fftw_wisdom_lookup'
//usr/lib/librfftw.so.2: undefined reference to `fftw_free'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_destroy_plan'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_work_size'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_fprint_plan'
//usr/lib/librfftw.so.2: undefined reference to `fftw_gettimeofday_get_time'
//usr/lib/librfftw.so.2: undefined reference to `fftw_make_plan'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_create_plans_generic'
//usr/lib/librfftw.so.2: undefined reference to `fftw'
//usr/lib/librfftw.so.2: undefined reference to `fftw_make_node_hc2hc'
//usr/lib/librfftw.so.2: undefined reference to `fftw_make_node_hc2real'
//usr/lib/librfftw.so.2: undefined reference to `fftw_destroy_plan_internal'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_create_plans_specific'
//usr/lib/librfftw.so.2: undefined reference to `fftw_reverse_int_array'
//usr/lib/librfftw.so.2: undefined reference to `fftw_fprint_plan'
//usr/lib/librfftw.so.2: undefined reference to `fftw_pick_better'
//usr/lib/librfftw.so.2: undefined reference to `fftw_lookup'
//usr/lib/librfftw.so.2: undefined reference to `fftw_factor'
//usr/lib/librfftw.so.2: undefined reference to `fftw_make_empty_table'
//usr/lib/librfftw.so.2: undefined reference to `fftw_insert'
//usr/lib/librfftw.so.2: undefined reference to `fftw_die'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_new_plan_array'
//usr/lib/librfftw.so.2: undefined reference to `fftw_estimate_node'
//usr/lib/librfftw.so.2: undefined reference to `fftwnd_create_plan_aux'
collect2: error: ld returned 1 exit status
Process exited with exit status 1 and signal code 0
The following command returned a non-zero value: g++ -o built/bin/pview -Lbuilt/lib -Lbuilt/tmp built/tmp/pview_pview.o built/lib/libp3framework.so built/lib/libpandaegg.so built/lib/libpanda.so built/lib/libpandaexpress.so built/lib/libp3dtool.so built/lib/libp3dtoolconfig.so built/lib/libp3pystub.a -pthread -L/usr/X11R6/lib -ldl
Storing dependency cache.
Elapsed Time: 2 sec
Build process aborting.

rdb · April 4, 2014, 11:20pm

I think you may have to explicitly get an older version of FFTW, perhaps libfftw2-dev?

We only use FFTW for a rather esoteric edge case, though: storing lossy compressed animation in .bam files. It’s pretty rare that someone would need such a feature.

Because of its rather restrictive license and its limited usefulness I would advise you to build without it (we don’t include it in the public builds either).

koudelka · April 5, 2014, 12:33am

Ubuntu has a fftw3 package for the new version and to my understanding fftw-dev is v2. Since it was not important I won’t bother with it

I have some other questions thought.

Are there any automated tests available or I have to test using sample applications?
Do you keep the github version of the code current with the CVS tree? Would you advice against following that code instead of CVS?

Thank you for all your great work

rdb · April 5, 2014, 8:41am

Not at the moment. We’re thinking about this.
If you do, follow this:
github.com/panda3d/incremental
Usually it’s a few days behind, sometimes a week or two. Right now, it’s running a month behind because of trouble with the automated build bots.
Keep in mind that that repository may eventually be deleted when we migrate to git.