Android Bounty


#141

It’s definitely loading the PRC file and getting the plugin-path okay. Here’s the result of my shell commands running on the tablet:

127|root@flo:/data/data/com.hipipal.qpyplus/files/bin # ./qpython.sh

Python 2.7.2 (default, Nov  2 2015, 01:07:37)
[GCC 4.9 20140827 (prerelease)] on linux4
Type "help", "copyright", "credits" or "license" for more information.

>>> import panda3d.core
>>> from panda3d.core import ConfigVariableString, loadPrcFileData, loadPrcFile
>>> print ConfigVariableString("plugin-path")

/data/data/com.hipipal.qpyplus/files/lib

>>> from panda3d.core import ExecutionEnvironment
>>> print ExecutionEnvironment.getDtoolName()

unknown

>>> exit()

#[QPython] Press enter to exit

And here’s the listing of the folder:

root@flo:/data/data/com.hipipal.qpyplus/files/lib # ls
libeggcacher.so
libgnustl_shared.so
libinterrogate.so
libinterrogate_module.so
libmultify.so
libp3android.so
libp3dcparse.so
libp3direct.so
libp3dtool.so
libp3dtoolconfig.so
libp3framework.so
libp3interrogatedb.so
libp3vision.so
libpackpanda.so
libpanda.so
libpandaai.so
libpandaegg.so
libpandaexpress.so
libpandafx.so
libpandagles.so
libpandaphysics.so
libpandaskel.so
libparse_file.so
libpunzip.so
libpview.so
libpython2.7_old.so
libpzip.so
libtest_interrogate.so
python2.7
python27.zip

And the contents of /sdcard/Config.prc:

root@flo:/ # cat /sdcard/Config.prc
###########################################################
###                                                     ###
### Panda3D Configuration File -  User-Editable Portion ###
###                                                     ###
###########################################################

# Uncomment one of the following lines to choose whether you should
# run using OpenGL, DirectX or TinyPanda (software) rendering.
# There can only be one load-display line, but you can use
# multiple aux-display lines to specify fallback modules.
# When the module indicated by load-display fails, it will fall
# back to the next display module indicated by aux-display,
# when that fails, the next aux-display line, and so on.

plugin-path /data/data/com.hipipal.qpyplus/files/lib

load-display pandagles

# These control the placement and size of the default rendering window.
# A value of -2 for the origin means to center it on the screen,
# while -1 lets the window manager choose the position.

win-origin -2 -2
win-size 800 600

# Uncomment this line if you want to run Panda fullscreen instead of
# in a window.

fullscreen #t

# The framebuffer-hardware flag forces it to use an accelerated driver.
# The framebuffer-software flag forces it to use a software renderer.
# If you don't set either, it will use whatever's available.

framebuffer-hardware #t
framebuffer-software #f

# These set the minimum requirements for the framebuffer.
# A value of 1 means: get as many bits as possible,
# consistent with the other framebuffer requirements.

depth-bits 1
color-bits 1
alpha-bits 0
stencil-bits 0
multisamples 0

# These control the amount of output Panda gives for some various
# categories.  The severity levels, in order, are "spam", "debug",
# "info", "warning", and "error"; the default is "info".  Uncomment
# one (or define a new one for the particular category you wish to
# change) to control this output.

notify-level warning
default-directnotify-level warning

# These specify where model files may be loaded from.  You probably
# want to set this to a sensible path for yourself.  $THIS_PRC_DIR is
# a special variable that indicates the same directory as this
# particular Config.prc file.

model-path    $MAIN_DIR
model-path    $THIS_PRC_DIR/..
model-path    $THIS_PRC_DIR/../models

# This enable the automatic creation of a TK window when running
# Direct.

want-directtools  #f
want-tk           #f

# Enable/disable performance profiling tool and frame-rate meter

want-pstats            #f
show-frame-rate-meter  #f

# Enable audio using the OpenAL audio library by default:

audio-library-name p3openal_audio

# Enable the use of the new movietexture class.

use-movietexture #t

# The new version of panda supports hardware vertex animation, but it's not quite ready

hardware-animated-vertices #f

# Enable the model-cache, but only for models, not textures.

model-cache-dir $HOME/.panda3d/cache
model-cache-textures #f

# This option specifies the default profiles for Cg shaders.
# Setting it to #t makes them arbvp1 and arbfp1, since these
# seem to be most reliable. Setting it to #f makes Panda use
# the latest profile available.

basic-shaders-only #f

I’m re-running makepanda as you suggested, just to see if that works. Also I’ll see if I can use ctypes as you say.

I’ll keep trying. As always thanks for the help you give to me and everyone!

EDIT: Think I may have found the problem:

>>> from ctypes import CDLL
>>> CDLL("/data/data/com.hipipal.qpyplus/files/lib/libpandagles.so")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/QPython/QPython2-core/build/python-install/lib/python2.7/ctypes/__init__.py", line 356, in __init__
OSError: dlopen failed: cannot locate symbol "panda_android_app" referenced by "libpandagles.so"...

#142

Ah, that is definitely the problem then. Note that you can use the adb (android debug bridge) to see the log and see these errors as well.

The panda_android_app symbol is defined in panda/src/android/config_android.* and assigned to by android_main(), which is what runs when the pview activity is launched. All this code was really originally designed to run from the PandaActivity java app, which also provides a few bindings so that we can call Android’s Java APIs for loading images (so we don’t have to bundle libpng/libjpeg).

But you’re loading Panda from the Python shell, so you don’t have libp3android.so linked in or an android_main defined. Hmm. You could just edit panda/metalibs/panda/panda.cxx to include config_android.h and add in a call to init_libandroid(), as well as link libp3android into libpanda, which would solve the immediate symbol load error.

(It may then also be necessary to add something like EXPCL_PANDA to the panda_android_app extern declarations in config_android.h and in androiddisplay/androidGraphicsWindow.cxx.)

However, that’s not enough, since we still have to assign panda_android_app to the “struct android_app” pointer in order for us to even create a window. I’m not sure how we could get that from the QPython app - is there some internal API that it exposes from which we could get the android_app pointer?

Another approach is that Panda still provides the main entry point (as we do in the pview case) but instead call into the Python library to spawn the Python interpreter. This would be more in line with how the Android integration is currently designed, and provides a clearer path to accessing the Android APIs we need.


#143

Hmm, I see what you mean. It does seem the correct way to go is allow people to package up an APK of Panda, with the Python interpreter bundled, allowing a .py/.pyc file to be started at that time.

The reason I decided to try and integrate Panda3D with QPython is because ultimately it would be seriously cool to be able to run Panda3D code from inside QPython itself (just load the script, press Play and off you go). Even seeing the main libraries work in the console now, without any failure, is very exciting. I was considering trying out Bullet integration too, amoungst many of the other plugins – basically trying to get as much working as possible. All of this is pretty much credit to yourself and anyone else who works on Panda.

There might be an API available as you mention – I’ll check later. There is an example of something similar that has been accomplished with Kivy. Kivy has a basic 3D rendering system itself (incomparible to Panda) and all you have to do is put “#qpy:kivy” as a remark at the top of the script and a GL window is spawned for it instead of the regular console. However, as at the time of writing I’ve not looked into why/how this works, or how it can be used/adjusted for Panda. It would be great if Panda could be integrated in the same way Kivy is though, and would open up so much opportunity with Panda on mobile, since simple projects could be directly portable from their PC counterparts. Panda deserves that kind of attention because it is seriously good.

Personally though, I don’t have to use QPython and am fine with Panda providing the initial entry point (so it has correct access to the screen), and then that spawning Python, if that’s the only really viable way of doing it. Basically so long as there is “a way” to run a script, or package up an APK that runs a script successfully then ultimately I (or anyone) could run their regular Python scripts on Panda3D/Android too.

I know it’s all possible, just how long it’ll take me to get it to work is another matter – but I appreciate all the help you give in the meantime.


#144

Hiya,

I’ve decided on your second way and am using your pview as a template to get started on integrating Python correctly. I think I’ve got most of it done – basically what I’m aiming for now is an APK can be built of the Python interpreter, the relevant modules, and Panda all combined together.

The only problem is, your sample works, but when building it exactly as per your instructions earlier (the only difference is I’m using NDK r9c since I noticed there were references to “sources/cxx-stl/gnu-libstc++/4.8” which isn’t present in r8d), the APK is created fine, however when you run the application, it quits out with “Unfortunately, Panda Viewer has stopped.”. I haven’t added Python or anything at that point.

The error shown in the console is similar. I’m using OS Monitor to read the logcat information at the moment, and it shows: FATAL_EXCEPTION … dlopenfailed: cannot locate symbol “panda_android_app” referenced by “libpandagles.so”… at [etc] org.panda3d.android.PandaActivity.(PandaActivity.java:56).

Do you happen to know what is wrong? Your downloadable APK sample works, but the instructions to build it don’t seem to work for me (I’m using the latest code from GitHub)…


#145

I wonder if it’s a result of the visibility settings we nowadays use when compiling. Try either removing the -fvisibility=hidden flag to makepanda, or add EXPORT_CLASS to the panda_android_app declaration in config_android.h.

In the meantime I’ll see if I can get the NDK set up for me again so I can try building for Android as well.


#146

OK, I pushed a bunch of fixes to master, which fixes a bunch of link issues as well as some runtime crashes in the OpenGL ES 1 renderer. I can now see the animating panda when running pview on my Android again. :slight_smile:


#147

Thanks rdb for all your help, that all builds well on my system too and runs nicely on the tablets.

I’ll let you know how I get on with the Python integration :slight_smile: I’ve been reading up on how to do it, and will be working with the pview example you kindly provided along with other examples that should give me everything I need. I may be off it for a few days but hopefully will soon have something successful to report!

  • Greg

#148

So, has Android port been on hold all these years because of no official Python support?


#149

It’s been on hold because nobody’s been working on it, presumably because people have been paying more attention to the WebGL port, which runs on both Android and iOS (and would therefore be a viable way of deploying to Android).

For WebGL, I made a custom (patched) build of Python and then froze a Python application using pfreeze, and compiled that for the target platform. I imagine a similar approach would work for Android, which would make deployment easier in that the Python interpreter is embedded and Python files would not have to be read from disk; plus, we could reuse much of the tools that I used to build Python programs for the WebGL port.

I haven’t personally made this my primary focus because I feel improving other parts of the engine is more important. I’d rather focus on making Panda a strong engine on a few platforms than a mediocre engine on many platforms. :slight_smile: Still, I welcome and will aid any effort to further the state of Panda on Android.


#150

Isn’t WebGL limited when it comes to certain rendering features as well as slower, even more so on mobile devices? I understand why you’d consider it, but for me these two reasons would make it a bad idea.
Plus, you’d limit yourself to newest IOS and Android versions as well, right?


#151

Please disregard the request that was here. It was down to a circular reference in the Android version of Python 2.7.2, which is now fixed.

The Python DLL is now sucessfully loaded along with Panda’s libraries in the core Activity, however the Python interpreter now doesn’t want to run my test scripts, grr.


#152

Huh, that is strange. It certainly seems that doing loadLibrary(“python2.7”) first is the way to go, and that it depending on itself (whatever that means) is something that needs to be addressed. Not sure how such a thing can even happen, though.


#153

Sorry :slight_smile: we were literally writing at the same time!

It was a buggy build of 2.7.2 that was at fault. I found a later version which built and loaded okay.


#154

Heya,

I’ve tried several ways I can think of to get this to work, but unfortunately haven’t been successful yet. I feel like I am ‘close’ but every time I get that feeling it seems to jinx it!

I have an APK with two activities – one to install and test-launch Python from the filesystem, which works perfectly with all the Panda libraries, but can’t draw anything, and the Native Activity which you wrote, which launches Python correctly but crashes out upon trying to start the renderer. If Python doesn’t render anything then pview can work as normal with the walking Panda.

So I modified config_android.cxx to reflect the change of APK location and I’ve tried a system() call from a version of pview.cxx’s main() function to launch Python. This sets all the environment variables and launches Python in one line (so the vars don’t get removed). My Python scripts do work when they are run from pview – Panda libraries can be imported, files can be written, etc, but as soon as test = ShowBase() is called, the Python script just exits (even in a try…except to catch all exceptions) and the pview.cxx program continues on to where it draws the walking Panda. The Android debugger outputs (amoungst other things):

I/DEBUG   ( 2111):     #00  pc 00028d44  /data/app-lib/com.android.python27-1/libpandagles.so
I/DEBUG   ( 2111):     #01  pc 00265cfb  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsWindow::set_properties_now(WindowProperties&)+110)
I/DEBUG   ( 2111):     #02  pc 00029ff9  /data/app-lib/com.android.python27-1/libpandagles.so
I/DEBUG   ( 2111):     #03  pc 00265c2b  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsWindow::process_events()+82)
I/DEBUG   ( 2111):     #04  pc 00029fbb  /data/app-lib/com.android.python27-1/libpandagles.so
I/DEBUG   ( 2111):     #05  pc 002556eb  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsEngine::process_events(ov_set<PointerTo<GraphicsOutput>, IndirectLess<GraphicsOutput>, pvector<PointerTo<GraphicsOutput> > > const&, Thread*)+26)
I/DEBUG   ( 2111):     #06  pc 00255cb7  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsEngine::WindowRenderer::do_windows(GraphicsEngine*, Thread*)+26)
I/DEBUG   ( 2111):     #07  pc 0025fa9f  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsEngine::open_windows()+70)
I/DEBUG   ( 2111):     #08  pc 0026037f  /data/app-lib/com.android.python27-1/libpanda.so (GraphicsEngine::make_output(GraphicsPipe*, std::string const&, int, FrameBufferProperties const&, WindowProperties const&, int, GraphicsStateGuardian*, GraphicsOutput*)+1026)
I/DEBUG   ( 2111):     #09  pc 0025aaa9  /data/data/com.android.python27/files/python/lib/python2.7/lib-dynload/panda3d/core.so
I/DEBUG   ( 2111):     #10  pc 0008dec8  /data/app-lib/com.android.python27-1/libpython2.7.so (PyCFunction_Call+296)

I’ve tried putting the system() call in different locations in the C++ program in the hope that Python will find the viewer created by C++ (knowing this was unlikely), or maybe it could find the NativeActivity if launched straight away in pview’s main(). It never does :frowning:

My Python script definitely is importing the Panda libraries correctly. These Python commands work when launched via pview. I tried using ctypes as suggested before to verify the DLLs are loaded, and it worked this time, from inside pview (it didn’t when I ran it from the shell).

	      from ctypes import CDLL
	      CDLL("/data/data/com.android.python27/lib/libgnustl_shared.so")
	      CDLL("/data/data/com.android.python27/lib/libp3dtool.so")
	      CDLL("/data/data/com.android.python27/lib/libp3dtoolconfig.so")
	      CDLL("/data/data/com.android.python27/lib/libpandaexpress.so")
	      CDLL("/data/data/com.android.python27/lib/libpanda.so")
	      CDLL("/data/data/com.android.python27/lib/libp3android.so")
	      CDLL("/data/data/com.android.python27/lib/libp3framework.so")
	      CDLL("/data/data/com.android.python27/lib/libpandagles.so")

	      import panda3d.core
	      from panda3d.core import ConfigVariableString, loadPrcFileData, loadPrcFile
	      from direct.showbase.ShowBase import ShowBase
	      loadPrcFile('/data/data/com.android.python27/files/python/bin/etc/Config.prc')

I’ve checked that plugin-path has been set correctly by writing it out to a file. It does point to all the Panda lib*.so libraries, and libpandagles.so is in there. ExecutionEnvironment.getDtoolName() always returns ‘unknown’ though, which is confusing, even though it knows where to find the library and the CDLL parts worked.

The Python script always gets to test = ShowBase(), but then completely quits and then the pview C++ program continues.

If you run the Python side via adb shell exactly the same as in pview, you can see Python all works, although it’ll fail when you try to do ‘test = ShowBase()’ with the usual claim that there is no valid graphics library to draw with (like I mentioned a while ago), instead of just quitting. That’s okay because I didn’t expect it to find/launch the NativeActivity if I’m launching Python from a PC via the shell, but I thought it might find it if I launched it via Panda’s NativeActivity & pview.

Another way I tried – I followed the Python embedding manual – if I include Python.h in pview, and try to embed Python code, Py_Initialize(); works, but Py_Finalize(); or any other Python-related command causes the main() function to not run at all. Meaning I can’t seem to run a Python script that way either. I’m still not sure if calling Python this way will make the renderer work though!

As usual, if you have any ideas I’d be very grateful!

  • Greg

#155

A bit of additional info that validates I have indeed loaded the renderer, if I launch Python from the shell it seems to find the renderer now, but then crashes with a segfault upon running any other Python code. I didn’t expect this to work but it would be kinda cool if you could control the renderer on the device from the PC shell!

I still have the problem just mentioned though!

Known pipe types:
  AndroidGraphicsPipe
(all display modules loaded.)
:ShowBase: Default graphics pipe is AndroidGraphicsPipe (OpenGL ES).
Segmentation fault

#156

Following the stack trace, it seems like it might be happening in AndroidGraphicsWindow::set_properties_now in panda/src/android/androidGraphicsWindow.cxx. This in particular looks suspect:

ANativeActivity_setWindowFlags(_app->activity, add_flags, del_flags);

You might want to add a cerr line to print out the values of _app and _app->activity like this:

cerr << "DEBUG _app is " << _app << endl;
cerr << "DEBUG _app->activity is " << _app->activity << endl;

This will make sure that those pointers are initialized properly. In particular, since _app is initialized from panda_android_app, it seems like perhaps you are not initializing the panda_android_app properly in the android_main() function? This pointer is required in order to do any kind of interaction with the Android windowing system.

Note that this means that you cannot open a Panda window from a subprocess that is not forked from Android’s zygote. Android just isn’t designed to support that.

It certainly seems that embedding the interpreter is the way to go. Perhaps you can share the code you’re using for embedding so I can take a look at it?

It might be possible to try and run pfreeze to ‘freeze’ the Python sources, which will generate a .c file containing a main() function that preloads the bytecode for the Python modules and embeds the interpreter. This is the approach I’ve taken for the WebGL port, and it may be the easiest one to get started with (and also suitable for production use).


#157

Thanks, you’ve been most helpful, rdb. That was enough of a hint to get things working.

I looked at your/my android_main function, and you were right of course. Additionally, my application was crashing because of a link that hadn’t been correctly set up (by me) in makepanda. It’s weird how Py_Initialize worked if left on its own without any other code, and yet Py_Finalize would cause the built library to silently fail without any info from logcat, but it doesn’t matter now, since it’s fixed. It’s always something simple, but as I’m still learning about how Panda works ‘under the bonnet’ I can miss the obvious sometimes. This is why I haven’t uploaded anything yet, because I keep changing things and experimenting, my code is very messy and will get an overhaul.

Anyway, my Python scripts now run on Android and I now appear to have all of the main libraries working. Just got to get packaging into APKs working, Bullet, ODE, OpenAL and the like.

What do you think?

EDIT: Just to say the artefacts don’t appear on the actual device, it’s just the way Android can’t screenshot the GL background.







#158

Wow, thats pretty cool! Just wondering, how fast does it run?


#159

Awesome! Great work!


#160

If all this works out 2.0 is going to be an epic release, the biggest since I joined back in 1.5.3.