how to get multi touch on android

i’m noob, sorry. i’m developing games for both android and windows platform. I was developing it on p3d 1.10 first but i got that only p3d1.11 could build android applications. then i’m using p3d 1.11 for my project. but i got some issues:

  1. how to get multi touch on android platform?
  2. p3d1.11 did not support nvidia cg, and i just know for mobile i could only use GLES. for opengl, filters.setAmbientOcclusion dont work. how could i replace it for the same effect?

this is my first time to develope projects on both android and windows(also p3d 1.11).(i’m weak in english, sorry) i need help, thank a lot!

okay, my account was been held on for a while. after unlocking my account and u saw my post, i’ve found the solution. and it’s exactly perfect.
my solution:
rewrite handle_motion_event in panda3d/panda/src/androiddisplay/androidGraphicsWindow.cxx and build android wheel by myself
as i thought, func handle_motion_event missed the multi touch information from ndk, it only send the first finger’s touch to python script. so i define a new event named ‘multi-touch-ACTION’. thats my code:

/**
 * Processes multi motion event.
 */
int32_t AndroidGraphicsWindow::
handle_motion_event(const AInputEvent *event) {
  int32_t action = AMotionEvent_getAction(event);
  int32_t pointer_index = (action >> AMOTION_EVENT_ACTION_POINTER_INDEX_SHIFT);
  action &= AMOTION_EVENT_ACTION_MASK;
 
  // =================================================================
 
  int32_t pointer_id = AMotionEvent_getPointerId(event, pointer_index);
  float mt_x = AMotionEvent_getX(event, pointer_index) - _app->contentRect.left;
  float mt_y = AMotionEvent_getY(event, pointer_index) - _app->contentRect.top;
 
  if (action == AMOTION_EVENT_ACTION_DOWN || action == AMOTION_EVENT_ACTION_POINTER_DOWN) {
      PT(Event) tevent = new Event("multi-touch-down");
      tevent->add_parameter(EventParameter(pointer_id));
      tevent->add_parameter(EventParameter(mt_x));
      tevent->add_parameter(EventParameter(mt_y));
      throw_event(tevent);
  } 
 
  else if (action == AMOTION_EVENT_ACTION_UP || action == AMOTION_EVENT_ACTION_POINTER_UP || action == AMOTION_EVENT_ACTION_CANCEL) {
      PT(Event) tevent = new Event("multi-touch-up");
      tevent->add_parameter(EventParameter(pointer_id));
      tevent->add_parameter(EventParameter(mt_x));
      tevent->add_parameter(EventParameter(mt_y));
      throw_event(tevent);
  } 
  else if (action == AMOTION_EVENT_ACTION_MOVE) {
      size_t pointerCount = AMotionEvent_getPointerCount(event);
      for (size_t i = 0; i < pointerCount; ++i) {
          int32_t pId = AMotionEvent_getPointerId(event, i);
          float pX = AMotionEvent_getX(event, i) - _app->contentRect.left;
          float pY = AMotionEvent_getY(event, i) - _app->contentRect.top;
          
          PT(Event) tevent = new Event("multi-touch-move");
          tevent->add_parameter(EventParameter(pId));
          tevent->add_parameter(EventParameter(pX));
          tevent->add_parameter(EventParameter(pY));
          throw_event(tevent);
      }
  }
 
  if (action == AMOTION_EVENT_ACTION_DOWN) {
    _primary_pointer_down = true;
  }
  else if (action == AMOTION_EVENT_ACTION_UP
        || action == AMOTION_EVENT_ACTION_CANCEL) {
    _primary_pointer_down = false;

by this way i finally solved my multi touch need, and my project was running well on my test phone. hope to help all of u.

Can you pls share in detail, exactly how you achieved this ? as i am also interested in trying the same. pls :folded_hands:

thanks for visiting my post.
in my solution, i wrote a module “mobile_ui.py” to achieve multi-touch.
i define class MobileControls like this:

class MobileControls():
    def __init__(self):
      self.fingers = {}
      self.is_android = _detect_android() # my custom method
      self._primary_pid = None

       #(achieve the virtual controller)
       if self.is_android: #as a constant
           self.accept("multi-touch-down", self._mt_down)
           self.accept("multi-touch-up", self._mt_up)
           self.accept("multi-touch-move", self._mt_move)
           self._task = taskMgr.add(self._android_block_task, "mobile_gesture_update")
       else: # for desktop
           self.accept("mouse1", ....

as the result, events we’ve defined are now tied to our custom functions. let’s check for the functions.

def _mt_down(self, pid, x, y):
        nx, ny = self._to_render2d(x, y)
        self._press(pid, nx, ny)

    def _mt_move(self, pid, x, y):
        nx, ny = self._to_render2d(x, y)
        if self._tri_tap_enabled():
            self._tri_move(pid, nx, ny)
        self._move(pid, nx, ny)

    def _mt_up(self, pid, x, y):
        nx, ny = self._to_render2d(x, y)
        if self._tri_tap_enabled():
            self._tri_up(pid, nx, ny)
        self._release(pid, nx, ny)

in these cases, we accepted the position from events, and processed it with “_to_render2d”. it is becaused that the position ndk returned is started from the left top, but in p3d (0, 0) is center. and, in android platform, y runs down, but we need it runs up. for _to_render2d:(dont pay attention to _tri_tap_enabled, it’s only for my project to achieve three fingers touch)

def _to_render2d(self, x, y):
        win_w = base.win.getXSize() or 1
        win_h = base.win.getYSize() or 1
        nx = (x / win_w) * 2.0 - 1.0
        ny = -((y / win_h) * 2.0 - 1.0)
        return nx, ny

then let’s check for _press and so on.

def _press(self, pid, x, y):
        if self._primary_pid is None:
            self._primary_pid = pid

        mw = base.mouseWatcherNode
        over_region = mw.getOverRegion(x, y) if mw is not None else None

        if over_region is not None:
            if pid != self._primary_pid:
                self._gui_click(over_region)
            return  

        if self._blocked():
            return
        self.fingers[pid] = {
            'sx': x, 'sy': y, 'cx': x, 'cy': y,
            't0': globalClock.getFrameTime(),
            'zone': 'left' if x < 0 else 'right',
            'mode': 'tap',
        }

in this method, we get finger press event into the fingers[]. first we check for _primary_pid(androidGraphicsWindow: ACTION_DOWN → _primary_pointer_down). then we used _gui_click function. lets check it.

def _gui_click(self, region):
        try:
            gui_id = region.get_name()
        except Exception:
            return
        if not gui_id:
            return
        now = globalClock.getFrameTime()
        last = self._last_gui_click
        if last is not None and last[0] == gui_id and (now - last[1]) < 0.2:
            return
        self._last_gui_click = (gui_id, now)
        try:
            base.messenger.send("click-mouse1-" + gui_id, [None])
        except Exception:
            pass

this method is to accept the second finger touching the gui(like button). it’s important for modern mobile games. if player is moving by touching virtual controller and also touches the skill button, we need to make the two events run in the same time.
(about virtual controller, also named virtual joystick, u can find lots of solution in the Internet. in my case, the different is u need to get touch events from fingers. )

uh, as u know, my english is not good. so if u read these words with troubles, i’m very sorry. u can ask me for the solution any time, i’ll reply to u as soon as possible.

If adding multi-touch is so easy, then can you pls open a PR on the panda3d repo with these changes, so that everyone can have the multi-touch support ?

of course. after i finished my project, i’ll do it. i’m wondering that if you solved ur problem by my solution. if so, reply to me, thanks a lot.

pr’s here: Added Android Multi-Touch Event Support by tiantian520tt · Pull Request #1884 · panda3d/panda3d

2 Likes