how to drive multiple touchscreens

a single touch screen can be used easily with DirectGui by virtue of the ‘virtual’ mouse base.mouseWatcherNode. you don’t necessarily have a lot of movement of the mouse across the screen, but all your button clicks and drags work just fine.

say that i wanted to drive two touchscreens from a single panda application. each has their own PGTop node and independent display region. but i can’t seem to get the button input from the raw mice on either screen. here’s an example:

left = win.makeDisplayRegion(0,.5,0,.5)
leftRoot = NodePath('l')
lens = OrthographicLens()
lens.setFilmSize(2,2)
cam = NodePath(Camera('lc'))
cam.setLens(lens)
cam.reparentTo(leftRoot)
left.setCamera(cam)
leftAsp = leftRoot.attachNewNode(PGTop('l'))
leftAsp.setScale(1,1,base.getAspectRatio())
lm = base.pointerWatherNodes[2]
leftAsp.node().setMouseWatcherNode(lm) #should i be doing
lm.setDisplayRegion(left)              #both of these?

buttons don’t get clicked in this region. also, when i query the mouse position, the touchscreens return crazy numbers, say, (-73.42837,28.098).

i always thought touchscreens acted more or less like mice as far as the software stack was concerned, but maybe i’m missing something?

jeremy

lm = base.pointerWatherNodes[2] 

Should that be pointerWatcherNodes?

yes, that’s my error. the code was not cut-and-pasted. i’m not sure if it will run or not. i’ll try to add some actual code soon, but this is the essence of my problem.

jeremy

i’ve broken this problem down into something much less complicated. after reading the following bits of code and documentation i was inspired.

////////////////////////////////////////////////////////////////////
//     Function: WindowFramework::get_mouse
//       Access: Public
//  Description: Returns the node in the data graph corresponding to
//               the mouse associated with this window.
////////////////////////////////////////////////////////////////////
NodePath WindowFramework::
get_mouse() {
  if (_mouse.is_empty()) {
    NodePath mouse = _panda_framework->get_mouse(_window);

    // Create a MouseWatcher to filter the mouse input.  We do this
    // mainly so we can constrain the mouse input to our particular
    // display region, if we have one.  This means the node we return
    // from get_mouse() is actually a MouseWatcher, but since it
    // presents the same interface as a Mouse, no one should mind.

    // Another advantage to using a MouseWatcher is that the PGTop of
    // aspect2d likes it better.
    PT(MouseWatcher) mw = new MouseWatcher("watcher");
    mw->set_display_region(_display_region_3d);
    _mouse = mouse.attach_new_node(mw);
  }
  return _mouse;
}

the important (imo) bit of the above code is the in-code comment about performing the action ‘so we can constrain the mouse input’ etc.

####  from (the end of) showbase's setupMouse() call
    # Tell the gui system about our new mouse watcher.
    self.aspect2d.node().setMouseWatcher(mw.node())
    self.aspect2dp.node().setMouseWatcher(mw.node())
    mw.node().addRegion(PGMouseWatcherBackground())

the above code is that which my file mimics (or attempts to).

////////////////////////////////////////////////////////////////////
//     Function: PGTop::set_mouse_watcher
//       Access: Published
//  Description: Sets the MouseWatcher pointer that the PGTop object
//               registers its PG items with.  This must be set before
//               the PG items are active.
////////////////////////////////////////////////////////////////////
void PGTop::
set_mouse_watcher(MouseWatcher *watcher) {
  if (_watcher_group != (PGMouseWatcherGroup *)NULL) {
    _watcher_group->clear_top(this);
  }
  if (_watcher != (MouseWatcher *)NULL) {
    _watcher->remove_group(_watcher_group);
  }

  _watcher = watcher;
  _watcher_group = (PGMouseWatcherGroup *)NULL;

  if (_watcher != (MouseWatcher *)NULL) {
    _watcher_group = new PGMouseWatcherGroup(this);
    _watcher->add_group(_watcher_group);
  }
}

‘This must be set before the PG items are active.’ makes me a little nervous.

now i believed that setMouseWatcher() could be used to bind a raw mouse to behave the same way that the system mouse does, and ultimately ‘replace’ the system mouse for a given DisplayRegion. that is, report ‘render2d coordinates’ when the pointer is over the panda window and fail the hasMouse() call when outside the window. here’s what i did (full file):

from pandac.PandaModules import *
## loadPrcFileData('','read-raw-mice 1') #NOTE: i get the same results if this function is called or not
import direct.directbase.DirectStart

from direct.task import Task

def u(task):
    for i,mw in enumerate(base.pointerWatcherNodes):
        if mw.hasMouse():
            print i,mw.getMouseX(),mw.getMouseY()
        else:
            print i
    return Task.cont

taskMgr.add(u,'update')

def useFirstRawMouse():
    base.pointerWatcherNodes[0].clearRegions()
    aspect2d.node().setMouseWatcher(base.pointerWatcherNodes[1])
    base.pointerWatcherNodes[1].addRegion(PGMouseWatcherBackground())

here’s what happened instead. running python and importing this file, then running did the expected thing for base.pointerWatcherNode[0]. pointerWatcherNode[1:], however always print (unbounded) coordinates to console and don’t update those coordinates unless the window has focus.

if you break out to the python shell and then call the useFirstRawMouse() function, then run again… nothing changes. well, that’s not totally true. inspecting the base.pointerWatcherNodes list reveals that system pointer now claims to have 0 regions, while the first raw mouse has 1.

but the data that’s printed to console is exactly the same: 0 shows ‘good’ coordinates as long as the pointer’s in frame and nothing otherwise (whether the window has focus or not) and the other pointers continue to dispaly their unbounded coordinates (which only change if the panda window has focus).

is this the correct behavior? it’s certainly not the behavior i expected.

btw, i’m not sure of the status of unifying the mouse interface across platforms, so let me say this is all under win(32)dows.

jeremy

((bump))

after reading https://discourse.panda3d.org/viewtopic.php?t=3139 i tried a new experiment, slightly modifying the code found in that tread like so:

from pandac.PandaModules import loadPrcFileData
loadPrcFileData("", "window-type none")
import direct.directbase.DirectStart
from direct.showbase.DirectObject import DirectObject
from pandac.PandaModules import WindowProperties
from pandac.PandaModules import MouseAndKeyboard
from pandac.PandaModules import MouseWatcher
from pandac.PandaModules import ModifierButtons
from pandac.PandaModules import KeyboardButton
from pandac.PandaModules import ButtonThrower
from pandac.PandaModules import DriveInterface


class World(DirectObject):
  def __init__(self, name, mouseNum):
    self.name = name
    wp = WindowProperties()
    wp.setSize(400, 400)
    wp.setTitle(self.name)
    ## wp.setRawMice(0)
    wp.setMouseMode(WindowProperties.MRelative)
    
    self.mouseNum = mouseNum
   
    win = base.openWindow(props = wp, aspectRatio = 1)
   
    dname = win.getInputDeviceName(self.mouseNum)
    print self.mouseNum, dname
    mk = base.dataRoot.attachNewNode(MouseAndKeyboard(win, self.mouseNum, dname))
    mw = mk.attachNewNode(MouseWatcher(dname))
    self.buttonThrower = mw.attachNewNode(ButtonThrower(dname))
    if (self.mouseNum != 0):
        self.buttonThrower.node().setPrefix('mousedev'+str(self.mouseNum)+'-')
    mods = ModifierButtons()
    self.buttonThrower.node().setModifierButtons(mods)


    self.mouseWatcher = mw
    self.mouseWatcherNode = self.mouseWatcher.node()

    self.accept('mousedev'+str(self.mouseNum)+'-mouse1', self.leftButtonClick)
    self.accept("mousedev"+str(self.mouseNum)+"-mouse1-up", self.leftButtonRelease)
   
  def leftButtonClick(self):
       if self.mouseWatcherNode.hasMouse():
           print self.name, "/ leftButtonClick /", self.mouseNum
  def leftButtonRelease(self):
       if self.mouseWatcherNode.hasMouse():
           print self.name, "/ leftButtonRelease /", self.mouseNum
   

w = World("first context",1)
w2 = World("second context",2)
run() 

i found that both windows responded to the second mouse’s button clicks, but neither responded to clicks from the first mouse. upon further inspection, the first window responds to the first mouse until the second World’s base.winOpen() call.

also, i have not been able to get one of these raw (i.e. non-system pointer) devices to give (-1,1),(-1,1) coordinates. i’m not sure of the vocabulary here. are these coordinates i’m after the ‘relative’ or ‘absolute’ coordinates? or is that a separate nomenclature from raw/‘not raw’ mouse mode?

after grepping through the source and trying to read snatches here and there i’m starting to think that what i’m trying is not supported under panda for windows. another straw on the camel’s back is a paper/software stack (http://klafbang.eu/personlig/publications/mouse.pdf) which claims that what i want is not natively supported or particularly easy in windows.

has anyone out there used panda to draw pointers driven by two different physical mice under any operating system/version of panda? if you’ve done it in one window, that’s great and i want to hear about it; but if you’ve had one mouse point in one window and a different mouse point in a different window, that’s what i’m really after.

thanks!
jeremy

i know this is bad form, but i have to bump this thread again. i’m at a loss here, and getting pretty desperate. it seems plain that noone has done what i’m trying, so could i ask for an official ruling from david, etc on whether or not the spirit of what i’m trying to do is theoretically supported by panda? the manual alludes to a much different (and fairly straightforward, in my experience) use of multiple mouse devices; is this the only supported use?

i’d love to see a reply like ‘dude, i ran your code. didn’t work for me either’, even.

i spent some small effort trying to get the ‘demo’ working in linux, but ioctl failures, permission trouble, etc made me realize that linux is a last-ditch answer, since there’s much more platform-level work i’d have to put in to bring what is effectively a complete project up to speed on a new system. i’m not unwilling to do this – i just want to know that i have to before i spend all that time/effort.

i’m confused about the (lack of) response to this topic, considering how with it drwr, pro-rsoft, thomasegi, etc are about replying to posts. does this mean that you’re furiously working away trying to get an answer to this question? or are you hoping that, if ignored long enough, it’ll just evaporate?

jeremy

Hi Jeremy,

Sorry, I haven’t been ignoring you. The simple fact is, I don’t know the answer. I’ve never played with multiple mice within Panda, and never even tried to plug multiple mice into one PC.

The multiple-mouse support you see there now was put in the by Josh Yelon, a former Panda developer; he was using it to support multiple pointing devices within one window, on a Windows system. I don’t know how well it supports multiple pointers for multiple windows. I also don’t know whether it works at all on Linux.

I do know that with the multiple mouse support Josh put in, the secondary mice are all run in “raw” mode, which means they are unconstrained to [-1…1] and allowed to roam as far in any direction as they like. I don’t know what this means for a touchscreen, which doesn’t have continuous motion in the same way that a mouse does, but it’s possible the logic in Panda to handle “raw mode” is confused by a touchscreen input.

Another thing I’ve been realizing lately is that Panda’s mouse-input system is due for a complete overhaul. Between Josh’s “raw mode”, and FPS-style lookaround support, and the iPhone’s multi-touch input, the existing system has been stretched much farther than its original design, and it needs a good reboot to make things sane again.

Would you like to contribute to the redesign effort? It seems you’ve accumulated some useful experience down in the device layer recently. :slight_smile:

david

ok, i figured this was the case. i’ll keep plugging away at it and keep everyone posted.

i didn’t realize that josh left. who is organizing the panda effort at cmu now?

i’ll help unify the input system, yes. i’d love to see a plugin architecture for creating third party input device wrappers for joysticks, multi-touch tablets – even the ARToolkit – that can control panda in a natural, ‘seamless’ way.

of course, these are just my personal goals. i want to hear the actual design for the system, too, and see where i can lend a hand.

jeremy

They’ve been running headless for a little while now, but I understand they will be hiring a replacement soon. Things happen slowly in a university setting. Pro-rsoft has been picking up some of the slack in the meantime and managing the releases and such.

David