Navigating DirectGUI by keyboard and gamepad

Scripting Issues
1

Is there a “standard” or reasonably elegant way of navigating DirectGUI controls by keyboard and/or gamepad?

(If I’m not much mistaken, DirectGUI has none such built-in–but perhaps there’s a nice way of handling this case.)

I’ve just implemented support for this in a simple project, and, even with only a handful of controls, it feels messy and inelegant: there are at least two types of input to check for (unless I’ve missed a built-in way of generating key-like events from gamepad thumbstick movements); one can directly call “commandFunc” on GUI-objects, but some versions of that method take a parameter, while other’s done; I’m preventing the thumbstick from being too fast via a timer-variable; and it involves messing around with GUI-related stuff in otherwise gameplay-related code.

It feels like there might be a better way, and that seems worth asking after–especially before I attempt anything similar in a bigger project. :/

And I do feel that, in a game that supports playing entirely by gamepad or keyboard, it’s worth allowing the player to navigate the UI in the same way. (Playing by keyboard alone might be sufficiently unusual that it can be excluded, I suppose–but gamepad navigation seems like a good idea, I think.)

2

I had this problem too a few times but haven’t found an elegant way yet. It is quite high on my list of things I want to implement directly in directGUI though.

In the meantime I’ve implemented mouse controlling via gamepads in my project if that’s something you would be interested to as it works for any controls without extending your gui code.

3

I think we should probably implement this in Panda. It should be easy for users to make their menus navigable by gamepad, and if we support that, it is also trivial to support keyboard.

I wonder if this functionality would best be implemented in the MouseWatcher class, in cooperation with PGui. It already is aware of all the different components and their relative position, and is listening for all keyboard and mouse events relating to focusing of different elements.

Do we make it possible to assign two “tab indices” for each element, one vertical and one horizontal, within a MouseWatcherGroup? Or do we allow one to assign a “next” and “previous” item for each item, in each direction? Or do we automatically determine this based on the relative position of the elements?

4

I would love to see this implemented in Panda! :slight_smile:

My main qualm right now is the currently-open question of whether we’re sticking with DirectGUI, or, as per this thread, we decide to move to a third-party GUI toolkit.

That’s encouraging to read! :slight_smile:

That sounds rather good, to me.

Is there a case in which someone might want to jump to a given tab-index? If so, then that might be better; if not, then my feeling is that “next” and “previous” items might be more intuitive–especially if we put together a “DirectGUI visual designer” at some point, in which case we could allow designers to simply “drag and drop” these connections. I imagine that tab-indices could become fiddly in such an environment.

(Although I might make the directions of the connections more explicit–I suggest having “up”, “down”, “left”, and “right” items.)

It would be lovely to have this be automatic, but I can see the potential for cases in which an automatic system misses an intended item (perhaps by not accounting for DirectLabels that have events attached via “bind”, or something similarly unusual), or creates a connection that isn’t particularly intuitive.

Perhaps it might be better to allow both: provide an automatic system that can produce a first-pass at making connections, but also allow a designer to hand-assign some of them.

I imagine that, when selected via keyboard or gamepad, a given control would simply show its “highlighted” state, yes?

5

As for your point which GUI system will be used in the future, this is definitely a good question and I guess there should be a decision sometime soon to make way for getting actual work done on this topic.

Navigating between controls isn’t the only thing one has to keep in mind. You would most certainly also want to be able to change values which, for example for buttons would be easy, but as soon as you get to sliders, scrollbars or textboxes (esp. multiline ones), this could become tricky. There has to be a certain way to check whether the user want to jump to the next control, change the value of a slider or type in a textfield.
As for the tab index implementation proposals @rdb came up with, I guess having an automatic approach in addition to give developers the possibility to overwrite those indexes manually would be the best way. Not sure how automagically this automatic aproach should be, maybe a simple index ordered by the time it was added to the scenegraph would also already be sufficient.

I once did start a document for enhancing DirectGUI which also contains enhanced gamepad and keyboard control which you can find here: https://paper.dropbox.com/doc/Panda3D-GUI-overhaul--AWunxZuuAJaIjK3KhVlH2oIjAQ-Rfyec320c9oUG0mlBqkZT

6

Ah, that’s a good point.

I’m not accustomed to navigating with a gamepad, so take my thoughts following with a grain of salt. That said:

With a slider, does it make much sense to attempt to move to another control that lies in a direction along the axis of the slider? That is, does it makes sense to move to the left or right from a horizontal slider, or up or down from a vertical one? I’m inclined to expect that, when a slider is selected, directional movement along the axis of the slider just moves the slider.

But perhaps there’s some (perhaps de facto) standard approach to this? Maybe another button that can be used to move from a slider to the “next” item?

With text-entries, I imagine that they would capture the keyboard until the “action” button (or enter) is pressed. But perhaps it would make sense to require that the same button be pressed to activate this behaviour, so that they can be easily skipped over?

Again, perhaps there’s some standard behaviour that we could look to.

I would suggest spatial arrangement, myself: compare the positions of the selectable GUI items and connect them based on what is “adjacent” to them: the control directly above is the “up” control; if there’s one to the right, it becomes the “right” control, and so on. There would likely be details and edge-cases to be worked out, I imagine, but that’s the basic idea, at least.

7

I actually do have an example of this. A volume slider right next to a mute button would be one case where this could very well exist.

From other GUI systems that can be controlled via gamepads, I know that some use a way where the controlls kinda have two states. First a selection state, where the user can navigate between the controls and then a Second state which can be entered by hitting the action button (Enter or ‘A’ button on a gamepad most commonly). This second states makes the focused element active and editable. Hitting the action button again will move back to the first state, the selection state again.

With a single line entry yes, but for a multiline text edit you can’t use the keyboards enter or tab to de-/activate the control. A combination like shift-tab might work for those though.
In a Gamepad controlled game you’d probably want to implement your own keyboard to enter text into such fields where you can also add a special finish or done button. Not sure if DirectGUI should provide such a virtual keyboard, though I guess it would be nice.

That indeed would be a nice way and I’m definitely open for that too. I guess the other way may just be a little more easy to implement, though I guess both aren’t that hard to implement after all once we have a way to navigate.

8

Ah, that’s a good point, indeed.

If that’s commonly used by other game-UIs, then I’m in favour of it! It seems like a reasonable solution to the problem, to me.

As long as the “enter” button isn’t also the “action” button, it seems to me that the solution that you mentioned just above should work here, too: the player presses “action” to activate the entry, types, then presses “action” again to deactivate it.

If the “enter” button is the “action” button, then there’s a conflict.

In this case, I might suggest the opposite of what you had in mind: shift-enter would insert a new line in the text-entry, while enter would act as the “action” button and deactivate the control. This is consistent with what I’ve seen in certain chat text-entries, where “enter” sends a message, while “shift-enter” inserts a new line.

Hmm… right now, Panda doesn’t support consoles, I believe, only PC–where a keyboard may, I think, be generally assumed. And in the edge-cases in which a keyboard isn’t present (such as with Android games, or use by people unable to effectively operate a keyboard), a device-provided keyboard may be available.

However, I do admit that there may be cases in which no keyboard is available–especially if Panda branches out into supporting consoles–in which case perhaps an Panda-provided keyboard would be a good idea.

It might be, but I suspect that it might also be less reliable: one doesn’t necessarily add controls in the intended tab-order.

9

That would probably work most of the times, but for textfields where are entering many lines, it would probably be better to map it to something like the escape button which is actually never used for text editing, at least I dind’t know of any textfield that would make use of it.

Panda may not support consoles yet, but there are other setups like home theatre PCs where it’s not usual to have a keyboard in front of you and someone in the community is as far as I know building settings for museums and exhibitions where I expect it to also not always be practicall to have a keyboard.

That definitely is true and would indeed be a pro for the other method.

10

Perhaps–I can also see that it might become tiresome for a very large number of lines.

However, I think that I’d rather have something intuitive for the common case, and then allow developers to override the behaviour for specific edge-cases.)

Ah, that’s a good point–fair enough!

11

A potential issue that I’ve just bumped into is that, when a DirectGUI item is selected by keyboard or gamepad, we presumably want this selection to be visible. One idea for this that occurred to me was to simply set the control into its “roll-over” state–it seems simple at first thought, and is consistent with the extant behaviour of DirectGUI.

However, it looks like DirectGUI is designed in such a way that this isn’t straightforward: attempting to set the state of a DirectButton or similar to “DGG.BUTTON_ROLLOVER_STATE” seems to have no effect. This conclusion is supported by this (admittedly old) thread. (Unless I was using the wrong state-value there?)

(For the purposes of the simple project that I mentioned above, I intend to just use an additional DirectLabel placed “behind” the controls, with a design on it, or something similar.)