This is the "important" part of my proposal, put here so others can give me feedback on different parts of it.
Did you select a project from our ideas list? If so, why did you select this project?
Yes, I've selected the "GUI Navigation" project. One of the reasons was that my Bachelor project involves some research in the area of computer input (e.g.: mouse, keyboard) and I will be able to put in practice some of the results I get in the research. It also occurred to me that the idea was a nice one.
Please give an estimated timeline for your work on the project. If you have any exams, vacations or other things that will keep your from programming that you know about already, please include them in your schedule.
The following table will show the estimated timeline for the tasks required on the project, down to a work week’s granularity. In the exams weeks and the Bachelor Thesis one, little or no work will be done. However, the impact on the project will be minimal, because there are three weeks of community bonding, before the actual work commences. So, I will start a bit earlier (that is, in the community bonding period) with the project implementation. The tasks listed in the timeline are detailed in the technical section afterwards.
27.05 - Get Accepted
27.05 07.06 - Demos.
10.06 21.06 - Exams
24.06 28.06 - Input Aggregator (implemented alongside the input injection system)
01.07 05.07 - Bachelor Thesis
08.07 12.07 - Input Aggregator (implemented alongside the input injection system)
15.07 26.07 - Remove the old input injection system
29.07 09.08 - GUI Navigation
12.08 30.08 - Focus management
02.09 06.09 - Visual Feedback
09.09 13.09 - Write tests & documentation
16.09 20.09 - Buffer week. Used in case some tasks fell a bit outside the proposed time.
Please give as much technical detail about your implementation plans as possible.
The following is a breakdown per project parts, each dealing with a certain part of the overall project.
Project part: Demos for the new functionality
For this, I will create some demos which will be used to show the implementation. The demos will be used also to test the implementation while the work is in progress.
- Form demo
-- Show a form with multiple labels and their respective editboxes, some checkboxes (`ToggleButton`) and fictive 'submit' and 'clear' buttons.
-- The user can press `tab` while in a editbox to select the next editbox, checkboxes or button.
-- When the focus reaches the 'submit' or 'clear' button, and the user presses `enter` the button's specific action will be executed.
- Menu demo
-- Show a simple menu with different items (a listbox with listboxitems or multiple buttons stacked vertically).
-- The user can press the `tab` key, or use the dpad to switch between the menu items, and use `enter` to activate the current focused one.
- 2D Menu demo
-- Show a menu comprised of buttons placed in a 4x4 matrix
-- The user can press the left/right/up/down keys, or use the dpad to switch the focus between them.
Project part: Input Aggregator
I propose an input abstraction so we can use the GUI in environments where we don’t have a keyboard and mouse. This will imply changing much (if not all) of the existing input architecture.
First of all we won’t use anymore the raw inputs we were used to (e.g.: injectMouseDown, injectMouseUp), but rather `Input Events`. An `InputEvent` is the basic input unit CEGUI will use, and that is generated from raw input. The input event is a basic POD class (just like the current EventArgs) that contains different information based on where it came from and from what raw input it was generated.
Some examples of input events are:
- • MovementInputEvent
- o Generated from mouse x/y
o Generated from joystick/thumbstick x/y
- o Generated from mouse button
o Generated from keyboard button
o Generated from a gamepad button
One can see that this solution leads us to allow a variety of input to be mapped to certain input events. Of course, each different input event will have some specific payload (the button pressed, the x/y coordinates of the movement, etc.).
Now, on the API part the window will have a single method: `injectInputEvent(InputEvent)`. With the use of “Int-To-Type” idiom, we can easily and at compile-time (maximum performance) dispatch the events to their respective handlers (without using dynamic_cast). Moreover, we get infinite (ok, it’s really a maximum of sizeof(int) but it’s enough) types of input events. The user can create new input events inside his custom controls as he sees fit, and just creates handlers for them, and they will be called accordingly by our system in a transparent way.
Then, after we’ve created the new input representation we need a way to generate those from raw input. For this, we’ll have the so-called `Input Aggregator` which will be able to aggregate raw inputs from multiple input devices into input events. In order to define the way we get those inputs, we will have an interface for each type of input device supported, and attach it to the aggregator dynamically as required by the user. This “supported input device” wording shouldn’t scare us. I am not talking about actually implementing the support for those input devices, but rather just defining the interface we require to be implemented so we can use those input directly, without having the user emulate it (e.g.: emulate the mouse movement from the input of a joystick maneuver).
In order for the library to work with some basic input events out of the box I will create a default `Input Aggregator` implementation for mouse and keyboard (and if time allows it for gamepad). Basically what is required to be done is: take the current methods used in the `InjectedInputReceiver` and move them in device-specific files (e.g.: MouseInputReceiver, KeyboardInputReceiver). In there, we’ll generate some input events from the raw input, to be used by the CEGUI library. Then, we’ll add the instances of the receivers to the input aggregator instance. And, at each update step in the game, when the users wants, it will take all aggregated input events from the input aggregator, and send each one of them to the `GUIContext` of CEGUI. Of course, the user can create new `InputReceivers` for each device they want – or replace the default ones - and add them to the aggregator.
At the end of the day, the samples that exist now should work (from the functionality point of view) the same way after the input system is replaced.
Project part: GUI Navigation
For the GUI navigation, the first thing that we need is to allow a `Window` to receive focus. The actual navigation in the UI will be made by focusing different widgets in the GUI tree at specific time. For this, we’ll add two more methods to the `Window` class:
- focus();
- unfocus();
At the basic level, the focus() method will just activate in the associated renderer the ‘focused’ state which can be configured in the look&feel of each widget.
Related to the actual navigation, the first proposed version of the navigation system was a linked list. That wouldn’t be too flexible, because we should be still be able to navigate in a non-linear fashion (e.g.: from a specific focused widget we can navigate in four directions, not just backwards or forwards). Then I proposed to allow the user to navigate in four direction, but that was still limited to four points. What if the user wanted eight directions, or totally different, like jumping in the L shape of the chess knight? With the 3rd iteration of the system, things are now left in the hands of the user, to be customised it like they want.
There will be a new class, attached to each `GUIContext` instance, called `WindowNavigator`. This will hook into the events the GUIContext gets and will `do the math`, which means it check if a new widget to activate. Here, we have two questions to be answered: When and What?
The when is a simple one. When we construct the navigator we’ll give it a mapping that will contain pairs like the following: <input event, payload>. The input event was presented in the previous section. The payload is a simple tag, which can be (to be decided) either a simple string, or a void* object that contains different info. What we do with this payload? Well, pass it on the next step, the “what”. We can already see the power of customization in the hand’s user.
But how will the navigator know what widget to focus? Well, we’ll introduce a helper, `NavigationStrategy`, that the navigator will query on a per-needed basis, to see which control to focus next. This way, by splitting the tasks, we (the user) are able to replace either one of the navigation parts (when/what), or both of them. This strategy will be using another mapping to base its logic on. The mapping will use the payloads we get from the previous step (the “when”) and will compute the next item based on the current selected one. The implementation will be user specific. For example, here we can supply two default implementations: a linear and a matrix mapping. The linear will navigate like in a linked list, while the matrix mapping will allow 2D movement.
Like we did for the input aggregator, we will provide some default-ish navigator mappings and strategy. For the mapping, we’ll have things like:
- pressedInputEvent(left_key), “left”
- pressedInputEvent(right_key), “right”
- pressedInputEvent(tab_key), “next” (fallback to left/right or top/down depending on the navigation strategy).
- pressedInputEvent(CTRL+tab_key), “previous”(opposite to the ‘next’).
For the strategy, we have a linear mapping, meaning that we’ll just return the next/previous child in the GUIContext’s children list.
To decide how to deal with: auto-focusing when in certain contexts.
Project part: Focus Management & visual feedback
Since per each `GUIContext` we have an instance of the `WindowNavigator`, that means there can be only one focused element at a time.
The edit boxes can be seen that they are focused because of the caret which is visible inside it. However, on other controls, like the button, there’s no “obvious” way of showing it. So, I propose that we add another layer of customization, by providing a new state, called “Focused”. This should be customizable from the .xml files just like the other states (Enabled, Hover, etc.).
(Optional) Project part: mnemonics & buddy system
Another form of GUI navigation would be the use of mnemonics. This way, the user can press a specified key combination (ALT modifier + a letter) and the specified item would activate. For a menu item that would seem obvious. However, in the case of labels, not really. But, if we add the so called “buddy system” (more info on the Qt Wiki), we suddenly have a very potent system. Basically, what this will allow us is: we can specify a mnemonic for a label, then set its buddy (a `NavigableWindow` element). When the label is activated by the mnemonic, we automatically focus its buddy instead. This will give us a greater degree of flexibility in navigating the GUI.
