Handling Input Devices

Handling input devices is the basis for managing interactivity in your application.

This tutorial covers the basic use of the 3DVIA Studio Device Manager. It goes over the basic modification of a 2D Node appearance using configured keyboard inputs, connecting a sight to the mouse displacements and how a gamepad can be used in a sample.

The Device Manager in 3DVIA Studio

In 3DVIA Studio, the device manager is responsible for managing the different system input/output devices. You can use it to access the keyboard, mouse and gamepad devices by default, as well as develop access to other devices.

In the following example, we’ll use both VSL and Schematic. Please remember, these examples are made to be re-used, modified, leveled up… In the experience we’re to develop, we chose to deal with the inputs through a single actor which we’ll call the InputManager. This actor handles the different inputs and then causes the desired behavior described at the beginning of each part.

Note: we assume you know the basics of authoring a simple sample with 3DVIA Studio (actor creation, property view settings…). If you don’t feel comfortable with notions you may encounter in this tutorial, please do not hesitate to take a look at other posted articles and tutorials. The finished project is available here.

Interact through the keyboard

(30% ~ 10 min)

The focus in this part of the tutorial is to display the arrow pad of a keyboard in the 3DView thanks to four 2DNodes. To accomplish this, we must change the appearance of these nodes according to which keys are pressed.

• Drag and drop 4 actors instantiated from the 2DNodeWithTexture Template in the 3DView (they correspond to the 4 arrrows).
• Also add an “Info Panel” instantiated from a 2D Text Node to explain what the user is supposed to do regarding the Keyboard part of the sample.
• Create a Project Behavior: it will be used to manage the whole sample (we named it “InputHandler”).

After this step, your project editor should more or less look like the following screen shot (assuming you’ve renamed your actors as we did):

In the attached final project, you will find 5 texture files. Four of the files correspond to the 4 arrows, and the last is used to display the “pressed state” of a key. The textures can be found under the project root folder, following the path Sources\Shared\Images. Feel free to use your own resources, it’s no problem!

• Drag and drop these resources in the 3DView. They can become the project’s resources.
• Set the different actors (size, position, textures or text) using their property views.

You should see a similar display in the top, left corner of your 3DView:

Now it’s time to develop the “Keyboard Part” of the Behavior responsible for the input handling:

• Open the behavior you’ve just created in the Behavior Editor.
• Create a new type EKeyPic in the SubTypes section with the following members:

- eKey – vkKeyboard::EKey – defines the keyboard key users press to test the sample.
- eNode2D – vkNode2DPtr – defines the node whose display is affected when the eKey key is pressed.
- eTexture – vkTexturePtr – defines the texture used by the related node when the eKey key is pressed.

• Create two new members in a Keyboard Category to the behavior

- AllKeysList – vkArray<InputHandler::EKeyPic> – defines the information related to each arrow of the keyboard
- DefaultTexture - vkTexturePtr – defines the texture used on all nodes when the related key is not pressed

You should see the following screens:

- in your EKeyPic Subtype definition:

- in your InputHandler Behavior definition:

At this point, we decided to use the schematic editor to implement the behavior. The idea is to check for each key if it’s pressed. If it is, we change the texture on the associated 2DNode and if it isn’t we set the corresponding texture to the default.

You may try to find your own loop to achieve such a result; take a few moments to think about it…but if you prefer getting an immediate solution, continue reading.

Our solution consists of a single loop on the AllKeysList array at each iteration. The value returned by the method vkKeyboard::IsKeyPressed with the corresponding eKey as a parameter, determines what texture (default one or specific one) we place on the associated 2DNode.

• Create a schematic task named Keyboard
• Expose both the AllKeysList and DefaultTexture members through the Target tab in the schematic editor (on the left).
• Create the schematic graph shown in the following screen shot (pay attention to the steps described above).
• Select the ArrayIteratorUse on the Pout tab on the left to expose only the outputs.

The final step in this first Keyboard Part consists of ordering the members to achieve the proper behavior.

• Navigate to the Assemble workset.
• Select the InputHandler actor’s property view.
• Set the array AllKeysList, as partly shown on the following screen shot, and the DefaultTexture member with the resources imported in the project.

Here it is! Play the sample, hit the keyboard arrows and see the effect…

Use the mouse to interact with your experience

(30% ~ 15 min)

The next step, is retrieving data and events from the mouse. In this second part of the tutorial, we must access the mouse events and info via the VSL.

We will illustrate this by developing a task to check whether the left and right buttons are pressed as soon as the pointer passes over a 2DNode:

1. no button pressed: a target follows the pointer
2. left button pressed: rotate the target clockwise
3. right button pressed: rotate the target counterclockwise
4. left and right button pressed: hide the target

The 4^th item allows us to show you a trick for accessing the pointer position quickly in the viewport, since the method may also return the position in the window (web page, desktop screen…).

• Create 2 actors instantiated from a 2DNodeWithTexture template.

• Set their properties to get a similar screen as below (texture files are available in the project folder under the path Sources\Shared\Images).

• Create a new VSL task in the InputHandler behavior, named Mouse.

At this point, you may take a few minutes to think about the algorithm you’re to use to translate the 4 states described above.

We wrote a piece of code which is a solution. Either keep your own or copy/paste ours.

• Create a new vkNode2DPtr member in your behavior, named CursorTarget.
• Set the Mouse task as inactive at the behavior activation.
• Copy/paste the following piece of code in the VSL editor.

// Definition of the task Mouse
task InputHandler::Mouse
{
Target {
InputHandlerPtr be;
};
bool Execute(const vkTaskContext& iCtx)
{
// By default the 2DNode is visible
CursorTarget.SetVisible(true);

// Retrieve the mouse through the Device Manager
vkIODeviceManager& ioman = vkIODeviceManager::Instance();
vkMousePtr mouse = ioman.Mouse();

// No mouse connected: we stop the task.
if (!mouse) {
Print("[ERROR] – No mouse found !");
return false;
}

// Deal the various info from the device
float kSpeed = 0;

if (mouse.IsButtonPressed(vkMouse::EButton::eButtonLeft)) {
kSpeed += 1;
}
if (mouse.IsButtonPressed(vkMouse::EButton::eButtonRight)) {
kSpeed -= 1;
}

// Rotate the CursorTarget
CursorTarget.RotationAngle += iCtx.clock.deltaTime*0.01*kSpeed;

// Both the buttons are pressed, hide the cursor
if (mouse.IsButtonPressed(vkMouse::EButton::eButtonLeft) && mouse.IsButtonPressed(vkMouse::EButton::eButtonRight)) {
CursorTarget.SetVisible(false);
}

// Follow the pointer with the CursorTarget
vkRenderManager& rman = vkRenderManager::Instance();
vkDevicePtr device = rman.GetDevice(0);

vkIVec2 TargetPos;

device.GetPointerPosition(TargetPos);
TargetPos.x -= CursorTarget.Size.x/2;
TargetPos.y -= CursorTarget.Size.y/2;

CursorTarget.SetPosition(vkVec2(TargetPos.x,TargetPos.y));

return true;
}
};

At this point, the task Mouse will never be activated. As we want, this happens only when the mouse hovers over a specific area (the other 2DNode you’ve instantiated before). We need to do a little more work…

• Drag and drop a vkClickable component on the area which will “activate” the task.
• Change the target of this component in its property view by choosing the InputHandler actor.
• In the InputHandler behavior, create two schematic functions:

-OnEnter – triggered by the vkClickableEnterEvent starts the Mouse task using the SartTask BB.
-OnExit – triggered by the vkClickableExitEvent stops the Mouse task using the StopTask BB and puts the CursorTarget at the area center.

Finally, give the CursorTarget member the expected value (MouseCursorTarget in our case) as shown in the next screen shot:

Now, you should have an idea of how to deal with the mouse (play the sample to check that your project is properly set).

Retrieve info from a gamepad

(30% ~ 15 min)

The final part of this tutorial details how to connect the gamepad to your computer. Regarding games, the gamepad is often a must-be-supported device. With your knowledge so far, this last part should be piece of cake to understand and follow!

Enough with 2D nodes, let’s work with 3D models now! You’ll find one model of a gamepad (named XBox_Pad.3dxml) in the attached project folder under the path Sources/Shared/Models. You may use this one or just simply choose one of your own models. The idea here is to simulate the behavior of both the sticks and buttons (A,B,X,Y and Start).

• Drag and drop the 3D model in the 3DView to create the actor of the gamepad.
• Create a new subtype, in the behavior InputHandler, named EButtonPos: it will associate a 3DNode with its default position and the corresponding button of the gamepad.
• Add a new VSL function called Gamepad.

As usual, you may want to implement your own code, but can use our solution. If you choose to use our code, please follow the next steps:

• Create a new Gamepad category and then the following members in your behavior InputHandler:

-5  EButtonPos members named ButtonA, ButtonB, ButtonX, ButtonY and ButtonStart
-2 vkEntity3DPtr members, named LeftStick and RightStick
-1 vkVec3 member, named vectorOffset
-1 float member, named angleOffset

• Copy and paste the following piece of code in the VSL editor.

// Definition of the task Gamepad
task InputHandler::Gamepad
{
Target {
InputHandlerPtr be;
};

void OnStart()
{
ButtonA.StartPos = ButtonA.Node.WorldPosition;
ButtonB.StartPos = ButtonB.Node.WorldPosition;
ButtonX.StartPos = ButtonX.Node.WorldPosition;
ButtonY.StartPos = ButtonY.Node.WorldPosition;
}

bool Execute(const vkTaskContext& iCtx)
{
vkIODeviceManager& ioman = vkIODeviceManager::Instance();
vkGamepadPtr gamepad = ioman.Gamepad();
vkKeyboardPtr keyboard = ioman.Keyboard();

if (!gamepad) {
Print("[ERROR]- No Gamepad detected!");
//return false;
}

if (gamepad.IsButtonPressed(ButtonA.GamepadButton) || keyboard.IsKeyToggled(vkKeyboard::EKey::eA)) {
ButtonA.Node.SetPositionInSpace(ButtonA.StartPos + vectorOffset,null);
} else {
ButtonA.Node.SetPositionInSpace(ButtonA.StartPos,null);
}

if (gamepad.IsButtonPressed(ButtonB.GamepadButton) || keyboard.IsKeyToggled(vkKeyboard::EKey::eR)) {
ButtonB.Node.SetPositionInSpace(ButtonB.StartPos + vectorOffset,null);
} else {
ButtonB.Node.SetPositionInSpace(ButtonB.StartPos,null);
}

if (gamepad.IsButtonPressed(ButtonX.GamepadButton) || keyboard.IsKeyToggled(vkKeyboard::EKey::eE)) {
ButtonX.Node.SetPositionInSpace(ButtonX.StartPos + vectorOffset,null);
} else {
ButtonX.Node.SetPositionInSpace(ButtonX.StartPos,null);
}

if (gamepad.IsButtonPressed(ButtonY.GamepadButton) || keyboard.IsKeyToggled(vkKeyboard::EKey::eZ)) {
ButtonY.Node.SetPositionInSpace(ButtonY.StartPos + vectorOffset,null);
} else {
ButtonY.Node.SetPositionInSpace(ButtonY.StartPos,null);
}

// Left and Right Thumbs

if (gamepad.GetAxisValue(vkGamepad::EAxis::eLeftThumbX > 0.05)) {
LeftStick.LocalAngles.z = angleOffset;
} else if (gamepad.GetAxisValue(vkGamepad::EAxis::eLeftThumbX < -0.05)) {
LeftStick.LocalAngles.z = -angleOffset;
} else {
LeftStick.LocalAngles.z = 0;
}

if (gamepad.GetAxisValue(vkGamepad::EAxis::eLeftThumbY > 0.05)) {
LeftStick.LocalAngles.x = angleOffset;
} else if (gamepad.GetAxisValue(vkGamepad::EAxis::eLeftThumbY < -0.05)) {
LeftStick.LocalAngles.x = -angleOffset;
} else {
LeftStick.LocalAngles.x = 0;
}

if (gamepad.GetAxisValue(vkGamepad::EAxis::eRightThumbX > 0.05)) {
RightStick.LocalAngles.z = angleOffset;
} else if (gamepad.GetAxisValue(vkGamepad::EAxis::eRightThumbX < -0.05)) {
RightStick.LocalAngles.z = -angleOffset;
} else {
RightStick.LocalAngles.z = 0;
}

if (gamepad.GetAxisValue(vkGamepad::EAxis::eRightThumbY > 0.05)) {
RightStick.LocalAngles.x = angleOffset;
} else if (gamepad.GetAxisValue(vkGamepad::EAxis::eRightThumbY < -0.05)) {
RightStick.LocalAngles.x = -angleOffset;
} else {
RightStick.LocalAngles.x = 0;
}

return true;
}
};

This piece of code can easily be expanded on, for example, managing the scenario when when a stick is placed in the top-left corner.

• Finally, set the values of the different members in the InputHandler property view.

• …and PLAY!

Alternative answers…

The tutorial is finished but you may be interested in this next bonus part

The Event System

Another way how to retrieve information from the input devices is to use the inner event system. You’ll find more explanations in the dedicated article, but here are some tips to quickly go through this point.

The Input Manager is actually able to send events regarding the activity of the devices it is in charge of. This is true for the default devices which are already handled by the IODeviceManager but it also can be implemented for the devices you would like to interface with 3DVIA Studio. These events might then be caught by the proper items in a task (a WaitEvent BB for example) or may also trigger functions.

In the case where you want a function to be triggered automatically by an event, you just have to choose the appropriate event from the Behavior Editor (double-click the Triggers column):

The events associated to the default input devices are the following:

• vkIODeviceEvent
• vkKeyEvent
• vkDigitalEvent
• vkButtonEvent
• vkAxisEvent

Then, in order to get the input parameters corresponding to the triggering event, the signature of the function has to be slightly modified. The following piece of code will provide you an example of what to use as a pattern, in the case of a vkKeyEvent:

// Definition of the function KeyPrinter
void InputHandler::KeyPrinter(vkKeyEvent& iEvent) {
Print(iEvent.ch);
}

If you’d like to do the same in a Schematic function, you should create a new member in the pIn section, as shown on the following screenshot:

The Clickable Component

If you want to use this same pattern for mouse click events, choose the appropriate trigger between:

• vkClickableClickEvent
• vkClickableDragEvent
• vkClickableEnterEvent
• vkClickableExitEvent
• vkClickableReleaseEvent

But in this case the IODeviceManager does not trigger automatically all these events (there would be too many of them). Indeed, you have to add a Clickable Item component (from the 3D or 2D sections of the Libraries) on the entities you want to track.

Lastly, if the actor A holding the Clickable Item component and the actor B holding the behavior with the triggered function are not the same, you will have to explicitely specify B as the Event Target of A‘s Clickable Item. If A and B are the same, just skip this step!