Add support for controller-based motion controls (gyroscope, accelerometer, …)

[pie75]

Describe the project you are working on

Non-VR Motion-Control-centric First Person Shooter.

Describe the problem or limitation you are having in your project

Godot does not support reading motion sensor output from non-mobile devices. (Android/iOS)

Describe the feature / enhancement and how it helps to overcome the problem or limitation

We would aim to extend godot's current Input format, either via core, or a module,
Adding Motion Controls: gyroscopes; accelerometers, as Vector3 in GDScript, and a separate class within the Input system, would be a natural way to handle this.
However, a more generic implementation would be to make available the individual motion sensors as axes within godot's existing axis system. These can later be combined into a Vector3, but will be available and function identically to an axis.
The developer having access to these hardware inputs via GDScript would make implementing motion controls natural.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

One way to do this is through binding an external library, such as JoyShockLibrary or the Input sections of Simple DirectMedia Layer, which has supported the same featureset as JSL since 2.0.14.
Alternatively, extending the godot core by replicating the features of those libraries natively is plausible.
Additional functions for managing these inputs, such as calibration, and 3D math, can be important, which can be managed by a module based around GamepadMotionHelper.

The Ideal scenario is one where a developer can simply handle the motion controls the same way as they would any axis, having both the actions system and direct access available to them, like this:
var p1_gyro = Vector3(get_joy_axis(Gyro_X), get_joy_axis(Gyro_Y), get_joy_axis(Gyro_Z)) or
var p1_gyro = Vector3(get_joy_gyroX, get_joy_gyroY, get_joy_gyroZ)
the Gyro_ structs or get_joy_gyro function might be just a binding for something like float JslGetGyroX/JslGetGyroY/JslGetGyroZ(int deviceId) from JSL, but it should function similarly.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This enhancement cannot be achieved without changes to godot.

Is there a reason why this should be core and not an add-on in the asset library?

Matters dealing with hardware may be best implemented at core level.

I'm personally willing to work on developing this feature myself, but it's been a few years since I've programmed anything, and I'm not sure whether core changes or modules are the better way to go about this.
I imagine a combination of both has the most longevity. Separating GMH into a module is definitely more reasonable. However, one might argue that the entire gyro functionality might be superfluous for many users and should be modular.
I'm making this proposal mostly to gauge support and get some advice from godot developers on what to do first, before I start doing something completely wrong. Mainly whether module or core should be the focus.

Thanks for reading this far, and also thanks to JibbSmart for making so many tools for motion control development.

[JibbSmart]

Hi! I'm the author of JoyShockLibrary and GamepadMotionHelpers, created and continue to contribute to JoyShockMapper, wrote the best-practices for good gyro controls on GyroWiki, and extended SDL's gyro and accelerometer reading features to support Switch controllers.

I'd LOVE to see gyro and accelerometer as first-class input features in Godot. Gyro controls are increasingly popular, and provide a path to controls in games that are both drastically easier to learn and have far more room for mastery (compared with stick-only aiming controls).

I'm not familiar enough with Godot to propose an interface or how it should affect the current input system, but I'd love to help with implementation details. This is essentially what I did with SDL.

[NicholasShatokhin]

Does JoyShockLibrary support the PS5 gamepad?

[NicholasShatokhin]

I think, you need to add additional axises to Input options for gamepad like it done for the stick axises but for gyroscope and accelerometer.

Another option, the input event like mouse input events.

[NicholasShatokhin]

And please, allow me to use all three axises, not only x and y coordinate on screen. It will allow to implement amazing Overwatch-style controls (Z-axis is the same as X-axis but with much greater sensitivity)

Also, please add android platform because almost all modern phones have gyroscope and accelerometer too.

[Calinou]

@NicholasShatokhin Please use the Edit button (hidden behind the … dropdown next to your comments) instead of multi-posting.

[JibbSmart]

@NicholasShatokhin JoyShockLibrary does support PS5's DualSense, but I usually recommend SDL2, since it supports more controllers overall and will support the same Switch controllers as JoyShockLibrary soon (I've gotten a little involved there).

We'd definitely expose all 3 gyro axes and all 3 accelerometer axes. I like some of what Overwatch did, but there's lots of room for improvement there :)

[NicholasShatokhin]

I usually recommend SDL2

I tried to read gyroscope/accelerometer value using SDL2 in past year and I didn't find any way how to do it. Do you have a manual?

[Calinou]

Alternatively, extending the godot core by replicating the features of those libraries natively is plausible.

Note that this would be the preferred course of action, as Godot doesn't use SDL and linking against SDL just for gyro functionality is overkill. It would significantly increase the binary size, and keeping a low binary size is important to us.

(Copy-pasting code from SDL is fine, if you need to do that.)

[JibbSmart]

@Calinou Sorry, I'm a bit of an outsider on this one. How does Godot handle mapping across different kinds of controllers? For example, PlayStation, Xbox, Logitech controllers all report buttons in different orders, so the application can't actually tell if the "south face button" was pressed. SDL has an ever-growing database of controller mappings to solve this. If Godot doesn't already have a solution for this, we could kill two birds with one stone.

I understand wanting to avoid increasing the binary size (SDL's ~2MB is not negligible), but natively supporting Switch controllers, for example, is a lot of work.

@NicholasShatokhin 2.0.14 added:
SDL_GameControllerHasSensor()
SDL_GameControllerSetSensorEnabled()
SDL_GameControllerIsSensorEnabled()
SDL_GameControllerGetSensorData()

These currently work with PlayStation's DualShock 4 and DualSense, and the next update should have Switch Pro and Joy-Cons (mapped to the same coordinate space as PlayStation).

[NicholasShatokhin]

@JibbSmart Awesome! Thank you. I'm waiting for the next update :)