Discussion: sdl2.ext2.joystick?

[knghtbrd]

Hi,

I've noticed there is not yet any remotely Pythonic API for the Joystick and GameController features of SDL2, and I'd like to change that. To that end, I wrote some stuff for my own little skeleton engine (could probably extract it in about 20 minutes if desired) that listens for joystick/controller events and puts the data in something more accessible to Python.

For those who haven't played with this code in C/C++, I'll explain: In SDL 1.2, there was a function you'd call to get the number of joysticks and you could iterate from 0 to that to open and return an opaque SDL_Joystick * sttructure. You could then call joystick functions with that pointer to poll the various axes, hats, buttons, and trackballs (although I've never seen something with a "trackball", and even my PS4 controller doesn't expose one.) Of course mapping any of these axes, buttons, hats, etc. into something sane is pretty much impossible. And these devices even in SDL 1.2 were often USB, which means hotplug is a problem.

Right, so SDL 2.0 kept this old API more or less intact, but introduced two new ones. First is a set of events, as with key, mouse, and other events. The sdl2 package exposes these to Python:

sdl2.SDL_JOYDEVICEADDED
sdl2.SDL_JOYDEVICEREMOVED
sdl2.SDL_JOYAXISMOTION
sdl2.SDL_JOYBUTTONDOWN
sdl2.SDL_JOYBUTTONUP
sdl2.SDL_JOYHATMOTION
sdl2.SDL_JOYBALLMOTION

All of these save SDL_JOYDEVICEADDED return an "instance ID". An instance ID can appear and disappear, and no two devices (or even the same device disconnected and reconnected) will share an instance ID. I think it's a Uint32, so there is a theoretical limit, but not really a practical one. The device index returned by SDL_JOYDEVICEADDED is guaranteed to be in the range 0 to SDL_NumJoysticks() for compatibility with code ported from SDL 1.2.

You can actually ignore all of these events save the first two, open a device on SDL_JOYDEVICEADDED and save the instance ID and the SDL_Joystick * handle. You can then just use SDL's polling functions once per frame to query the specific controls that you want. This is reasonable in C, but it seemed kind of clunky in Python so I didn't do it.

What I did was this:

class Joystick:
    """Tracks the state of one SDL_Joystick."""

    def __init__(self, device_index, open_joystick=True):
        """Creates a joystick structure that can be polled.

        The open_joystick argument is intended to create an empty Joystick
        structure without doing anything else. (Convenience for GameController
        later on.
        """
        self.id = None
        self.name = None
        self.axes = []
        self.buttons = []
        self.hats = []
        self.balls = []
        self.guid = None

        if not open_joystick:
            return

        self.jdevice = sdl2.SDL_JoystickOpen(device_index)
        if self.jdevice:
            # Note intance ID is not the same as device_index
            self.id = sdl2.SDL_JoystickInstanceID(self.jdevice)
            self.name = sdl2.SDL_JoystickName(self.jdevice).decode()
            num_axes = sdl2.SDL_JoystickNumAxes(self.jdevice)
            self.axes = [[str(idx), 0] for idx in range(num_axes)]
            for idx in range(num_axes):
                self.axes[idx][1] = sdl2.SDL_JoystickGetAxis(self.jdevice, idx)
            num_buttons = sdl2.SDL_JoystickNumButtons(self.jdevice)
            self.buttons = [[str(idx), False] for idx in range(num_buttons)]
            num_hats = sdl2.SDL_JoystickNumHats(self.jdevice)
            self.hats = [[str(idx), 0] for idx in range(num_hats)]
            num_balls = sdl2.SDL_JoystickNumBalls(self.jdevice)
            self.balls = [[str(idx), 0, 0] for idx in range(num_balls)]
            guid = bytes(sdl2.SDL_JoystickGetGUID(self.jdevice).data)
            self.guid = uuid.UUID(bytes=guid)

    def close(self):
        sdl2.SDL_JoystickClose(self.jdevice)
        self.jdevice = None

That's the whole class. I could possibly poll the states of the buttons and whatnot, but I'm not doing that currently. The Joystick class is just a data bag, though. The event code throws joysticks into dictionary of joysticks with instance ID keys. Simple enough. Too simple? Too clunky?

Now the elephant in the room: Joysticks are insane. You can't assume anything about a given joystick beyond the first two axes are PROBABLY the first stick, if there are any axes at all, and I would not guarantee even that much. If there's a digital D-Pad, it might be mapped to axes, it might be a hat, or it might be four buttons. You have no idea.

This insanity is why someone from Valve wrote the GameController API. The basic functionality of controllers is pretty much established now. You have four face buttons, two shoulder buttons, two more shoulder controls (buttons or analog triggers), a D-Pad, and two or three buttons in the middle corresponding originally to Nintendo's start/select (but now often start/back/share/whatever) and if there's another button, it's basically a "home" or "system menu" button. Basically an XBox 360 and/or Playstation 3 style controller.

And even if your joystick device doesn't have all of these functions or has more than is included, it can probably be mapped to a standard controller somehow. And just about anything resembling a gamepad seems to have a mapping already. There's a very similar (but reduced) set of events for handling a GameController—reduced because a GameController always has six axes and a finite number of buttons, always in the same order.

Really what SDL does is open the SDL_Joystick (which is refcounded for memory management purposes) and compares its platform-dependent GUID against a database of mappings. It can also read mappings from a file or an environment variable. I don't know if anybody ever wrote software to let you map a joystick into a GameController outside of Steam BigPicture mode … I got busy with grad school as this was being discussed a decade or more ago, so I didn't write it. And I didn't really implement the API needed to do it in my code.

I won't include my Controller class here because it's basically a descendant of Joystick that calls Joystick.init to create an empty structure, then adds a cdevice and populates the buttons and axes.

The sdl2.SDL_CONTROLLER_DEVICEADDED code throws each opened Controller into a controllers dictionary with the same format as joysticks. The reason for that is I wanted to make it easy to access a Joystick with the raw controls and the Controller which is mapped and predictable. The instance IDs are the same, the SDL_Joystick * handles are the same, and the GUIDs are the same. (In fact, to get the instance ID and GUID when you create the Controller, you have to access the SDL_Joystick * to do it.) Unless your code has reason to check, it can just be written to handle a joystick and a GameController will look to be a joystick in a familiar and predictable format.

This doesn't quite seem ready for inclusion in sdl2.ext to me, but it'd be really nice if something were. So … how should I improve upon this before submitting it?

FWIW, my notion of how to use GameControllers is to simply open every one I can find and for the program to tell the user to press start on one of them. When they do, that's player 1. For TMNT for example, player 1 would press start and then select their turtle. If anyone else wants to jump in, they can press start and left/right to select theirs. I think you get the idea on that.

Anyway, suggest away on the API! I have working Python Joystick/Controller code. It can be modified to do what is desired, test it a bit, then submit it.

[knghtbrd]

Oh … I stick a string for the name of the control in each item. That's because my program does a thing where it prints the controls and labels them. I override the (text) names of the GameController API to L1/R1/L2/R2 so the labels are compact.

Dictionaries instead?

[a-hurst]

Wow, this is super-informative. Thanks!

Yeah, the Joystick API is definitely... something. I thankfully haven't had to use it much myself, but as a consequence it hasn't gotten the attention in pysdl2 that it deserves. I agree that there should be a .ext module for both (or at least GameControllers, for the sake of simplicity), and I think your proposed class is a good start.

I guess my main question is how Joystick and GameController events are handled: do they default to going through the main SDL Event queue with keyboard and mouse events? If so, how would a .ext class handle this to receive events while avoiding flushing the event queue? Would you just set it up to peek gamepad events without pumping them, or would you need to use the SDL_JoystickEventState and SDL_GameControlerEventState functions to disable joystick/gamepad events and then use their respective "update" functions internally within the class to get new data? Alternatively, would it be better to use the SDL_JoystickGetButton, SDL_JoystickGetAxis, etc. functions to query axis and button states directly?

If the Joystick/GameController events can be separated from the main event process, the class could have an update or pump method to refresh the data for that controller. I'm not sure whether that would cause problems with multiple controllers, however (if updating one controller would inadvertently update all others). Maybe it would be better to have that as a global function of some sort, in that case (e.g. update_controllers()).

I like the idea of dictionaries for accessing values, as well as the the idea of methods for accessing those values (e.g. controller.get_button("Y") == True or controller.get_buttons() == ["Y", "X", "L2"]). I'll think on this a bit more this week while I'm doing the 2.0.14 updates!

[knghtbrd]

GameController and Joystick events do go through the main event queue. I'm still not sure how I want to handle these things in my own code. In C, you basically just wind up with a massive switch block testing for the kind of event. It makes for one very long function that would be rather ugly in Python. Something a lot of people seem to do is have an on_key_event, on_window_event, etc. Not sure I 100% like that solution either.

Anyway, what a lot of C libraries do when they need to look at the event queue is use SDL_AddEventWatch. Your callback would be called when the event is added, so it should probably return quickly for an event that it isn't interested in.

Another thing to consider is that SDL has a polling-based API for a controller and I literally literally turned the event-based API into a polling-based one above. Possibly not ideal. Maybe a better solution could be to implement something that uses the polling API except for adding/removing?

Generally for any device that is a GameController, you should probably prefer that interface over the Joystick interface, unless you know you don't want to. MOST Joystick devices get mapped to GameControllers sooner or later either by virtue of XInput or XBox pad emulation, or because someone's already done the mapping via Steam. However it's entirely possible to write a program that can create a GameController mapping from a Joystick or allow you to remap an existing GameController. That might be a worthwhile app to write as an example maybe.

[a-hurst]

@knghtbrd No clue if you're still actively using PySDL2, but I've recently made a good deal of local progress working on a Joystick/GameController API for PySDL2 (needed it for a personal project).

For the sake of simplicity I think I'll start with a wrapper for the GameController module since that's a bit more standardized and has a simpler/friendlier overall API relative to SDL2's Joystick module, but adding a second class for joysticks should be fairly straightforward afterwards.

If you're still interested, I'll ping you when I create the PR for this if you'd like to give feedback before it's merged!

[ordovice]

I am totally interested in this as well, as I'm working on interpreting SDL values for GUIDs from a linux implementation in Batocera Linux to the current GUIDs in use by applications like Ryujinx (and I'm sure Yuzu isn't far behind), as they are using newer SDL as well as abstraction of the current GUIDs into a generalized GUID for each of the bigger name/type controllers (for example, all DS4 models - generic and otherwise) appear to use the same GUID for these apps. And since the system wants to pre-configure controllers I'm trying to write the config files ahead of time. So please, feel free to tag me when your PR is in and youve got a working POC.