"Library" version, for incorporating into other interactive programs?

[doctea]

Hi, this looks really useful, thanks for sharing it. I've been working on a Teensy-based sequencer/arranger/Eurorack/USB/MIDI-interface thing for a while, and already have some BeatStep functionality in it (over here) - for example I can advance the currently playing pattern, change pattern length and direction etc.

I'm interested in adapting your code to handle some of the trickier bits that you've already got working (loading/saving presets would be quite handy, since I find the Arturia software very clunky!). I can see there may be some changes needed to allow this (eg making the getter asynchronous, instead of sleeping while waiting for a response, so that it can be used in an interactive context).

Do you have any plans to take it in a direction like this already, or would I just be better off forking and adapting the code I need?

Cheers,

[konsumer]

Hi, this looks really useful, thanks for sharing it. I've been working on a Teensy-based sequencer/arranger/Eurorack/USB/MIDI-interface thing for a while, and already have some BeatStep functionality in it (over here) - for example I can advance the currently playing pattern, change pattern length and direction etc.

Ooh, looks cool! Does the update-flashing stuff work? Mine doesn't quite work, and that would be sick!

I made a js library/cli at one point, that I am planning on sunsetting once I get this one packaged up in a native js module. It was the first ideas around this stuff, and has a cool alternative sequencer (you can see video here) which uses sysex to make the device behave in a more useful way (at least for me, for making beats.) I liked how it worked, but the native js stuff was really flakey for windows builds and the CLI tool it creates is way bigger than it needs to be (it embeds a whole node interpretor, which is just silly.) I think doing it directly in C++/rtmidi fixes a lot of these problems.

I can see there may be some changes needed to allow this (eg making the getter asynchronous, instead of sleeping while waiting for a response, so that it can be used in an interactive context).

The sleep seems to help with a few things, and I was doing a more async pattern before in js, but it would sometimes miss some of the messages waiting for responses async, at least the way I was doing it, so sleep might be the best way (it's more similar to how the official stuff does it, as far as I can tell.) I think underneath it all, rtmidi does a sleep + poll to make async work right, so although it feels very different in code, underneath it, it's roughly the same. If yours gets around this, we can totally incorporate that, but I couldn't get async call/response (like to get) to sync up reliably. Maybe some sort of async queue ("I'm waiting for these sysex answers, trigger the callbacks when they are found") would help, but I haven't figured it out.

Do you have any plans to take it in a direction like this already, or would I just be better off forking and adapting the code I need?

I am not as strong with C/C++ as other languages, but these ideas sound good. I can make stuff with it, and have a few projects I maintain that are mostly in C/C++ (like node-raylib is maybe my most popular) but it goes really slow for me compared to rust, python, js, nim, or almost anything else. Every time I work on the C stuff, it feels like a bit of a slow-grind. I originally planned to make this as a basic CLI for testing first (which I think is maybe done, minus the firmware-update stuff) since rtmidi is a C++ lib, and I thought it would be easier to work out the initial prototype, then port it to something I like better (nim and rust both compile to tiny & efficient native code, and are more fun to work with, for me.)

Since it goes so slow for me, you might want to fork it, but I'd also be totally into giving you commit access, if you want to just add stuff to it, and be a co-maintainer. I had some other (possibly unrelated) ideas, too, like using webmidi directly, in the js lib, so I might just improve the native node midi bindings (like make a better rtmidi binding for JS, that works better on modern versions) and use those from js. I'm really not sure. I have also thought that maybe PortMidi would work better than rtmidi, so that might be worth exploring.

If you want to work together on it as a C-lib, here is my next goals for this:

• Get firmware update working. I could not figure out how to capture firmware-update traffic, so never really worked out how to send .led files, but this would be useful for linux people, where there is no official support, and also I found mac M1 to be really flakey with official app
• Package it for better Cmake use. If people use it for C/C++, this would make their lives much easier
• Explore PortMidi. If it's better, maybe use that, instead.
• Async: I like this idea a lot, but never got get working

Longterm beatstep goals:

• Reverse-engineer the firmware, so we can send it custom firmware. I always wanted to get an alternative sequencer running on the actual device (instead of sysex hackery on the host)
• wrap the native lib for different js runtimes (web, node, deno, bun, etc)

[konsumer]

Maybe we should also rename it to whatever the transport is, like beatstep-rtmidi or beatstep-portmidi or whatever it ends up. I really can't decide on how I want to actually use this code (is it a demo or lib that I use in C? nim? js? what midi transport should it use? etc) so maybe for me, that is the first step :)

[konsumer]

Maybe if we made it generic enough it would be more useful for multiple transports/libs/targets, like you give it a "send sysex" function, and connect it's callback for async "received message" and then it would work easier with rtmidi, portmidi, teensy, and any other midi lib.

[doctea]

Ooh, looks cool! Does the update-flashing stuff work? Mine doesn't quite work, and that would be sick!

Oh, no, I haven't attempted anything nearly as advanced! I think like you I was just trying to make the BeatStep a little bit more useful, by breaking out some of those sysex commands into something more usable. Even if I could figure it out, I definitely wouldn't trust myself not to brick the BeatStep if I tried to upload new code to it!

My app started out originally as a Eurorack clock generator that would also clock USB-MIDI and MIDI-serial devices, but feature creep has taken over as I try to make the most of the cheap MIDI devices I have connected up.. some quite simple things like dividing the MIDI clock on a per-device basis, delaying MIDI clock for X notes of a bar, and advancing the BeatStep preset automatically is going a long way. Being able to save, reload and manipulate BeatStep presets would be even more useful. (Maybe even have a little editor that can program sequences, since doing it via the BeatStep knobs is a bit hit-and-miss).

I made a js library/cli at one point, that I am planning on sunsetting once I get this one packaged up in a native js module. It was the first ideas around this stuff, and has a cool alternative sequencer (you can see video here) which uses sysex to make the device behave in a more useful way (at least for me, for making beats.) I liked how it worked, but the native js stuff was really flakey for windows builds and the CLI tool it creates is way bigger than it needs to be (it embeds a whole node interpretor, which is just silly.) I think doing it directly in C++/rtmidi fixes a lot of these problems.

I think I actually came across your JS library when I was searching for BeatStep stuff originally! I hadn't seen your video though, or appreciated before everything that it does -- that's incredibly cool! So if I understand right then you're listening to messages from the BeatStep for the user input, and turning LEDs on and off to show the state?

I'm sequencing drums in my setup using a set of CV gate outs and an Akai APCMini as UI (top 4 rows as clock outputs, with divisions from every 16th note up to every 8 bars, and bottom 4 rows as a very simple 8-step (beat) sequencer where each step can be cycled through 4th-8th-16th notes for that step). As I'm using other ways of generating and mangling drum triggers in my rack, that's been plenty to keep me entertained, but what you've done there is very very cool and inspiring.

I feel like what you've got there could be abstracted to allow almost any MIDI controller with pads and LEDs to be used as an interface to your sequencer, not just the BeatStep, by adjusting what messages are sent and received? I actually had the basic USB+APCMini+sequencer sketch running on an Arduino Uno before I started running out of memory and moved to Teensy - so it makes me think that this could be the basis of a cheap and powerful sequencer that would breathe new life into a whole array of cheap controllers (the Akai "mini" series seem ripe for this and can be found cheap second hand)!

The sleep seems to help with a few things, and I was doing a more async pattern before in js, but it would sometimes miss some of the messages waiting for responses async, at least the way I was doing it, so sleep might be the best way

I wondered if that was the reason -- my code doesn't do any querying of the BeatStep at all so far, and I had a few problems with the Teensy USB library to even get the BeatStep to respond to sysex at all originally. I'll have to have a crack at it, and see if I have any trouble querying for BeatStep values.

Maybe some sort of async queue ("I'm waiting for these sysex answers, trigger the callbacks when they are found") would help, but I haven't figured it out.

That's the approach that came to mind for me immediately -- may not even need to bother remembering "this is what I've asked for" for what I need?

I am not as strong with C/C++ as other languages, but these ideas sound good.

I've been learning C++ objects and template features as I've been going with my project, so its been a bit of a learning curve! I suspect you're probably as proficient in your "weaker" languages as I am in my stronger ones (varying and rusty experience in C/C++/Java/PHP/Python!). I've not done any large C/C++ projects on any desktop platform yet though. C/C++ is definitely what I'm interested in here though, as I'm targeting embedded platforms and Arduino framework - for desktop I can see it makes sense to use something a bit more high level - almost like doing advanced MIDI scripting.

Since it goes so slow for me, you might want to fork it, but I'd also be totally into giving you commit access, if you want to just add stuff to it, and be a co-maintainer.

Well, let's see how we go :D. It might be that the sorts of things I want to do to make it work on Arduino/Teensy would needlessly overcomplicate the basic library - for example I would likely need to abstract out the preset-saving, as well as the underlying MIDI communication parts, to use libraries more suited to the microcontroller environment - which might be overkill for your use case..

If you want to work together on it as a C-lib, here is my next goals for this:

• Get firmware update working. I could not figure out how to capture firmware-update traffic, so never really worked out how to send .led files, but this would be useful for linux people, where there is no official support, and also I found mac M1 to be really flakey with official app

I find the official app flakey and confusing even on Windows, which I presume is their biggest target -- if there were a quick way to plug in the BeatStep, hit a button and backup all the presets then that would be much more useful to me! If it can do all that purely over sysex then I guess I could even do that through my usb_midi_clocker without the hassle of disconnecting and reconnecting to desktop (but backing up and managing preset banks directly from my usb_midi_clocker without my desktop being used at all would be even more ideal for my use case).

• Package it for better Cmake use. If people use it for C/C++, this would make their lives much easier
• Explore PortMidi. If it's better, maybe use that, instead.

I know very very little indeed about these things, I'm afraid!

• Async: I like this idea a lot, but never got get working

I'll report back if I have any luck doing async querying in my app or anything I find.

Longterm beatstep goals:

• Reverse-engineer the firmware, so we can send it custom firmware. I always wanted to get an alternative sequencer running on the actual device (instead of sysex hackery on the host)

This is very intriguing. You mentioned that you can't seem to sniff the firmware upload -- I wondered if the sysex just puts the device into 'firmware upload' mode, and then the actual binary is sent over the USB in some other way than as MIDI/sysex?

This also reminds me of the third-party sequencer firmware that someone developed for the Behringer BCR2000 - I've never read into that project, but perhaps that could be worth some investigation to see how that came about. There's also the USBMidiKlik firmware that started as third-party firmware for a MIDI thru device, maybe there's some clues to be had there. Just some thoughts for things to look into further when its not so late at night!

• wrap the native lib for different js runtimes (web, node, deno, bun, etc)

Very much outside of my area again I'm afraid ;D

This has been a very inspiring and thought-provoking conversation so far though, thanks! I'll continue watching your projects, and let you know how I get on with mine and if there are opportunities for some cross-pollination and collaboration. Also, I don't know if you had considered it before, but a little cheap Arduino-driven box that could turn various cheap MIDI controllers into a MIDI/Eurorack sequencer/clocker is sounding pretty useful to me right now and your work on your alternative BeatStep sequencer so far seems very close to this!

[doctea]

Maybe if we made it generic enough it would be more useful for multiple transports/libs/targets, like you give it a "send sysex" function, and connect it's callback for async "received message" and then it would work easier with rtmidi, portmidi, teensy, and any other midi lib.

Ah! Yes, that would be the sort of thing I'm thinking of that I could easily plug into my existing project. Parcelling things out, so there'd just be core helper functions/class for doing stuff on the device, agnostic of the underlying transports and file format..

[doctea]

Alright, I seem to be able to receive sysex asynchronously in my app, so that's a good start! The Teensy USB library just has you do "setHandleSysEx()" and pass in a function to receive the data, so its pretty straightforward from that point of view - I don't know how more desktop-oriented MIDI libraries handle this sort of thing, though..?

00:28:01.078 > Sending Sysex to beatstep [f0 00 20 6b 7f 42 01 00 50 04 f7 ]
00:28:01.079 > BeatStep replied with SYSEx: [f0 00 20 6b 7f 42 02 00 50 04 03 f7 ] complete

[konsumer]

Alright, I seem to be able to receive sysex asynchronously in my app, so that's a good start!

Nice!

The Teensy USB library just has you do "setHandleSysEx()" and pass in a function to receive the data, so its pretty straightforward from that point of view - I don't know how more desktop-oriented MIDI libraries handle this sort of thing, though..?

I could wrap it in portmidi/rtmidi, so it works similar.

[konsumer]

Oh, no, I haven't attempted anything nearly as advanced! I think like you I was just trying to make the BeatStep a little bit more useful, by breaking out some of those sysex commands into something more usable. Even if I could figure it out, I definitely wouldn't trust myself not to brick the BeatStep if I tried to upload new code to it!

For sure! I have the same worry. If I can work out a good way to spy on the MIDI port that shows the firmware-update process, I think I can do it, at least the actual update part (maybe not reversing the firmware code, though.) Maybe it does something weird like not send it as sysex (like sends plain serial) which would explain why it doesn't show in MIDI spy tools (it's not sysex.)

My app started out originally as a Eurorack clock generator that would also clock USB-MIDI and MIDI-serial devices, but feature creep has taken over as I try to make the most of the cheap MIDI devices I have connected up.. some quite simple things like dividing the MIDI clock on a per-device basis, delaying MIDI clock for X notes of a bar, and advancing the BeatStep preset automatically is going a long way. Being able to save, reload and manipulate BeatStep presets would be even more useful. (Maybe even have a little editor that can program sequences, since doing it via the BeatStep knobs is a bit hit-and-miss).

That sounds sick!

I think I actually came across your JS library when I was searching for BeatStep stuff originally! I hadn't seen your video though, or appreciated before everything that it does -- that's incredibly cool! So if I understand right then you're listening to messages from the BeatStep for the user input, and turning LEDs on and off to show the state?

Yep, exactly. I remap all the "special buttons" and trigger LEDs and stuff to make a new kind of interface than it normally has. The regular sequencer is a single track/pattern, and you have to dial in the note it sends with the control-knobs. Very clunky. I found the alternative sequencer I made was really good for making beats fast. Feels very "physical" (no screen.)

I'm sequencing drums in my setup using a set of CV gate outs and an Akai APCMini as UI (top 4 rows as clock outputs, with divisions from every 16th note up to every 8 bars, and bottom 4 rows as a very simple 8-step (beat) sequencer where each step can be cycled through 4th-8th-16th notes for that step). As I'm using other ways of generating and mangling drum triggers in my rack

That sounds cool!

but what you've done there is very very cool and inspiring.

Thanks! It could probly run on a teensy too, to be honest, would just need to work out how to host it for midi (since midi-in is USB only, as far as I understand). it seems like maybe you already figured that out?

I feel like what you've got there could be abstracted to allow almost any MIDI controller with pads and LEDs to be used as an interface to your sequencer, not just the BeatStep, by adjusting what messages are sent and received?

Yep, for sure, just have to work out the (sometimes secret) MIDI messages to turn lights on/off.

I actually had the basic USB+APCMini+sequencer sketch running on an Arduino Uno before I started running out of memory and moved to Teensy - so it makes me think that this could be the basis of a cheap and powerful sequencer that would breathe new life into a whole array of cheap controllers (the Akai "mini" series seem ripe for this and can be found cheap second hand)!

Hell yeah! Lemme know if you need help with some other device. I find this stuff interesting, in general.

I wondered if that was the reason -- my code doesn't do any querying of the BeatStep at all so far, and I had a few problems with the Teensy USB library to even get the BeatStep to respond to sysex at all originally. I'll have to have a crack at it, and see if I have any trouble querying for BeatStep values.

Yeh, it's a bit tricky. You can see it in my code here. You send it a message like this:

0xF0, 0x00, 0x20, 0x6B, 0x7F, 0x42, 0x01, 0x00, pp, cc, 0xF7

and it sends back (at some point):

0xF0, 0x00, 0x20, 0x6B, 0x7F, 0x42, 0x01, 0x00, pp, cc, VALUE, 0xF7

Maybe another approach would be to get all the values up-front and keep state for them, then when you set it or receive a message like that, update the internal state. It sort of seems like the official app does this, because it sends a lot of "hey what is this setting?" messages when you first plug it in. That still wouldn't really be async, for get (because you have to wait for it to give you the value) but maybe all the get could just be fast sync with no sleep (just return the current in-mem state.) I think that would really improve this lib for all usecases.

I've been learning C++ objects and template features as I've been going with my project, so its been a bit of a learning curve! I suspect you're probably as proficient in your "weaker" languages as I am in my stronger ones (varying and rusty experience in C/C++/Java/PHP/Python!). I've not done any large C/C++ projects on any desktop platform yet though. C/C++ is definitely what I'm interested in here though, as I'm targeting embedded platforms and Arduino framework - for desktop I can see it makes sense to use something a bit more high level - almost like doing advanced MIDI scripting.

Yeh, the low-level approach is kind of a drag to me, in general, unless I am using arduino/teensy/etc (where there are lots of easy-to-use libs to take away from the suck of it.) It takes so long to get basic stuff done, but also for hardware-interfacing, it's often the only way. C needs a really good universal package-manager sort of thing, like node, python, rust, etc have.

It might be that the sorts of things I want to do to make it work on Arduino/Teensy would needlessly overcomplicate the basic library

Yeh, I could see that, too. Maybe we could share just some parts, like the actual sysex parsing, but leave the other stuff slightly more platform-dependant (desktop vs teensy vs web, etc.)

I find the official app flakey and confusing even on Windows,

Yes! That was part of my motivation for messing with this stuff. I wanted a reliable way to mess with presets in the CLI, on linux/mac/windows, because it was so jenky.

directly from my usb_midi_clocker without my desktop being used at all would be even more ideal for my use case).

Yeh, that sounds seriously cool. Reminds me of FPGAs with a microcontroller piggyback, so they can be re-programmed in the field. Seems like a really cool way to go, and would enable you to use the device in lots of configurable ways on the fly, with no computer.

I know very very little indeed about these things, I'm afraid!

No prob! These are just my plans, which may has some overlap with yours, but totally cool, if not.

This is very intriguing. You mentioned that you can't seem to sniff the firmware upload -- I wondered if the sysex just puts the device into 'firmware upload' mode, and then the actual binary is sent over the USB in some other way than as MIDI/sysex?

Yep, exactly. I have working code to put it into "update mode" but after that, I can't seem to sniff it. I am starting to suspect "update mode" is some plain serial (not midi/sysex) thing, so it just sends the led file as if it was a microcontroller hooked up to midi serial. I will see if I can find a lower-level USB serial spy sort of program.

This also reminds me of the third-party sequencer firmware that someone developed for the Behringer BCR2000 - I've never read into that project, but perhaps that could be worth some investigation to see how that came about. There's also the USBMidiKlik firmware that started as third-party firmware for a MIDI thru device, maybe there's some clues to be had there.

Yes! this is exactly my thinking. The beatstep feels so nice to use, and has a great pricetag, but I really don't like the default firmware sequencer behavior. I don't do eurorack, just computers and I have a Polyend Tracker, so the regular beatstep (not pro) ends up feeling very limited to me, even though it's pretty amazing compared to like a classic 70s analog sequencer. I am using the Tracker more for this kind of stuff, but originally it seemed perfect for a track-based step-sequencer for drums. Might still be nice for that in my setup, as a dedicated "beat sequencer" device...

Very much outside of my area again I'm afraid ;D

No prob!

This has been a very inspiring and thought-provoking conversation so far though, thanks!

For me too!

I'll continue watching your projects, and let you know how I get on with mine and if there are opportunities for some cross-pollination and collaboration.

Sweet!

Also, I don't know if you had considered it before, but a little cheap Arduino-driven box that could turn various cheap MIDI controllers into a MIDI/Eurorack sequencer/clocker is sounding pretty useful to me right now and your work on your alternative BeatStep sequencer so far seems very close to this!

Yeh, I made a thing a while ago for using arduino to send sysex to Juno synths (a friend has one) so they didn't have to dive through menus, which I think is a similar idea. I think a major barrier on Beatstep is that it has no plain midi in, so you need a USB host to send it colors/etc. Other devices would probly be easier to work with, and there are microcontroller USB host things (although they are kinda expensive.) Personally, a pizero generally seems like a better option, since it already has USB, and only costs like $5, all ready-made, but it is a computer, if you are trying to avoid that. You can use them fairly low-level though, so they boot fast, and essentially work like you would use a micro, so I dunno if the distinction really matters. It's an extremely high-powered microcontroller, or low-powered computer.

[doctea]

Ah-ha, so, I see what you mean - if I send requests for two parameters without any delay in between, then I only get a response to the first request! I'm finding I need a minimum of a 2ms delay.

What a nuisance. I guess a queue system that ensures a response to the last request has been received before sending a new one is the best way to do this without impacting on the rest of my app... not ideal!

There certainly is room for a whole new firmware that addresses all these shortcomings!

[konsumer]

Yes, for sure!
I think we could do it like I am assuming the official software does it (based on sniffed messages.) Sometimes just describing the problem to someone else helps come up with a solution!

• Make a big local byte vector of all params (there aren't that many) and set a "got params" flag to false.
• send a bunch of "give me X param"
• make a callbalck for sysex that fills in the correct position in vector, and checks if all the messages have come through, and sets "got params" to true, if so
• any set also sets the local vector
• "get" becomes just "give me the vector value in this position" so it's sync, but no pause (just a memory lookup, so very fast)

[konsumer]

Ok, so I am starting to think about a rewrite. Maybe I will try to make it a more general "process sysex messages" and state-holder thing and wrap the actual midi stuff in the CLI part, so it will work better for your thing (but also maybe be easier to use in other things.)

[konsumer]

I rewrote it all to use async callback, and it seemed to work, but it still needs sleep, because if you bombard it with messages (like to get/set initial state with a bunch of gets or sets), the device just can't keep up. Since it still uses sleep and it eats a bunch of memory to hold the state (and complicates getting settings, and requires a longer delay on init) it seems like it's not a great way to do it. I'm not really sure I see how to change it to work better for both targets, so maybe it just makes more sense to fork it and make your own teensy version, and throttle the messages in whatever is the best way to do that on teensy. For regular computers, in a CLI program, I think sleep is probably the best way to do that.

I dunno, I think using async would help with other things, too, so I will keep playing around with it.

[doctea]

Wow, you work fast!

I implemented a little queue last night (the important bits here https://github.com/doctea/usb_midi_clocker/blob/main/include/queue.h + https://github.com/doctea/usb_midi_clocker/blob/main/include/behaviours/behaviour_beatstep.h#L126).

The queue waits for a response to the previous request, also allows to schedule the request with a delay. Also added a timeout so the queue won't get stuck if a response isn't received.

This has improved things. But didn't solve all the problems -- some requests are still missed/ignored. Also seeing sometimes that the BeatStep actually responds to the /previous/ request rather than the most recent!

I've noticed that making requests can be much more effective sometimes than others; for example when I am sending a pair of requests at the start of a bar, one of the requests gets ignored most of the time; but the requests that I send immediately after sending a MIDI Start message (and there seem to be a bunch of them as I presume I'm accidentally sending several instead of one) then they all get acknowledged perfectly.

I can see this being a big problem though if there isn't a way to confirm that the BeatStep has received a 'set' message. Have you seen if the official app do anything like requesting the value its just set, to make sure that it was received OK?

Example debug output - you can see here that sometimes it can process 3 requests within the same millisecond, other times it needs 10 milliseconds before it'll respond..:

21:38:48.625 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)
21:38:48.626 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:48.626 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)
21:38:48.636 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 06 10 f7 ] complete
21:38:48.637 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)
21:38:48.637 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:49.100 > testRequest!!
21:38:49.101 > queueing request for 50, 06
21:38:49.101 > queueing request for 50, 04
21:38:49.101 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)
21:38:49.125 > testRequest!!
21:38:49.126 > queueing request for 50, 06
21:38:49.126 > queueing request for 50, 04
21:38:49.154 >  queue timed out (77182492 vs 77182495)
21:38:49.218 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)
21:38:49.218 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:49.218 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)
21:38:49.218 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 06 10 f7 ] complete
21:38:49.218 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)
21:38:49.218 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:50.620 > testRequest!!
21:38:50.621 > queueing request for 50, 06
21:38:50.621 > queueing request for 50, 04
21:38:50.621 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)
21:38:50.670 >  queue timed out (77184012 vs 77184012)
21:38:50.671 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)
21:38:50.672 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:52.646 > testRequest!!
21:38:52.650 > queueing request for 50, 06
21:38:52.650 > queueing request for 50, 04
21:38:52.650 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)  
21:38:52.650 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 06 10 f7 ] complete
21:38:52.650 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)  
21:38:52.651 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:54.668 > testRequest!!
21:38:54.669 > queueing request for 50, 06
21:38:54.669 > queueing request for 50, 04
21:38:54.669 > testRequest!!
21:38:54.669 > queueing request for 50, 06
21:38:54.669 > queueing request for 50, 04
21:38:54.669 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)  
21:38:54.671 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 06 10 f7 ] complete
21:38:54.671 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)  
21:38:54.673 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:54.675 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 06 f7 ] (request)  
21:38:54.675 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 06 10 f7 ] complete
21:38:54.678 > Sending Sysex to BeatStep        [ f0 00 20 6b 7f 42 01 00 50 04 f7 ] (request)  
21:38:54.680 > BeatStep replied with Sysex:     [ f0 00 20 6b 7f 42 02 00 50 04 00 f7 ] complete
21:38:56.673 > testRequest!!
21:38:56.686 > queueing request for 50, 06

[konsumer]

I like the queue idea, but as you noted, different things takes a different wait, and sleeps themselves are going to be annoying for teensy. I think this might be a good angle, with a little bit of adjustment, but I'm not totally sure how you are using it.

On #1 I have a demo program that uses an async receive-queue to just print incoming bytes. It still has sleep to wait before sending more bytes (to not overload the device, and make sure it has time to answer) but you could do something with an async event loop on teensy to say "if this amount of time has passed, move on to next request" similar to async light-blink examples.

I think if you're going to do async, you really have to go full async (no response on get, just check if you got that "get response" later.)

I really think both issues are usecases that are out of scope for what I was originally trying to do, but I could probly help both of you use the parsing stuff I have figured out to make your things work.

BeatStep actually responds to the /previous/ request rather than the most recent!

yep, I have seen that too. I think it's "you're sending messages too fast" response, like it doesn't have time to catch up.

I can see this being a big problem though if there isn't a way to confirm that the BeatStep has received a 'set' message. Have you seen if the official app do anything like requesting the value its just set, to make sure that it was received OK?

This is what I mean by a big object in memory for all the settings. Here is an example:

• It uses json to save/load settings, so I make an empty json object when it starts up.
• When "response to get" comes in async callback, you can add it to the JSON object (since they include the setting they are responding to.)
• send a bunch of "gimme this setting" messages, each with a very polite (maybe 10 or 100 ms) pause in-between
• wait for all the answers you need for your preset file, then output that

SInce you know what responses you should get, you could also loop through the thing that holds the json data and make sure all the keys have been set, and if a certain amount of time elapsed, look for fields that need to be re-requested (to handle any messages that got lost.)