Best way to handle several hundred/thousand subscriptions

[darkwatchuk]

Hi,

Quick question. I need to make hundreds, maybe a couple of thousand subscriptions to nats core subjects (no jetstream). I can't use wildcards. Apparently NATS core is built to handle this which is great. But what's the best way to subscribe to all of these in the new C# client? The received messages (ideally in the same loop) would be of the same type.

Obviously I can run up hundreds/thousands of tasks to run subscribe loops and could process all messages directly or push them in to a queue, but that seems a bit heavy handed. Ideally I'd want one read loop and be able to add subscriptions on to that same read loop.

Is that possible? Or is there a better way? Wildcarding is NOT an option!

If not currently possible, would tweaking the code to allow passing a Channel<NatsMsg> into a NatsSub work?

Thanks in advance!

[mtmk]

I don't think there is at the moment. you can use SubscribeCore() and end up with lots of channels but i guess thats still the same problem.

just a few thoughts/suggestions: you might be able to create a sub class of NatsSubBase and put all the subscriptions into one custom NatsSub class. also you might have to alter or create your own subscriptionmanager possibly improving the ISubscriptionManager interface. the main thing though at the protocol level you essentially need to send SUB x.y.z messages and keep track of their subscription ids (sid) so you can match the subject when it arrives on the wire and unsubscribe when done and that's the job of subscription manager.

interesting one. i hope this helps a bit.

[darkwatchuk]

Thanks @mtmk . Yes, have been digging around a bit on this. Not quite sure where best to squeeze this in without duplication lots of code. Will have a bit more of a play with it today as important that I get a solution for this for work.

[darkwatchuk]

Have added PR#819 to make NatsMsg.Build public. This allows for create of new subscriptions outside of the library. Everything else is available that I need.