Benchmarking memory usage

[sondrelg]

Hi!

I'm interested in testing socketioxide and socketio for a project, and just wanted to get an initial impression of what overhead the protocol comes with. I tried benchmarking the implementation against a pure websocket implementation in this repo.

The socketioxide implementation seems to require an order of magnitude more memory than the websocket counter-part. This surprised me a bit, and I'm wondering if maybe I've made a mistake somewhere. I understand that there is more setup involved in establishing a websocket connection, but once established shouldn't the memory footprint be similar?

Would love feedback if you have any!

[Totodore]

Hi!
Thanks a lot for benchmarking this, indeed there is clearly an issue which was introduced in 0.16 with #427. I will propose a fix ASAP.

[Totodore]

If you want, it would also greatly help to compare against official socket.io server and also run benchmarks for socketioxide with heaptrack or valgrind massif to have a precise track of which allocations is causing issues.

[Totodore]

Corresponding issue #448

[sondrelg]

Looks like you've found the root cause @Totodore 👏 Will you change the buffer size as a fix?

[Totodore]

Hi @sondrelg, I invite you to check again, it should be much better now. There is still a significant gap however that I will defend here:
Yes socket.io has a significant overhead compared to websocket. As it supports 2 multiplexing layers and compatibily to legacy systems http polling most notably, as well as horizontal scaling and some other cool features. It is made to be "user-friendly" and really easy to use. At the cost of this overhead.

If you have a really precise idea of what you want to build and that it is something that must supports 50K sockets one a single machine with less than a Gig of memory then, this may be not the project you are looking for. Otherwise I think that socketioxide is great for building adding realtime application and focusing on business logic and eventually horizontally scale if it is needed.

Thanks you for you benchmark though, Thanks to thisn I realised there was clearly some memory issues introduced in the v0.16 version and even before!

[sondrelg]

Thanks for the reply @Totodore! 🙇 We're fine to horizontally scale to more pods, and 1Gi per 50k connections is definitely not a hard requirement!

Just out of curiosity: Is the remaining difference due to the protocol alone? My impression was that if websockets are an option, socketio will eventually (with a few more steps than normal) upgrade the connection to a websocket connection. Once we're at that stage, couldn't the overhead theoretically be similar? Perhaps there's more state kept track of?

[Totodore]

This is mostly a question of tracking state and building more features:

• There is an mpsc channel that serves as an abstraction between the transport (ws/polling) to seemlessly change between the too. Therefore this requires spawning two tasks per sockets to pipe data in the two directions.
• Custom parsers to support variety of features (acknowledments, binary data embedded in your structs, namespaces). For more performance/memory efficiency I invite you to use the msgpack parser rather than the default one.