-
Notifications
You must be signed in to change notification settings - Fork 890
Design Principles
To achieve low-latency with minimal variance, while achieving high throughput, it is important to define a set of design principles that guide the development. When a trade off is required then the design principles help guide the decision in choosing between competing alternatives.
Many design principles come to bear on any implementation. Not all design principles require trade-offs but many do. The following set of design principles are key to the design of Aeron and the likely to drive most trade-offs. When two or more design principles conflict with each other then the one with the highest rank wins.
- Garbage free in steady state running
- Apply Smart Batching in the message path
- Wait-free algorithms in the message path
- Non-blocking IO in the message path
- No exceptional cases in the message path
- Apply the Single Writer Principle
- Prefer unshared state
- Strict Separation of Concerns
- Avoid unnecessary data copies
When talking about the "message path" we are referring to the path taken through Aeron by data messages sent from a publisher to a subscriber, and not any internal messages Aeron may use for it operational purposes.
Garbage collection pauses are one of the biggest causes of latency in a virtual machine. The best cases for these pauses are just under a millisecond which is a significantly longer duration than the round trip time being two computers over a local area network. Aeron should not contribute to GC pauses and thus makes it a candidate to be included in an application that carefully manages its own allocation.
Communications traffic tends to be very bursty in nature. To mitigate the queuing latency introduced by burst traffic then Smart Batching should be employed to amortise the expensive costs within an algorithm.
A thread can be subject to an interrupt at any stage in its execution. If an algorithm is not wait-free then other threads can be blocked until the interrupted can resume. To avoid this blocking, all concurrent algorithms must be able to complete in a finite number of steps without blocking other threads.
The wakeup cost by the scheduler for a thread blocked on IO can be greater than the RTT on a fast local network. Also a blocked thread cannot do other work while blocked. To avoid this cost and restriction, non-blocking IO should be employed.
To keep code paths simple and predictable to the processor, the main paths of execution in the program should not be burdened by the exceptional cases. For example the code to fragment and reassemble larger messages should not burden the path of the common small messages. Exceptional cases should take the exceptional cost.
Contended access to mutable state requires mutual exclusion or conditional update protection. Either of these protection mechanisms cause queues to form as contended updates are applied. To avoid this contention and associated queueing effects all state should be owned by a single writer for mutation purposes.
Stuff
Stuff
Stuff