You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Bus is the core of how onetun passes events around internally to different components, such as when data is transmitted, or a connection is closed.
There are some advantages to this design, mainly code readability and extendability. However, I realize that the implementation has important performance drawbacks.
The Bus is implemented using a tokio::sync::broadcast queue, which allows for multiple consumers and producers. However, some of the critical onetun events only need to reach 1-2 listeners, yet each Event is cloned for all receivers (even if dropped/ignored).
Here are the list of Bus endpoints in onetun:
tcp_tunnel: each local connection has an endpoint
udp_tunnel: each UDP port-forward server has an endpoint
tcp_virtual: there is only 0-1 TCP virtual interface with 1 endpoint
udp_virtual: there is only 0-1 UDP virtual interface with 1 endpoint
device: the virtual layer4 smoltcpDevice implementation bridges data between tcp/udp_virtual and wg, it has 1 endpoint. There is 0-1 TCP device and 0-1 UDP device
wg: has 1 endpoint for the consumer (decapsulation) and 1 for the producer (encapsulation)
pcap: (optional) has 1 endpoint for recording inbound and outbound data
Here is the list of event channels in onetun (producer -> Event -> consumer):
tcp_tunnel -> ClientConnectionInitiated(PortForwardConfig, VirtualPort) -> tcp_virtual: When a client starts a TCP connection to the local port, this event creates a new client socket in the virtual interface (binding with the peer IP and the virtual port, i.e. this is the advertised port to remote peers). This a critical but quiet channel (only when new connections are created).
tcp_tunnel -> ClientConnectionDropped(VirtualPort) -> tcp_virtual: This event is dispatched when the tcp_tunnel has closed the connection with the local client (can be caused by an error, or if the local client terminated the connection, or if the remote peer has terminated the connection)
tcp_virtual -> ClientConnectionDropped(VirtualPort) -> tcp_tunnel: Similar to the one before, this is dispatched when communication with the remote peer has concluded to notify the tcp_tunnel to forcefully close the connection with the local client. Note: the tcp_tunnel will "reply" with the same event as the connection with the local client is closed (see previous).
tcp_tunnel -> LocalData(PortForwardConfig, VirtualPort, Bytes) -> tcp_virtual: Whenever data is polled from the local client, it is sent to the virtual interface. The virtual interface will enqueue this data and immediately send it off on the loop (poll delay is set to 0). This is a performance-critical channel and a major problem, since each packet is cloned for each endpoint.
udp_tunnel -> LocalData(PortForwardConfig, VirtualPort, Bytes) -> udp_virtual: Whenever data is polled from the local UDP socket, it is sent to the virtual interface. This is also a performance-critical channel.
tcp_virtual -> RemoteData(VirtualPort, Bytes) -> tcp_tunnel: Forwards data from the remote to the local client. This is also a performance-critical channel
udp_virtual -> RemoteData(VirtualPort, Bytes) -> udp_tunnel: See above. Also performance-critical
wg -> InboundInternetPacket(PortProtocol, Bytes) -> device: Forwards decrypted IP packets to the device, which is then propagated to the virtual interface. This is also a performance-critical channel
wg -> InboundInternetPacket(PortProtocol, Bytes) -> pcap: Used to record pcap files. This is a heavy and chatty channel but low-priority; the fact that each packet must wait for the pcap file to flush is bad though.
device -> OutboundInternetPacket(Bytes) -> wg: Forwards IP packets to boringtun (which encrypts and sends to wg). This is also a performance-critical channel
device -> OutboundInternetPacket(Bytes) -> pcap: Used to record pcap files. This is a heavy and chatty channel but low-priority; the fact that each packet must wait for the pcap file to flush is bad though.
device -> VirtualDeviceFed(PortProtocol) -> tcp_virtual: This is how the device informs the tcp_virtual interface that new data has arrived and is ready to be processed immediately (triggers a poll). This is performance-critical
device -> VirtualDeviceFed(PortProtocol) -> udp_virtual: This is how the device informs the tcp_virtual interface that new data has arrived and is ready to be processed immediately (triggers a poll). This is performance-critical
The text was updated successfully, but these errors were encountered:
The
Bus
is the core of howonetun
passes events around internally to different components, such as when data is transmitted, or a connection is closed.There are some advantages to this design, mainly code readability and extendability. However, I realize that the implementation has important performance drawbacks.
The
Bus
is implemented using atokio::sync::broadcast
queue, which allows for multiple consumers and producers. However, some of the criticalonetun
events only need to reach 1-2 listeners, yet eachEvent
is cloned for all receivers (even if dropped/ignored).Here are the list of Bus endpoints in
onetun
:tcp_tunnel
: each local connection has an endpointudp_tunnel
: each UDP port-forward server has an endpointtcp_virtual
: there is only 0-1 TCP virtual interface with 1 endpointudp_virtual
: there is only 0-1 UDP virtual interface with 1 endpointdevice
: the virtual layer4smoltcp
Device
implementation bridges data betweentcp/udp_virtual
andwg
, it has 1 endpoint. There is 0-1 TCP device and 0-1 UDP devicewg
: has 1 endpoint for the consumer (decapsulation) and 1 for the producer (encapsulation)pcap
: (optional) has 1 endpoint for recording inbound and outbound dataHere is the list of event channels in
onetun
(producer -> Event -> consumer
):tcp_tunnel -> ClientConnectionInitiated(PortForwardConfig, VirtualPort) -> tcp_virtual
: When a client starts a TCP connection to the local port, this event creates a new client socket in the virtual interface (binding with the peer IP and the virtual port, i.e. this is the advertised port to remote peers). This a critical but quiet channel (only when new connections are created).tcp_tunnel -> ClientConnectionDropped(VirtualPort) -> tcp_virtual
: This event is dispatched when thetcp_tunnel
has closed the connection with the local client (can be caused by an error, or if the local client terminated the connection, or if the remote peer has terminated the connection)tcp_virtual -> ClientConnectionDropped(VirtualPort) -> tcp_tunnel
: Similar to the one before, this is dispatched when communication with the remote peer has concluded to notify thetcp_tunnel
to forcefully close the connection with the local client. Note: thetcp_tunnel
will "reply" with the same event as the connection with the local client is closed (see previous).tcp_tunnel -> LocalData(PortForwardConfig, VirtualPort, Bytes) -> tcp_virtual
: Whenever data is polled from the local client, it is sent to the virtual interface. The virtual interface will enqueue this data and immediately send it off on the loop (poll delay is set to 0). This is a performance-critical channel and a major problem, since each packet is cloned for each endpoint.udp_tunnel -> LocalData(PortForwardConfig, VirtualPort, Bytes) -> udp_virtual
: Whenever data is polled from the local UDP socket, it is sent to the virtual interface. This is also a performance-critical channel.tcp_virtual -> RemoteData(VirtualPort, Bytes) -> tcp_tunnel
: Forwards data from the remote to the local client. This is also a performance-critical channeludp_virtual -> RemoteData(VirtualPort, Bytes) -> udp_tunnel
: See above. Also performance-criticalwg -> InboundInternetPacket(PortProtocol, Bytes) -> device
: Forwards decrypted IP packets to the device, which is then propagated to the virtual interface. This is also a performance-critical channelwg -> InboundInternetPacket(PortProtocol, Bytes) -> pcap
: Used to record pcap files. This is a heavy and chatty channel but low-priority; the fact that each packet must wait for the pcap file to flush is bad though.device -> OutboundInternetPacket(Bytes) -> wg
: Forwards IP packets to boringtun (which encrypts and sends to wg). This is also a performance-critical channeldevice -> OutboundInternetPacket(Bytes) -> pcap
: Used to record pcap files. This is a heavy and chatty channel but low-priority; the fact that each packet must wait for the pcap file to flush is bad though.device -> VirtualDeviceFed(PortProtocol) -> tcp_virtual
: This is how the device informs thetcp_virtual
interface that new data has arrived and is ready to be processed immediately (triggers a poll). This is performance-criticaldevice -> VirtualDeviceFed(PortProtocol) -> udp_virtual
: This is how the device informs thetcp_virtual
interface that new data has arrived and is ready to be processed immediately (triggers a poll). This is performance-criticalThe text was updated successfully, but these errors were encountered: