You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/install-upgrade/build-wiring.md
+161-2Lines changed: 161 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -83,8 +83,29 @@ This section is meant to help the reader understand how to assemble the primitiv
83
83
84
84
### VPC
85
85
86
-
A VPC allows for isolation at layer 3. This is the main building block for users when creating their architecture. Hosts inside of a VPC belong to the same broadcast domain and can communicate with each other, if desired a single VPC can be configured with multiple broadcast domains. The hosts inside of a VPC will likely need to connect to other VPCs or the outside world. To communicate between two VPC a *peering* will need to be created. A VPC can be a logical separation of workloads. By separating these workloads additional controls are available. The logical separation doesn't have to be the traditional database, web, and compute layers it could be development teams who need isolation, it could be tenants inside of an office building, or any separation that allows for better control of the network. Once your VPCs are decided, the rest of the fabric will come together. With the VPCs decided traffic can be prioritized, security can be put into place, and the wiring can begin. The fabric allows for the VPC to span more than a than one switch, which provides great flexibility, for instance workload mobility.
87
-
86
+
A VPC allows for isolation at layer 3. This is the main building block for users when creating their architecture. Hosts inside of a VPC belong to the same broadcast domain and can communicate with each other, if desired a single VPC can be configured with multiple broadcast domains. The hosts inside of a VPC will likely need to connect to other VPCs or the outside world. To communicate between two VPC a *peering* will need to be created. A VPC can be a logical separation of workloads. By separating these workloads additional controls are available. The logical separation doesn't have to be the traditional database, web, and compute layers it could be development teams who need isolation, it could be tenants inside of an office building, or any separation that allows for better control of the network. Once your VPCs are decided, the rest of the fabric will come together. With the VPCs decided traffic can be prioritized, security can be put into place, and the wiring can begin. The fabric allows for the VPC to span more than one switch, which provides great flexibility.
87
+
88
+
```mermaid
89
+
graph TD
90
+
L1([Leaf 1])
91
+
L2([Leaf 2])
92
+
S1["Server 1
93
+
10.7.71.1"]
94
+
S2["Server 2
95
+
172.16.2.31"]
96
+
S3["Server 3
97
+
192.168.18.85"]
98
+
99
+
L1 <--> S1
100
+
L1 <--> S2
101
+
L2 <--> S3
102
+
103
+
subgraph VPC 1
104
+
S1
105
+
S2
106
+
S3
107
+
end
108
+
```
88
109
### Connection
89
110
90
111
A connection represents the physical wires in your data center. They connect switches to other switches or switches to servers.
@@ -98,6 +119,55 @@ A server connection is a connection used to connect servers to the fabric. The f
98
119
-*MCLAG* - Two cables going to two different switches, also called dual homing. The switches will need a fabric link between them.
99
120
-*ESLAG* - Two to four cables going to different switches, also called multi-homing. If four links are used there will need to be four switches connected to a single server with four NIC ports.
100
121
122
+
```mermaid
123
+
graph TD
124
+
S1([Spine 1])
125
+
S2([Spine 2])
126
+
L1([Leaf 1])
127
+
L2([Leaf 2])
128
+
L3([Leaf 3])
129
+
L4([Leaf 4])
130
+
L5([Leaf 5])
131
+
L6([Leaf 6])
132
+
L7([Leaf 7])
133
+
134
+
TS1[Server1]
135
+
TS2[Server2]
136
+
TS3[Server3]
137
+
TS4[Server4]
138
+
139
+
S1 & S2 ---- L1 & L2 & L3 & L4 & L5 & L6 & L7
140
+
L1 <-- Bundled --> TS1
141
+
L1 <-- Bundled --> TS1
142
+
L1 <-- Unbundled --> TS2
143
+
L2 <-- MCLAG --> TS3
144
+
L3 <-- MCLAG --> TS3
145
+
L4 <-- ESLAG --> TS4
146
+
L5 <-- ESLAG --> TS4
147
+
L6 <-- ESLAG --> TS4
148
+
L7 <-- ESLAG --> TS4
149
+
150
+
subgraph VPC 1
151
+
TS1
152
+
TS2
153
+
TS3
154
+
TS4
155
+
end
156
+
157
+
subgraph MCLAG
158
+
L2
159
+
L3
160
+
end
161
+
162
+
subgraph ESLAG
163
+
L3
164
+
L4
165
+
L5
166
+
L6
167
+
L7
168
+
end
169
+
170
+
```
101
171
#### Fabric Connections
102
172
103
173
Fabric connections serve as connections between switches, they form the fabric of the network.
@@ -107,14 +177,103 @@ Fabric connections serve as connections between switches, they form the fabric o
107
177
108
178
VPCs need VPC Peerings to talk to each other. VPC Peerings come in two varieties: local and remote.
109
179
180
+
```mermaid
181
+
graph TD
182
+
S1([Spine 1])
183
+
S2([Spine 2])
184
+
L1([Leaf 1])
185
+
L2([Leaf 2])
186
+
TS1[Server1]
187
+
TS2[Server2]
188
+
TS3[Server3]
189
+
TS4[Server4]
190
+
191
+
S1 & S2 <--> L1 & L2
192
+
L1 <--> TS1 & TS2
193
+
L2 <--> TS3 & TS4
194
+
195
+
196
+
subgraph VPC 1
197
+
TS1
198
+
TS2
199
+
end
200
+
201
+
subgraph VPC 2
202
+
TS3
203
+
TS4
204
+
end
205
+
```
206
+
110
207
#### Local VPC Peering
111
208
112
209
When there is no dedicated border/peering switch available in the fabric we can use local VPC peering. This kind of peering tries sends traffic between the two VPC's on the switch where either of the VPC's has workloads attached. Due to limitation in the Sonic network operating system this kind of peering bandwidth is limited to the number of VPC loopbacks you have selected while initializing the fabric. Traffic between the VPCs will use the loopback interface, the bandwidth of this connection will be equal to the bandwidth of port used in the loopback.
113
210
211
+
```mermaid
212
+
graph TD
213
+
214
+
L1([Leaf 1])
215
+
S1[Server1]
216
+
S2[Server2]
217
+
S3[Server3]
218
+
S4[Server4]
219
+
220
+
L1 <-.2,loopback.-> L1;
221
+
L1 <-.3.-> S1;
222
+
L1 <--> S2 & S4;
223
+
L1 <-.1.-> S3;
224
+
225
+
subgraph VPC 1
226
+
S1
227
+
S2
228
+
end
229
+
230
+
subgraph VPC 2
231
+
S3
232
+
S4
233
+
end
234
+
```
235
+
The dotted line in the diagram shows the traffic flow for local peering. The traffic originates in VPC 2, travels to the switch, travels out the first loopback port, into the second loopback port, and finally out the port destined for VPC 1.
236
+
237
+
114
238
#### Remote VPC Peering
115
239
116
240
Remote Peering is used when you need a high bandwidth connection between the VPCs, you will dedicate a switch to the peering traffic. This is either done on the border leaf or on a switch where either of the VPC's are not present. This kind of peering allows peer traffic between different VPC's at line rate and is only limited by fabric bandwidth. Remote peering introduces a few additional hops in the traffic and may cause a small increase in latency.
117
241
242
+
```mermaid
243
+
graph TD
244
+
S1([Spine 1])
245
+
S2([Spine 2])
246
+
L1([Leaf 1])
247
+
L2([Leaf 2])
248
+
L3([Leaf 3])
249
+
TS1[Server1]
250
+
TS2[Server2]
251
+
TS3[Server3]
252
+
TS4[Server4]
253
+
254
+
S1 <-.5.-> L1;
255
+
S1 <-.2.-> L2;
256
+
S1 <-.3,4.-> L3;
257
+
S2 <--> L1;
258
+
S2 <--> L2;
259
+
S2 <--> L3;
260
+
L1 <-.6.-> TS1;
261
+
L1 <--> TS2;
262
+
L2 <--> TS3;
263
+
L2 <-.1.-> TS4;
264
+
265
+
266
+
subgraph VPC 1
267
+
TS1
268
+
TS2
269
+
end
270
+
271
+
subgraph VPC 2
272
+
TS3
273
+
TS4
274
+
end
275
+
```
276
+
The dotted line in the diagram shows the traffic flow for remote peering. The traffic could take a different path because of ECMP. It is important to note that Leaf 3 cannot have any servers from VPC 1 or VPC 2 on it, but it can have a different VPC attached to it.
0 commit comments