Skip to content

Commit df0d667

Browse files
matzfmatzf
authored
doc: control service manual (#4361)
Add a conceptual description of the control plane and a reference manual for the control service. The control plane description is mostly based on the IETF SCION control plane specification draft (https://www.ietf.org/archive/id/draft-dekater-scion-controlplane-00html). The reference manual for the control service describes the many command line flags, environment variables, configuration files and APIs. This documentation highlights a variety of usability shortcomings, redundancies and inconsistencies that should be addressed eventually.
1 parent 2ce7932 commit df0d667

File tree

15 files changed

+1323
-70
lines changed

15 files changed

+1323
-70
lines changed

‎doc/beacon-metadata.rst

Lines changed: 3 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -294,64 +294,10 @@ are available:
294294
| Notes | The notes for each AS on the path |
295295
+-------------------+---------------------------------------------------------------------------------+
296296

297-
Configuration File Format
298-
=========================
299-
300-
The control service obtains the information for the ``StaticInfoExtension``
301-
for the PCBs it sends out from a JSON configuration file, ``staticInfoConfig.json``.
302-
303-
There is one top-level entry for each type of metadata. All entries are optional.
304-
305-
- ``Latency`` is a map where the key is Interface ID ``i`` and the values are:
306-
307-
+-------------+-------------------------------------------+-------------------------------------------------+
308-
| Name | Type | Description |
309-
+=============+===========================================+=================================================+
310-
| ``Inter`` | Duration | Latency from interface ``i`` to remote AS |
311-
+-------------+-------------------------------------------+-------------------------------------------------+
312-
| ``Intra`` | Map: Interface ID ``j`` : Duration | Latency from interface ``i`` to interface ``j`` |
313-
+-------------+-------------------------------------------+-------------------------------------------------+
314-
315-
- ``Bandwidth`` is a map where the key is Interface ID ``i`` and the values are:
316-
317-
+-------------+-------------------------------------------+-----------------------------------------------------------------+
318-
| Name | Type | Description |
319-
+=============+===========================================+=================================================================+
320-
| ``Inter`` | Integer | Bandwidth in Kbit/s between interface ``i`` and the remote AS |
321-
+-------------+-------------------------------------------+-----------------------------------------------------------------+
322-
| ``Intra`` | Map: Interface ID ``j`` : Integer | Bandwidth in Kbit/s between interface ``i`` and interface ``j`` |
323-
+-------------+-------------------------------------------+-----------------------------------------------------------------+
324-
325-
- ``Geo`` is a map where the key is Interface ID ``i`` and the values are:
326-
327-
+-----------------+-----------------+-----------------------------------------------+
328-
| Name | Type | Description |
329-
+=================+=================+===============================================+
330-
| ``Latitude`` | Decimal value | Longitude GPS coordinates of interface ``i`` |
331-
+-----------------+-----------------+-----------------------------------------------+
332-
| ``Longitude`` | Decimal value | Latitude GPS coordinate of interface ``i`` |
333-
+-----------------+-----------------+-----------------------------------------------+
334-
| ``Address`` | String | Address of interface ``i`` |
335-
+-----------------+-----------------+-----------------------------------------------+
336-
337-
- ``LinkType`` is a map where the key is Interface ID ``i`` and the value is one of
338-
339-
- ``"direct"``
340-
- ``"multihop"``
341-
- ``"opennet"``
342-
343-
- ``Hops`` is a map where the key is Interface ID ``i`` and the values are:
344-
345-
+-------------+------------------------------------+----------------------------------------------------------------------+
346-
| Name | Type | Description |
347-
+=============+====================================+======================================================================+
348-
| ``Intra`` | Map: Interface ID ``j`` : Integer | Number of internal hops between interface ``i`` and interface ``j`` |
349-
+-------------+------------------------------------+----------------------------------------------------------------------+
350-
351-
- ``Note`` is a string.
297+
.. _path-metadata-example-conf:
352298

353299
Example Configuration
354-
---------------------
300+
=====================
355301

356302
Let us look at an AS with three interfaces with IDs 1, 2, 3 and 5 which
357303
looks like the diagram below. The values attached to the connections
@@ -360,7 +306,7 @@ represent the latency in milliseconds between interfaces.
360306
.. figure:: fig/beacon_metadata/example_config_metrics.png
361307
:width: 50%
362308

363-
The configuration file for this AS could then look like this:
309+
The :ref:`staticInfoConfig.json <control-conf-path-metadata>` configuration file for this AS could then look like this:
364310

365311
.. code:: JSON
366312

‎doc/conf.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
'recommonmark',
2222
'sphinx_rtd_theme',
2323
'sphinx.ext.extlinks',
24+
'sphinxcontrib.openapi',
2425
]
2526

2627
# Add any paths that contain templates here, relative to this directory.

‎doc/control-plane.rst

Lines changed: 291 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,299 @@ Control Plane
1010
hidden-paths
1111
beacon-metadata
1212

13+
14+
Introduction
15+
============
16+
1317
The SCION control plane is responsible for discovering path segments and making them available to
1418
endpoints. This includes path-segment exploration (also called "beaconing"), registration, lookup,
1519
and finally the combination of path-segments to end-to-end paths.
1620

17-
.. admonition:: TODO
21+
.. Note: content based on (extracts from) IETF draft draft-dekater-scion-controlplane-00.
22+
23+
The **control service** is responsible for the path exploration and registration processes in the
24+
control plane.
25+
It is the main control-plane infrastructure component within each SCION :term:`AS`.
26+
The control service of an AS has the following tasks:
27+
28+
- Generating, receiving, and propagating :term:`Path Construction Beacons (PCBs) <PCB>`.
29+
Periodically, the control service of a core AS generates a set of PCBs, which are forwarded to the
30+
child ASes or neighboring core ASes.
31+
In the latter case, the PCBs are sent over policy-compliant paths to discover multiple paths
32+
between any pair of core ASes.
33+
- Selecting and registering the set of path segments via which the AS wants to be reached.
34+
- Managing certificates and keys to secure inter-AS communication.
35+
Each PCB contains signatures of all on-path ASes.
36+
Every time the control service of an AS receives a PCB, it validates the PCB's authenticity.
37+
When the control service lacks an intermediate certificate, it can query the control service of
38+
the neighboring AS that sent the PCB.
39+
40+
Path Segments
41+
-------------
42+
43+
As described previously, the main goal of SCION's control plane is to create and manage path
44+
segments, which can then be combined into forwarding paths to transmit packets in the data plane.
45+
SCION distinguishes the following types of path segments:
46+
47+
- A path segment from a non-core AS to a core AS is an *up-segment*.
48+
- A path segment from a core AS to a non-core AS is a *down-segment*.
49+
- A path segment between core ASes is a *core-segment*.
50+
51+
So each path segment either ends at a core AS, or starts at a core AS, or both.
52+
53+
.. note::
54+
55+
There are no SCION path segments that start and end at a non-core AS. However, when combining
56+
path segments into an end-to-end SCION path, shortcuts and peering-links can be used.
57+
58+
All path segments are reversible: A core-segment can be used bidirectionally, and an up-segment can
59+
be converted into a down-segment, or vice versa, depending on the direction of the end-to-end path.
60+
This means that all path segments can be used to send data traffic in both directions.
61+
62+
.. _control-plane-beaconing:
63+
64+
Path Exploration (Beaconing)
65+
============================
66+
67+
**Path exploration** is the process where an AS discovers paths to other ASes. In SCION, this
68+
process is referred to as *beaconing*.
69+
70+
In SCION, the *control service* of each AS is responsible for the beaconing process.
71+
The control service generates, receives, and propagates *path-segment construction beacons (PCBs)*
72+
on a regular basis, to iteratively construct path segments.
73+
PCBs contain topology and authentication information, and can also include additional metadata that
74+
helps with path management and selection.
75+
The beaconing process itself is divided into routing processes on two levels, where *inter-ISD* or
76+
core beaconing is based on the (selective) sending of PCBs without a defined direction, and
77+
*intra-ISD* beaconing on top-to-bottom propagation.
78+
This division of routing levels is a key architectural decision of SCION and important for achieving
79+
a better scalability.
80+
81+
- *Inter-ISD or core beaconing* is the process of constructing path segments between core ASes in
82+
the same or in different ISDs. During core beaconing, the control service of a core AS either
83+
initiates PCBs or propagates PCBs received from neighboring core ASes to other neighboring core
84+
ASes. Core beaconing is periodic; PCBs are sent over policy-compliant paths to discover multiple
85+
paths between any pair of core ASes.
86+
- *Intra-ISD beaconing* creates path segments from core ASes to non-core ASes. For this, the control
87+
service of a core AS creates PCBs and sends them to the non-core child ASes (typically customer
88+
ASes). The control service of a non-core child AS receives these PCBs and forwards them to its
89+
child ASes, and so on. This procedure continues until the PCB reaches an AS without any customer
90+
(leaf AS). As a result, all ASes within an ISD receive path segments to reach the core ASes of
91+
their ISD.
92+
93+
On its way, a PCB accumulates cryptographically protected path- and forwarding information per
94+
traversed AS. At every AS, metadata as well as information about the AS's ingress and egress
95+
interfaces are added to the PCB.
96+
97+
Origination of PCBs
98+
-------------------
99+
100+
Every core AS originates PCBs at regular intervals, and sends these to all egress interfaces to
101+
connected neighbor ASes.
102+
An originated PCB sent to a neighboring core ASes initiates an inter-ISD beacon, ultimately
103+
resulting in a core-segment.
104+
An originated PCB sent to a child AS initiates the intra-ISD beacon creating an up/down segment.
105+
106+
Propagation of PCBs
107+
-------------------
108+
109+
PCBs are propgated at regular intervals at each AS.
110+
When PCBs are received, they are not propagated immediately, but put into temporary storage
111+
until the next propagation event.
112+
The selection and propagation of PCBs differs between the inter-ISD and intra-ISD beacon schemes.
113+
114+
Core ASes implement the inter-ISD / core beaconing scheme.
115+
For every interface connecting to a neighboring core AS:
116+
117+
1. Select the best :math:`N` PCBs for each origin core AS.
118+
This can take into account both the available PCBs as well as local policies and information
119+
about the link to the neighbor.
120+
2. Extend the selected PCBs by adding an *AS entry*
121+
3. Send the extended PCBs over the interface
122+
123+
Non-core ASes implement the intra-ISD / non-core beaconing scheme.
124+
For every interface connecting to a child AS:
125+
126+
1. Select the best :math:`N` PCBs.
127+
This can take into account both the available PCBs as well as local policies and information
128+
about the link to the child AS.
129+
2. Extend the selected PCBs by adding an *AS entry*
130+
3. Send the extended PCBs over the interface
131+
132+
AS Entries
133+
----------
134+
135+
Every AS adds a signed *AS entry* to the PCBs it originates, propagates or :ref:`registers <control-plane-registration>`.
136+
137+
This AS entry includes the relevant network topology information for this AS-hop
138+
defined by the ingress and egress :term:`interface IDs <Interface ID>` of the beacon.
139+
The so-called *hop field* includes a MAC that authorizes the use of this hop in the path
140+
segment defined by the PCB, until it expires.
141+
See the description of the :ref:`SCION Path <path-type-scion>` in the data plane section for more
142+
details on the hop field format and the MAC chaining mechanism.
143+
144+
Additionally, an AS entry can contain :doc:`metadata <beacon-metadata>` such as the link MTU,
145+
geographic locations of the AS routers, latencies, etc.
146+
147+
For illustration, the following code blocks show the definition of the protobuf message definitions
148+
for the AS entry "body" and the contained hop field information.
149+
This is just a small excerpt of the relevant definitions.
150+
See the `SCION Control Plane IETF draft (section "Components of a PCB") <https://www.ietf.org/archive/id/draft-dekater-scion-controlplane-00.html#name-components-of-a-pcb-in-mess>`_
151+
for a more complete discussion of the message formats and signature inputs,
152+
or :file-ref:`proto/control_plane/v1/seg.proto` for the raw protocol definitions used in this project.
153+
154+
.. literalinclude:: /../proto/control_plane/v1/seg.proto
155+
:caption: AS entry protobuf message definition.
156+
This data will be signed by the creating AS.
157+
A PCB is essentially a sequence of such signed AS entries.
158+
:language: proto
159+
:start-at: message ASEntrySignedBody {
160+
:end-at: }
161+
162+
.. literalinclude:: /../proto/control_plane/v1/seg.proto
163+
:caption: Hop field protobuf message definition. This is a part of the ``HopEntry``, refererred to
164+
in the ``ASEntrySignedBody`` definition above.
165+
:language: proto
166+
:start-at: message HopField {
167+
:end-at: }
168+
169+
Peering Links
170+
-------------
171+
172+
PCBs do not traverse peering links.
173+
Instead, available peering links are announced along with a regular path in the individual AS
174+
entries of PCBs.
175+
If both ASes at either end of a peering link have registered path segments that include a specific
176+
peering link, then it can be used to during segment combination to create an end-to-end path.
177+
178+
.. _control-plane-registration:
179+
180+
Registration of Path Segments
181+
=============================
182+
183+
**Path registration** is the process where an AS transforms selected PCBs into path segments,
184+
"terminating" them by adding a final AS entry with a zero egress interface,
185+
and adds these segments to the relevant path databases, thus making them available for the path
186+
lookup process.
187+
188+
As mentioned previously, a non-core AS typically receives several PCBs representing path segments to
189+
the core ASes of the ISD the AS belongs to.
190+
Out of these PCBs, the non-core AS selects those down-path segments through which it wants to be
191+
reached, based on AS-specific selection critera.
192+
The next step is to register the selected down-segments with the control service of the
193+
core AS that originated the PCB.
194+
195+
Intra-ISD Path-Segment Registration
196+
-----------------------------------
197+
198+
Every *registration period* (determined by each AS), the AS's control service selects of
199+
PCBs to transform into path segments:
200+
201+
- Up-segments, which allow the infrastructure entities and endpoints in this AS to communicate with
202+
core ASes.
203+
Up-segments are registered in the local path database of the AS.
204+
- Down-segments, which allow remote entities to reach this AS.
205+
Down-segments are registered, via a remote-procedure call, in the path-segment database of the
206+
core AS that originated the PCB.
207+
As a result, a core AS's path database contains all down-segments registered by their
208+
direct or indirect customer ASes.
209+
210+
Core Path-Segment Registration
211+
------------------------------
212+
213+
The core beaconing process creates PCBs from core AS to core AS.
214+
Every *registration period*, the AS's control service selects sets of PCBs to turn into path
215+
segments and register.
216+
These selected core-segments are added to the local path database of the core AS that created the
217+
segment (i.e. the one at the end of the beacon chain), so that local and remote endpoints can obtain
218+
and use these core-segments.
219+
In contrast to the down-segment registration procedure, there is no need to register core-segments
220+
with other core ASes (as each core AS will receive PCBs originated from every other core AS).
221+
222+
Path Lookup
223+
===========
224+
225+
An endpoint (source) that wants to start communication with another endpoint (destination), needs
226+
up to three path segments:
227+
228+
- An up-path segment to reach the core of the source ISD
229+
- a core-path segment to reach
230+
231+
- another core AS in the source ISD, in case the destination AS is in the same source ISD, or
232+
- a core AS in a remote ISD, if the destination AS is in another ISD, and
233+
234+
- a down-path segment to reach the destination AS.
235+
236+
The process to look up and fetch path segments consists of the following steps:
237+
238+
1. First, the source endpoint queries the control service in its own AS (i.e., the source AS) for
239+
the required segments.
240+
The control service has up-path segments stored in its path database.
241+
2. The control service in the source AS queries the control services of the reachable core ASes in
242+
the source ISD, for core-path segments to core ASes in the destination ISD (which is either the
243+
local or a remote ISD).
244+
To reach the core control services, the control service of the source AS uses the locally stored
245+
up-path segments.
246+
3. The control service then queries the control services of the remote core ASes in the destination
247+
ISD, to fetch down-path segments to the destination AS.
248+
To reach the remote core ASes, the control service of the source AS uses the previously obtained
249+
and combined up- and core segments.
250+
4. Finally, the control service of the source AS returns all retrieved path segments to the source
251+
endpoint.
252+
5. The endpoint combines all path segments into an end-to-end path
253+
254+
All remote path-segment lookups by the control service are cached.
255+
256+
On SCION end hosts, a :doc:`SCION daemon <manuals/daemon>` is usually employed to do the
257+
path-lookup on behalf of applications. This SCION daemon also caches path-segment lookup results.
258+
259+
.. table:: Control services responsible for different types of path segments
260+
261+
============ ===========================
262+
Segment Type Responsible control service(s)
263+
============ ===========================
264+
Up-segment Control service of the source AS
265+
Core-segment Control service of core ASes in source ISD
266+
Down-segment Control service of core ASes in destination ISD (either the local ISD or a remote ISD)
267+
============ ===========================
268+
269+
Path-Segment Combination
270+
========================
271+
272+
The last step of the path-resolution process is to combine the available up-, core- and down-
273+
path segments to end-to-end forwarding paths.
274+
This path-segment combination process is done by each endpoint separately.
275+
Typically, end hosts run the :doc:`SCION daemon <manuals/daemon>` which centralizes the
276+
path-resolution process and returns fully formed end-to-end paths to applications.
277+
However, applications could also choose to bypass the daemon and perform the path-resolution
278+
directly.
279+
280+
The figures below illustrate the various ways in which segments can be combined
281+
to form end-to-end paths.
282+
See the description of the :ref:`SCION Path<path-type-scion>` for the specifics on how these
283+
end-to-end paths are encoded in the packet header.
284+
285+
.. figure:: fig/beacon_metadata/path_combinations.png
286+
:alt: Path Combinations
287+
288+
Combination of path segments to paths: the blue circles represent the end
289+
hosts; the shaded gray circles represent core ASes, possibly in different
290+
ISDs; blue lines without arrow heads denote hops of created forwarding
291+
paths; the dashed blue line denotes a peering link (labeled "p"); orange
292+
lines with arrows stand for PCBs and indicate their dissemination direction;
293+
dashed orange lines represent core beacons exchanged over core links
294+
(labeled "c"). All created forwarding paths in cases 1a-1e traverse the ISD
295+
core(s), whereas the paths in cases 2-4 do not enter the ISD core.
296+
297+
298+
.. seealso::
299+
300+
:doc:`overview`
301+
Introduction to the SCION architecture and core concepts.
302+
303+
:doc:`data-plane`
304+
Description of SCION packet header formats and processing rules for packet forwarding based
305+
the packed-carried forwarding state.
18306

19-
Control plane overview and detailed description of beaconing, registration, lookup, and
20-
combination
307+
`IETF Draft SCION Control Plane <https://datatracker.ietf.org/doc/draft-dekater-scion-controlplane/>`_
308+
Formal description and specification of the SCION control plane.

0 commit comments

Comments
 (0)