Skip to content

Commit 9e8b319

Browse files
committed
Improve documentation about build configuration
This commit rearranges existing documentation and adds more explanation, including a comprehensive overview of the various parts (platforms, modifiers and transitions), and how they are relevant for end users. In particular, a lot of information was moved from `rule_authors/` to `concepts/`, as build configuration is important for end users. A lot of information was also deduplicated across pages.
1 parent c91c241 commit 9e8b319

File tree

8 files changed

+719
-640
lines changed

8 files changed

+719
-640
lines changed

docs/concepts/configurations.md

Lines changed: 364 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,355 @@ id: configurations
33
title: Configurations
44
---
55

6-
For rule authors see also: [Configurations](../rule_authors/configurations.md)
7-
8-
When building a target, buck always builds it in a particular "configuration."
9-
The configuration typically includes information like the target os, target
10-
arch, sanitizers, opt level, etc. One way to understand the effect that a
11-
configuration has is via the `cquery` and `uquery` commands. The cquery command
12-
will compute the appropriate configuration for a target and display a version of
13-
that target's attributes with the configuration applied. The `uquery` command
6+
Build configurations are how Buck models building the same target in
7+
different ways. This can include (but is not limited to):
8+
9+
- Target architecture
10+
- Target OS
11+
- Optimization level/build mode
12+
- Compiler type/version
13+
- Language version (e.g. C++ standard version)
14+
- Sanitizers
15+
- Passing arbitrary flags to build tools
16+
17+
When building a target, Buck always builds it in a particular
18+
configuration. Build configurations are also sometimes called
19+
"platforms". While technically separate, those two concepts are almost
20+
identical.
21+
22+
Build configurations are [composed](../api/build/Configuration) of a set
23+
of constraints and a set of values.
24+
25+
## Configuration constraints
26+
27+
Configuration constraints are enum-like constructs. Here is an example
28+
definitions:
29+
30+
```python
31+
# //config/BUCK
32+
33+
constraint(
34+
name = "build_mode",
35+
default = "debug",
36+
values = [
37+
"debug",
38+
"release",
39+
],
40+
)
41+
```
42+
43+
This will generate two configuration targets:
44+
`//config:build_mode[debug]` and `//config:build_mode[release]`.
45+
46+
Note that extra contraint values can be added outside of the main
47+
`constraint` definition:
48+
49+
```python
50+
# //somewhere/else/BUCK
51+
52+
constraint_value(
53+
name = "release_no_debug_info",
54+
constraint_setting = "//config:build_mode",
55+
)
56+
```
57+
58+
Now, the three configuration targets that can be used to control the
59+
build mode would be:
60+
61+
- `//config:build_mode[debug]`
62+
- `//config:build_mode[release]`
63+
- `//somewhere/else:release_no_debug_info`
64+
65+
Constraint values can also be grouped into larger logical pieces.
66+
Assuming that we have also defined other constraints:
67+
68+
```python
69+
config_setting(
70+
name = "dev",
71+
constraint_values = [
72+
":build_mode[debug]",
73+
":compiler[clang_21]",
74+
":asan[enabled]",
75+
],
76+
)
77+
```
78+
79+
Note that the prelude defines some generic constraints, e.g. under
80+
`prelude//os:` and `prelude//cpu:`, which you might want to consider
81+
using for interoperability.
82+
83+
Once defined, this constraint can used in various ways, such as:
84+
85+
- Being passed on the command line to run a build in debug or release
86+
mode.
87+
- Being "selected on" so that building in debug vs release mode has
88+
different effects.
89+
- Being used for constraining compatibility (e.g. "this target can only
90+
be built in release mode" for a benchmark).
91+
- Being used to "transition" part of the build (e.g. "this target and
92+
its dependencies are always built in release mode, regardless of the
93+
dependent")
94+
95+
## Configuration values
96+
97+
`config_setting` can also include values taken from the buckconfig.
98+
These can ease a migration from a legacy buckconfig setting to a build
99+
constraint by allowing you to `select()` (more on that later) on known
100+
buckconfig values:
101+
102+
```python
103+
config_setting(
104+
name = "fastmode_enabled",
105+
values = {
106+
"build.fastmode": "true",
107+
},
108+
)
109+
```
110+
111+
This setting will be satisfied if the associated buckconfig matches,
112+
i.e. if the user passes `build.fastmode=true` via the `-c`/`--config`
113+
CLI flag, or if the following is set in the cell's `.buckconfig` file:
114+
115+
```ini
116+
[build]
117+
fastmode = true
118+
```
119+
120+
This feature only allows reading buckconfig values, not reading them.
121+
122+
They are also incompatible with [configuration modifiers](./modifiers.md):
123+
`--modifier :fastmode_enabled` does nothing.
124+
125+
## Using configuration: `select()`
126+
127+
Configurations can be used to change the build behavior based on which
128+
value is currently active:
129+
130+
```cpp
131+
cxx_binary(
132+
name = "bin",
133+
srcs = ["main.cpp"],
134+
compiler_flags = select({
135+
"//config:build_mode[debug]": ["-O0", "-g"],
136+
"//config:build_mode[release]": ["-O3"],
137+
}),
138+
)
139+
```
140+
141+
The above example is simplistic, and build mode compiler flags would
142+
typically be set at the toolchain level, rather than per-target, but it
143+
shows how build constraints can be used to change a build's behavior.
144+
145+
`select()` can appear in almost all attributes, and it can be composed
146+
with other collection types. For example, the following is valid:
147+
148+
```python
149+
cxx_library(
150+
name = "lib",
151+
exported_deps = [
152+
"//common:lib",
153+
] + select({
154+
"//config:os[linux]": ["//linux:lib"],
155+
"//config:os[mac]": ["//mac:lib"],
156+
# `DEFAULT` is a special value that is always available.
157+
# In this case, we do not link against any extra libraries.
158+
"DEFAULT": [],
159+
}),
160+
)
161+
```
162+
163+
If only one condition matches, the `select()` resolves to that
164+
condition.
165+
166+
If multiple conditions match, then the select will be resolved to the
167+
"most refined" of the conditions that match. A set of constraints (as in
168+
a `config_setting`) is said to "refine" another if it is a superset of
169+
that other's constraints. The "most refined" of a set is then the
170+
condition that refines all the others.
171+
172+
Note that `select()` is resolved during configuration. This happens
173+
after the evaluation of the BUCK file is completed, and so Starlark code
174+
run during BUCK file evaluation does not have access to the resolved
175+
value. This can make it difficult to have macros that do extensive
176+
modification or inspection of attributes (which should be done in rules
177+
instead). However, some functions
178+
([`select_map`](../api/build/#select_map) and
179+
[`select_test`](../api/build/#select_test)) allow performing
180+
limited operations on these objects.
181+
182+
## Using configuration: compatibility
183+
184+
Constraints can also be used to limit target compatibility. For example,
185+
assuming that our repo supports C++20, C++23 and C++26:
186+
187+
```python
188+
# Reflection is only available starting with C++26, so we require it.
189+
cxx_library(
190+
name = "uses_reflection",
191+
exported_headers = ["foo.h"],
192+
target_compatible_with = ["//:cxx_standard[26]"]
193+
)
194+
195+
# Deducing this is not available in C++20, so we make it incompatible.
196+
cxx_library(
197+
name = "uses_deducing_this",
198+
exported_headers = ["foo.h"],
199+
target_compatible_with = select({
200+
"//:cxx_standard[20]": ["prelude//:none"],
201+
"DEFAULT": [],
202+
})
203+
)
204+
```
205+
206+
Target compatibility requires all transitive dependencies to be
207+
compatible as well. In other words, a node is compatible if and only if
208+
the node itself and all of its transitive dependencies are compatible.
209+
In the usual cases of a dependency via `attrs.dep()`, a target's
210+
dependency will be configured and then checked for compatibility with
211+
the same configuration as the dependent target.
212+
213+
When trying to build a target with the wrong configuration (we will see
214+
how shortly), the build will just fail (unless
215+
`--skip-incompatible-targets` is passed).
216+
217+
When trying to build a set of targets using a
218+
[pattern](./target_pattern)) (e.g. `//some/package:` or
219+
`//some/package/...`), Buck will simply ignore incompatible targets.
220+
221+
See the [reference
222+
documentation](../api/build/Select/#target_compatible_with) for more
223+
information.
224+
225+
## Changing the build configuration
226+
227+
The build configuration is determined as follows:
228+
229+
1. A base platform is resolved:
230+
1. If the user passed `--target-platforms` via the CLI, use that.
231+
2. Else, if the target being built has a `default_target_platform`
232+
attribute, use that. Note that since it is used to determine the
233+
configuration, it is one of the few attributes that are not
234+
`select`able.
235+
3. Else, use the default (`parser.target_platform_detector_spec` in
236+
the `.buckconfig` file).
237+
2. [Configuration modifiers](./modifiers.md) are applied. Those are a
238+
lightweight way to add ad-hoc constraints to an existing
239+
configuration (e.g. "build with the default configuration/platform,
240+
except with a different compiler").
241+
3. [Configuration transitions](./transitions.md) are applied. Those
242+
allow changing the configuration of parts of the build graph based on
243+
arbitrary logic (e.g. "this part of the build graph should always be
244+
built in release mode").
245+
246+
The target platform resolution is not applied to all nodes in the graph.
247+
Once the top-level nodes have been configured via the target platform
248+
resolution, the configuration is propagated to dependencies (possibly
249+
altered by transitions).
250+
251+
For example:
252+
253+
```sh
254+
# Build this target with the default configuration.
255+
buck2 build :my_target
256+
# Build it with an entirely different configuration.
257+
buck2 build :my_target --target-platforms //my/other:platform
258+
# Build it with the default configuration, plus release mode.
259+
buck2 build :my_target?release
260+
# Equivalent to the above, but applies to all targets if multiple were built.
261+
buck2 build :my_target -m release
262+
```
263+
264+
See the [configurations for author](../rule_authors/configurations.md)
265+
page for information on how to define a platform.
266+
267+
Other example:
268+
269+
```python
270+
java_binary(
271+
name = "cats",
272+
default_target_platform = "//platforms:windows-arm64-dev",
273+
deps = ["//libs:foo"],
274+
)
275+
276+
java_binary(
277+
name = "dogs",
278+
default_target_platform = "//platforms:mac-x86-dev",
279+
deps = ["//libs:foo"],
280+
)
281+
282+
java_library(
283+
name = "foo",
284+
deps = [
285+
"//libs:common",
286+
] + select({
287+
"//constraints:x86": ["//libs:x86"],
288+
"//constraints:mac-arm64": ["//libs:mac-arm64"],
289+
"//constraints:windows-arm64": ["//libs:win-arm64"],
290+
"DEFAULT": ["//libs:generic"],
291+
})
292+
)
293+
```
294+
295+
When running `buck2 build //binaries:cats //binaries:dogs`, the
296+
`//binaries:cats` binary will be built in the `//platforms:windows-arm64-dev`
297+
configuration and the `//binaries:dogs` binary will be built in the
298+
`//platforms:mac-x86-dev` configuration.
299+
300+
Each of those binaries depend on `//libs:foo`, but they will get
301+
different versions of it as the binaries' configurations will each be
302+
passed down to their dependencies. For `//binaries:cats`, its resolved
303+
dependencies will include `//libs:win-arm64` and for `//binaries:dogs`,
304+
it would contain `//libs:x86`.
305+
306+
Note that `//libs:common` will be built twice, once for each
307+
configuration.
308+
309+
When running `buck2 build //binaries:cats //binaries:dogs --target-platforms
310+
//platforms:mac-x86-opt`, both `//binaries:cats` and `//binaries:dogs` will
311+
be built in the `//platforms:mac-x86-opt` configuration, use the same
312+
dependencies, which would only be built once.
313+
314+
## Configurations and output paths
315+
316+
Since a target may appear within a build in multiple different configurations,
317+
output paths cannot be derived based on just targets (as multiple actions would
318+
map to the same outputs). For this reason, the target and the configuration are
319+
encoded into output paths. The configuration is currently represented as a hash
320+
of its values (a "hashed buck-out").
321+
322+
## Target platform vs execution platform
323+
324+
Buck distinguishes two kinds of platforms: "regular" ones (where your
325+
code will run), and the ones used to run compilers and other tools.
326+
Those are distinct because it is typical to want build tools to use a
327+
different build configuration. For example, you may want a compiler to
328+
be built/run in release mode, even when building debug
329+
targets.
330+
331+
For this reason, Buck requires both _target_ platforms and _execution_
332+
platforms to be defined. The execution platforms are specified via the
333+
`build.execution_platforms` value in `.buckconfig`.
334+
335+
## Queries
336+
337+
### Getting configuration constraints from its hash
338+
339+
Build configurations are uniquely identified by their hash, which is not
340+
human friendly.
341+
342+
To determine what constraints are part of a configuration, run `buck2
343+
cquery //...` sot that Buck will discover all existing configurations,
344+
then run `buck2 audit configurations`.
345+
346+
This will list all available configurations and print their composing
347+
contraints.
348+
349+
### `cquery` and `uquery`
350+
351+
One way to understand the effect that a configuration has is via the
352+
`cquery` and `uquery` commands. The `cquery` command will compute the
353+
appropriate configuration for a target and display a version of that
354+
target's attributes with the configuration applied. The `uquery` command
14355
will not apply a configuration.
15356

16357
Here is a heavily trimmed version of the outputs of invoking `uquery` and
@@ -89,3 +430,18 @@ while the `nix` dependency is needed only for Linux. In `cquery` that
89430
distinction has been resolved; because the target has been configured for Linux,
90431
the `nix` dependency is present and indistinguishable from any other, while the
91432
`common-path` dependency is gone.
433+
434+
## Execution groups
435+
436+
Execution groups are a future feature that will allow a rule to perform
437+
execution platform resolution multiple times and then specify in which of the
438+
resolved platforms each action runs in.
439+
440+
Traditionally, each target resolves a single execution platform.
441+
442+
## See also
443+
444+
- [Configuration modifiers](./modifiers.md)
445+
- [Configuration transitions](./transitions.md)
446+
- [Configurations for rule authors](../rule_authors/configurations.md)
447+
- [Configuration transitions for rule authors](../rule_authors/configuration_transitions.md)

docs/concepts/modifiers.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,12 @@ title: Configuration modifiers
55

66
Modifiers (also referred to as configuration modifiers) are a feature that lets
77
users add [constraints](../../rule_authors/configurations) to individual
8-
directories, target definitions and individual `buck2` invocations. They are the
9-
recommended to customize build configuration.
8+
directories, target definitions and individual `buck2` invocations.
9+
10+
They are the recommended to customize build configurations when building
11+
targets directly. If you need to customize parts of your build graph
12+
(e.g. always build some specific dependencies in release mode),
13+
[configuration transitions](transitions.md) are more appropriate.
1014

1115
## (Open-Source Only) Getting started with modifiers
1216

0 commit comments

Comments
 (0)