From 9e7f4b935a0de5f98d06a582b256909193b486e1 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 9 Mar 2023 12:49:04 -0500 Subject: [PATCH 01/13] Add beginning of design for simplifying workload versioning --- accepted/2023/simplify-workload-versioning.md | 140 ++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 accepted/2023/simplify-workload-versioning.md diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md new file mode 100644 index 000000000..a717b0d7f --- /dev/null +++ b/accepted/2023/simplify-workload-versioning.md @@ -0,0 +1,140 @@ +# Simplify Workload Versioning + +.NET SDK Workloads are optional components of the .NET SDK. When designing workloads, one of the goals was to be able to update workloads separately from the .NET SDK. As a result, workloads version and update separately from the .NET SDK. This has made it hard to understand what versions of workloads are installed or available. Complicating this is the fact that workloads themselves don't directly have a version. Workload manifests, which can have a many-to-many relationship with workloads, have versions. These versions can be displayed with the `dotnet workload update --print-rollback` command, which displays output in the following format: + +``` +==workloadRollbackDefinitionJsonOutputStart== +{ + "microsoft.net.sdk.android": "33.0.26/7.0.100", + "microsoft.net.sdk.ios": "16.2.1030/7.0.100", + "microsoft.net.sdk.maccatalyst": "16.2.1030/7.0.100", + "microsoft.net.sdk.macos": "13.1.1030/7.0.100", + "microsoft.net.sdk.maui": "7.0.59/7.0.100", + "microsoft.net.sdk.tvos": "16.1.1527/7.0.100", + "microsoft.net.workload.mono.toolchain.net6": "7.0.3/7.0.100", + "microsoft.net.workload.mono.toolchain.net7": "7.0.3/7.0.100", + "microsoft.net.workload.emscripten.net6": "7.0.3/7.0.100", + "microsoft.net.workload.emscripten.net7": "7.0.3/7.0.100" +} +==workloadRollbackDefinitionJsonOutputEnd== +``` + +This is very complicated and doesn't make it easy to understand workload versions. + +## Goals + +- Introduce a single version number that represents all workload versions for the the .NET SDK +- Workloads can continue to release outside the normal .NET SDK release schedule +- Deadlines for workload completion and insertion do not change from current processes +- Workload updates should require minimal (or no) additional validation of non-workload scenarios + +## Workload patch versions + +Currently, we use a 3 part version number for the .NET SDK, for example 8.0.100. Typically we release a patch each month, for example 8.0.101, 8.0.102, etc. + +We will create a workload patch version number that encapsulates all the workload manifest versions that are released together. We will represent the workload patch version by adding an additional component to the .NET SDK version. The versions of the workloads that release on the same release date as the .NET SDK will be number `.0`. Subsequent workload updates before the next .NET SDK patch will be numbered `.1`, `.2`, etc. For example, the full workload patch version for the initial 8.0 release would be `8.0.100.0`, and if there was a workload update before the first .NET SDK patch, the updated workload patch version would be `8.0.100.1`. + +A workload patch is essentially a mapping from the workload patch version to the different versions for each workload manifest. Creating a workload patch should be a lightweight process, which involves creating a NuGet packages and corresponding installers to deliver this mapping. The .NET SDK assets and installers, workload manifests, and workload packs should already be built and should not need to be created when building a workload patch. + +## Experience + +`dotnet --version` would continue to print the SDK version, as it does today: + +``` +> dotnet --version +8.0.201 +``` + +We will add a new `dotnet workload --version` command to print the workload patch version: + +``` +> dotnet workload --version +8.0.201.2 +``` + +We will also update `dotnet --info` and `dotnet workload --info` to display the workload patch version. For example: + +``` +> dotnet --info +.NET SDK: + Version: 8.0.201 + Commit: + Workloads version: 8.0.201.2 + + + +> dotnet workload --info + Workloads version: 8.0.201.2 + [wasm-tools] + Installation Source: SDK 8.0.100-preview.4 + Manifest Version: 8.0.0-preview.4.23181.9/8.0.100-preview.4 + Manifest Path: C:\git\dotnet-sdk-main\.dotnet\sdk-manifests\8.0.100-preview.4\microsoft.net.workload.mono.toolchain.current\WorkloadManifest.json + Install Type: FileBased +``` + +When updating workloads, console output will include the old and new workload patch versions: + +``` +> dotnet workload install wasm-tools +Checking for updated workloads version... +Updating workloads version from 8.0.201.0 to 8.0.201.2... +Installing workload manifest microsoft.net.sdk.android version 34.0.0-preview.4.230... + +Installing pack Microsoft.NET.Runtime.WebAssembly.Sdk version 8.0.0-preview.4.23181.9... + +Garbage collecting for SDK feature band(s) 8.0.200... + +Successfully updated workloads version from 8.0.201.0 to 8.0.201.2. +Successfully installed workload(s) wasm-tools. +``` + +The proposed [workload history](https://github.com/dotnet/sdk/pull/30486) command will also display the workload patch versions. + +The .NET SDK will periodically (once a day, by default) check for updated workload versions in order to notify users if there is an update available. Currently, commands such as `dotnet build` print out the following message: + +``` +Workload updates are available. Run `dotnet workload list` for more information. +``` + +The `dotnet workload list` command, in turn, prints out the following message: + +``` +Updates are available for the following workload(s): wasm-tools. Run `dotnet workload update` to get the latest. +``` + +We will modify both commands to print the same message: + +``` +A workload update (version 8.0.201.2) is available. Run `dotnet workload update` to install the latest workloads. +``` + +We will add a new `--version` option to `dotnet workload update` to allow installing (or downgrading to) a specific workloads version: + +``` +> dotnet workload update --version 8.0.201.0 +Updating workloads version from 8.0.201.2 to 8.0.201.0... + +Successfully updated workloads version from 8.0.201.2 to 8.0.201.0. +``` + +## Specifying workload versions with global.json + +We will add support for specifying the workloads version in global.json. For example: + +```json +{ + "sdk": { + "version": "8.0.201.0", + "rollForward": "disable" + } +} +``` + +This would force the SDK to use workloads version `8.0.201.0`, and would error if that version was not installed. + +We will support side-by-side workload version installations. If 8.0.201.2 is installed, we would support running `dotnet workload install --version 8.0.201.0` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workloads version. An earlier workloads version would only be used if it was specified in a global.json file. + +NOTE: Various tools (such Azure DevOps and [GitHub actions](github.com/actions/setup-dotnet)) read global.json in order to install the right version of the .NET SDK. Those tools would need to be updated ignore the fourth section of the SDK version number. + +NOTE: We may not implement the global.json workloads version support in the same release as the rest of the changes described in this design. + From e49df94ce0c7920d9f3edb4a07255ebd604a6be8 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Wed, 19 Apr 2023 13:27:51 -0400 Subject: [PATCH 02/13] Switch to using SDK version numbers for workloads versions --- accepted/2023/simplify-workload-versioning.md | 57 ++++++++++++------- 1 file changed, 35 insertions(+), 22 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index a717b0d7f..c1b553219 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -28,13 +28,29 @@ This is very complicated and doesn't make it easy to understand workload version - Deadlines for workload completion and insertion do not change from current processes - Workload updates should require minimal (or no) additional validation of non-workload scenarios -## Workload patch versions +## Workloads versions Currently, we use a 3 part version number for the .NET SDK, for example 8.0.100. Typically we release a patch each month, for example 8.0.101, 8.0.102, etc. -We will create a workload patch version number that encapsulates all the workload manifest versions that are released together. We will represent the workload patch version by adding an additional component to the .NET SDK version. The versions of the workloads that release on the same release date as the .NET SDK will be number `.0`. Subsequent workload updates before the next .NET SDK patch will be numbered `.1`, `.2`, etc. For example, the full workload patch version for the initial 8.0 release would be `8.0.100.0`, and if there was a workload update before the first .NET SDK patch, the updated workload patch version would be `8.0.100.1`. +We will create a workloads version number that encapsulates all the workload manifest versions that are released together. A workloads version is essentially a mapping from the workloads version to the different versions for each workload manifest. For public releases, the workloads version will use the same version number as the .NET SDK. For example, when .NET SDK 8.0.101 releases, there will be a corresponding workloads version 8.0.101 that corresponds to the versions of the workloads that were released together with that .NET SDK. -A workload patch is essentially a mapping from the workload patch version to the different versions for each workload manifest. Creating a workload patch should be a lightweight process, which involves creating a NuGet packages and corresponding installers to deliver this mapping. The .NET SDK assets and installers, workload manifests, and workload packs should already be built and should not need to be created when building a workload patch. +## Baseline workload versions + +.NET SDK workloads have similar shipping deadlines as the .NET SDK. That means that the .NET SDK itself can't include final versions of workloads which are built outside of the .NET build, as building and signing off on those workloads happens in parallel with .NET build and signoff. So the .NET SDK ships with *baseline* workload manifests, which are mainly used to enable the .NET SDK to list which workloads are available, and to know if a project requires a workload which isn't installed. By default, when a workload is installed via the .NET CLI, the .NET SDK will first look for and update to the latest workload manifests for the current feature band before installing workloads. + +When we build the .NET SDK, we will assign a workloads version to the baseline workload manifest versions that are included in that SDK. For releases with stabilized version numbers, we need a different workloads version number for the baseline workload manifest versions than for the stabilized version number for the .NET SDK. In that case, we will use `baseline` as a the first semantic version pre-release identifier, followed by build number information. For example, the baseline workloads version for 8.0.201 could be `8.0.201-baseline.23919.2`. + +For non-stabilized builds of the .NET SDK, we will use the same (prerelease) version number for the workloads version as we do for the .NET SDK version. + +QUESTION: Will it be OK for the `baseline` versions to be semantically less than `preview` and `rc` versions? + +## Workload versions in Visual Studio + +Visual Studio includes the .NET SDK, but rather than including baseline manifest versions for the .NET SDK workloads, it includes the final workload manifests for a given release. These manifests are inserted into Visual Studio together with the workloads. + +For public releases of Visual Studio, the workloads version information should be created and inserted into Visual Studio together with the workloads. However, internal builds of Visual Studio may not have an assigned workloads version. In that case, the .NET SDK will use the same basic logic as it currently does to select manifests. This involves finding the latest installed manifest for the current feature band, and if an expected manifest isn't found for the current feature band, then falling back to previous feature bands. In this case, when a workloads version is needed, the .NET SDK will create a version using the .NET SDK feature band, the pre-release specifier `vs`, and a hash of the manifest IDs and versions. For example, `8.0.200-vs.9e7f4b93`. + +NOTE: This means that these `vs` prerelase workloads versions would be semantically greater than any `baseline`, `preview`, or `rc` workloads versions. ## Experience @@ -45,26 +61,26 @@ A workload patch is essentially a mapping from the workload patch version to the 8.0.201 ``` -We will add a new `dotnet workload --version` command to print the workload patch version: +We will add a new `dotnet workload --version` command to print the workloads version: ``` > dotnet workload --version -8.0.201.2 +8.0.201 ``` -We will also update `dotnet --info` and `dotnet workload --info` to display the workload patch version. For example: +We will also update `dotnet --info` and `dotnet workload --info` to display the workloads version. For example: ``` > dotnet --info .NET SDK: Version: 8.0.201 Commit: - Workloads version: 8.0.201.2 + Workloads version: 8.0.201 > dotnet workload --info - Workloads version: 8.0.201.2 + Workloads version: 8.0.201 [wasm-tools] Installation Source: SDK 8.0.100-preview.4 Manifest Version: 8.0.0-preview.4.23181.9/8.0.100-preview.4 @@ -72,23 +88,23 @@ We will also update `dotnet --info` and `dotnet workload --info` to display the Install Type: FileBased ``` -When updating workloads, console output will include the old and new workload patch versions: +When updating workloads, console output will include the old and new workloads versions: ``` > dotnet workload install wasm-tools Checking for updated workloads version... -Updating workloads version from 8.0.201.0 to 8.0.201.2... +Updating workloads version from 8.0.201-baseline.23919.2 to 8.0.201... Installing workload manifest microsoft.net.sdk.android version 34.0.0-preview.4.230... Installing pack Microsoft.NET.Runtime.WebAssembly.Sdk version 8.0.0-preview.4.23181.9... Garbage collecting for SDK feature band(s) 8.0.200... -Successfully updated workloads version from 8.0.201.0 to 8.0.201.2. +Successfully updated workloads version from 8.0.201-baseline.23919.2 to 8.0.201.2. Successfully installed workload(s) wasm-tools. ``` -The proposed [workload history](https://github.com/dotnet/sdk/pull/30486) command will also display the workload patch versions. +The proposed [workload history](https://github.com/dotnet/sdk/pull/30486) command will also display the workloads versions. The .NET SDK will periodically (once a day, by default) check for updated workload versions in order to notify users if there is an update available. Currently, commands such as `dotnet build` print out the following message: @@ -105,16 +121,16 @@ Updates are available for the following workload(s): wasm-tools. Run `dotnet wor We will modify both commands to print the same message: ``` -A workload update (version 8.0.201.2) is available. Run `dotnet workload update` to install the latest workloads. +A workload update (version 8.0.202) is available. Run `dotnet workload update` to install the latest workloads. ``` We will add a new `--version` option to `dotnet workload update` to allow installing (or downgrading to) a specific workloads version: ``` -> dotnet workload update --version 8.0.201.0 -Updating workloads version from 8.0.201.2 to 8.0.201.0... +> dotnet workload update --version 8.0.202 +Updating workloads version from 8.0.201 to 8.0.202... -Successfully updated workloads version from 8.0.201.2 to 8.0.201.0. +Successfully updated workloads version from 8.0.201 to 8.0.202. ``` ## Specifying workload versions with global.json @@ -124,17 +140,14 @@ We will add support for specifying the workloads version in global.json. For ex ```json { "sdk": { - "version": "8.0.201.0", - "rollForward": "disable" + "workloadsVersion": "8.0.201" } } ``` -This would force the SDK to use workloads version `8.0.201.0`, and would error if that version was not installed. - -We will support side-by-side workload version installations. If 8.0.201.2 is installed, we would support running `dotnet workload install --version 8.0.201.0` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workloads version. An earlier workloads version would only be used if it was specified in a global.json file. +This would force the SDK to use workloads version `8.0.201`, and would error if that version was not installed. -NOTE: Various tools (such Azure DevOps and [GitHub actions](github.com/actions/setup-dotnet)) read global.json in order to install the right version of the .NET SDK. Those tools would need to be updated ignore the fourth section of the SDK version number. +We will support side-by-side workload version installations. If 8.0.202 is installed, we would support running `dotnet workload install --version 8.0.201` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workloads version. An earlier workloads version would only be used if it was specified in a global.json file. NOTE: We may not implement the global.json workloads version support in the same release as the rest of the changes described in this design. From 892da872175e1d16eb4cbd377bceccc5fc94e125 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 27 Apr 2023 10:55:16 -0400 Subject: [PATCH 03/13] Update design --- accepted/2023/simplify-workload-versioning.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index c1b553219..1a99f0055 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -40,6 +40,12 @@ We will create a workloads version number that encapsulates all the workload man When we build the .NET SDK, we will assign a workloads version to the baseline workload manifest versions that are included in that SDK. For releases with stabilized version numbers, we need a different workloads version number for the baseline workload manifest versions than for the stabilized version number for the .NET SDK. In that case, we will use `baseline` as a the first semantic version pre-release identifier, followed by build number information. For example, the baseline workloads version for 8.0.201 could be `8.0.201-baseline.23919.2`. +There are various ways we could display this baseline version number: + +- `8.0.201-baseline.23919.2` +- `8.0.201-baseline` +- `8.0.201 (Baseline)` + For non-stabilized builds of the .NET SDK, we will use the same (prerelease) version number for the workloads version as we do for the .NET SDK version. QUESTION: Will it be OK for the `baseline` versions to be semantically less than `preview` and `rc` versions? @@ -48,9 +54,7 @@ QUESTION: Will it be OK for the `baseline` versions to be semantically less than Visual Studio includes the .NET SDK, but rather than including baseline manifest versions for the .NET SDK workloads, it includes the final workload manifests for a given release. These manifests are inserted into Visual Studio together with the workloads. -For public releases of Visual Studio, the workloads version information should be created and inserted into Visual Studio together with the workloads. However, internal builds of Visual Studio may not have an assigned workloads version. In that case, the .NET SDK will use the same basic logic as it currently does to select manifests. This involves finding the latest installed manifest for the current feature band, and if an expected manifest isn't found for the current feature band, then falling back to previous feature bands. In this case, when a workloads version is needed, the .NET SDK will create a version using the .NET SDK feature band, the pre-release specifier `vs`, and a hash of the manifest IDs and versions. For example, `8.0.200-vs.9e7f4b93`. - -NOTE: This means that these `vs` prerelase workloads versions would be semantically greater than any `baseline`, `preview`, or `rc` workloads versions. +For public releases of Visual Studio, the workloads version information should be created and inserted into Visual Studio together with the workloads. However, internal builds of Visual Studio may not have an assigned workloads version. In that case, the .NET SDK will use the same basic logic as it currently does to select manifests. This involves finding the latest installed manifest for the current feature band, and if an expected manifest isn't found for the current feature band, then falling back to previous feature bands. In this case, when a workloads version is needed, the .NET SDK will create a version using the .NET SDK feature band, the pre-release specifier `manifests`, and a hash of the manifest IDs and versions. For example, `8.0.200-manifests.9e7f4b93`. ## Experience @@ -100,7 +104,7 @@ Installing pack Microsoft.NET.Runtime.WebAssembly.Sdk version 8.0.0-preview.4.23 Garbage collecting for SDK feature band(s) 8.0.200... -Successfully updated workloads version from 8.0.201-baseline.23919.2 to 8.0.201.2. +Successfully updated workloads version from 8.0.201-baseline.23919.2 to 8.0.201. Successfully installed workload(s) wasm-tools. ``` @@ -151,3 +155,10 @@ We will support side-by-side workload version installations. If 8.0.202 is inst NOTE: We may not implement the global.json workloads version support in the same release as the rest of the changes described in this design. +## Development build behavior + +Currently, when the .NET SDK looks for workload updates, it looks for the latest available version of each individual manifest. We will change this so that it will look for the latest "workloads version", which is a package that we create and publish. + +However, for development builds of the .NET SDK, we may want to retain the old behavior of updating each manifest individually. This would allow the latest workloads to be used with a development build even if they hadn't flowed into the installer build and we hadn't created a "workloads version" package. + +We will add a `--update-manifests-separately` parameter to the `dotnet workload update` and `install` commands, which will opt in to the old update behavior. We may also make this the default behavior for development builds, though there may not be a good way to tell whether a given build is a "development" build or a build that will be released as an official public preview. If development builds aren't signed the same way as full preview builds are, we may be able to key off of that. \ No newline at end of file From f4033fd2cd8c44a551b712dded121c9d17dbb8f6 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Wed, 17 May 2023 13:29:59 -0400 Subject: [PATCH 04/13] Update terminology to worlkoad sets --- accepted/2023/simplify-workload-versioning.md | 52 +++++++++---------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index 1a99f0055..5cb2d0b7b 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -28,25 +28,25 @@ This is very complicated and doesn't make it easy to understand workload version - Deadlines for workload completion and insertion do not change from current processes - Workload updates should require minimal (or no) additional validation of non-workload scenarios -## Workloads versions +## Workload sets Currently, we use a 3 part version number for the .NET SDK, for example 8.0.100. Typically we release a patch each month, for example 8.0.101, 8.0.102, etc. -We will create a workloads version number that encapsulates all the workload manifest versions that are released together. A workloads version is essentially a mapping from the workloads version to the different versions for each workload manifest. For public releases, the workloads version will use the same version number as the .NET SDK. For example, when .NET SDK 8.0.101 releases, there will be a corresponding workloads version 8.0.101 that corresponds to the versions of the workloads that were released together with that .NET SDK. +We will create the concept of a "workload set" which has a version number that encapsulates all the workload manifest versions that are released together. A workload set is essentially a mapping from the workload set version to the different versions for each workload manifest. For public releases, the workload set version will use the same version number as the .NET SDK. For example, when .NET SDK 8.0.101 releases, there will be a corresponding workload set version 8.0.101 that corresponds to the versions of the workloads that were released together with that .NET SDK. ## Baseline workload versions .NET SDK workloads have similar shipping deadlines as the .NET SDK. That means that the .NET SDK itself can't include final versions of workloads which are built outside of the .NET build, as building and signing off on those workloads happens in parallel with .NET build and signoff. So the .NET SDK ships with *baseline* workload manifests, which are mainly used to enable the .NET SDK to list which workloads are available, and to know if a project requires a workload which isn't installed. By default, when a workload is installed via the .NET CLI, the .NET SDK will first look for and update to the latest workload manifests for the current feature band before installing workloads. -When we build the .NET SDK, we will assign a workloads version to the baseline workload manifest versions that are included in that SDK. For releases with stabilized version numbers, we need a different workloads version number for the baseline workload manifest versions than for the stabilized version number for the .NET SDK. In that case, we will use `baseline` as a the first semantic version pre-release identifier, followed by build number information. For example, the baseline workloads version for 8.0.201 could be `8.0.201-baseline.23919.2`. +When we build the .NET SDK, we will assign a workload set version to the baseline workload manifest versions that are included in that SDK. For releases with stabilized version numbers, we need a different workload set version number for the baseline workload manifest versions than for the stabilized version number for the .NET SDK. In that case, we will use `baseline` as a the first semantic version pre-release identifier, followed by build number information. For example, the baseline workload set version for 8.0.201 could be `8.0.201-baseline.23919.2`. -There are various ways we could display this baseline version number: +There are various ways we could display this baseline workload set version number: - `8.0.201-baseline.23919.2` - `8.0.201-baseline` - `8.0.201 (Baseline)` -For non-stabilized builds of the .NET SDK, we will use the same (prerelease) version number for the workloads version as we do for the .NET SDK version. +For non-stabilized builds of the .NET SDK, we will use the same (prerelease) version number for the baseline workload set version as we do for the .NET SDK version. QUESTION: Will it be OK for the `baseline` versions to be semantically less than `preview` and `rc` versions? @@ -54,7 +54,7 @@ QUESTION: Will it be OK for the `baseline` versions to be semantically less than Visual Studio includes the .NET SDK, but rather than including baseline manifest versions for the .NET SDK workloads, it includes the final workload manifests for a given release. These manifests are inserted into Visual Studio together with the workloads. -For public releases of Visual Studio, the workloads version information should be created and inserted into Visual Studio together with the workloads. However, internal builds of Visual Studio may not have an assigned workloads version. In that case, the .NET SDK will use the same basic logic as it currently does to select manifests. This involves finding the latest installed manifest for the current feature band, and if an expected manifest isn't found for the current feature band, then falling back to previous feature bands. In this case, when a workloads version is needed, the .NET SDK will create a version using the .NET SDK feature band, the pre-release specifier `manifests`, and a hash of the manifest IDs and versions. For example, `8.0.200-manifests.9e7f4b93`. +For public releases of Visual Studio, the workload set version information should be created and inserted into Visual Studio together with the workloads. However, internal builds of Visual Studio may not have an assigned workload set version. In that case, the .NET SDK will use the same basic logic as it currently does to select manifests. This involves finding the latest installed manifest for the current feature band, and if an expected manifest isn't found for the current feature band, then falling back to previous feature bands. In this case, when a workload set version is needed, the .NET SDK will create a version using the .NET SDK feature band, the pre-release specifier `manifests`, and a hash of the manifest IDs and versions. For example, `8.0.200-manifests.9e7f4b93`. ## Experience @@ -65,26 +65,26 @@ For public releases of Visual Studio, the workloads version information should b 8.0.201 ``` -We will add a new `dotnet workload --version` command to print the workloads version: +We will add a new `dotnet workload --version` command to print the workload set version: ``` > dotnet workload --version 8.0.201 ``` -We will also update `dotnet --info` and `dotnet workload --info` to display the workloads version. For example: +We will also update `dotnet --info` and `dotnet workload --info` to display the workload set version. For example: ``` > dotnet --info .NET SDK: - Version: 8.0.201 - Commit: - Workloads version: 8.0.201 + Version: 8.0.201 + Commit: + Workload set version: 8.0.201 > dotnet workload --info - Workloads version: 8.0.201 + Workload set version: 8.0.201 [wasm-tools] Installation Source: SDK 8.0.100-preview.4 Manifest Version: 8.0.0-preview.4.23181.9/8.0.100-preview.4 @@ -92,23 +92,23 @@ We will also update `dotnet --info` and `dotnet workload --info` to display the Install Type: FileBased ``` -When updating workloads, console output will include the old and new workloads versions: +When updating workloads, console output will include the old and new workload set versions: ``` > dotnet workload install wasm-tools -Checking for updated workloads version... -Updating workloads version from 8.0.201-baseline.23919.2 to 8.0.201... +Checking for updated workload set... +Updating workload set version from 8.0.201-baseline.23919.2 to 8.0.201... Installing workload manifest microsoft.net.sdk.android version 34.0.0-preview.4.230... Installing pack Microsoft.NET.Runtime.WebAssembly.Sdk version 8.0.0-preview.4.23181.9... Garbage collecting for SDK feature band(s) 8.0.200... -Successfully updated workloads version from 8.0.201-baseline.23919.2 to 8.0.201. +Successfully updated workload set version from 8.0.201-baseline.23919.2 to 8.0.201. Successfully installed workload(s) wasm-tools. ``` -The proposed [workload history](https://github.com/dotnet/sdk/pull/30486) command will also display the workloads versions. +The proposed [workload history](https://github.com/dotnet/sdk/pull/30486) command will also display the workload set versions. The .NET SDK will periodically (once a day, by default) check for updated workload versions in order to notify users if there is an update available. Currently, commands such as `dotnet build` print out the following message: @@ -128,18 +128,18 @@ We will modify both commands to print the same message: A workload update (version 8.0.202) is available. Run `dotnet workload update` to install the latest workloads. ``` -We will add a new `--version` option to `dotnet workload update` to allow installing (or downgrading to) a specific workloads version: +We will add a new `--version` option to `dotnet workload update` to allow installing (or downgrading to) a specific workload set version: ``` > dotnet workload update --version 8.0.202 -Updating workloads version from 8.0.201 to 8.0.202... +Updating workload set version from 8.0.201 to 8.0.202... -Successfully updated workloads version from 8.0.201 to 8.0.202. +Successfully updated workload set version from 8.0.201 to 8.0.202. ``` ## Specifying workload versions with global.json -We will add support for specifying the workloads version in global.json. For example: +We will add support for specifying the workload set version in global.json. For example: ```json { @@ -149,16 +149,16 @@ We will add support for specifying the workloads version in global.json. For ex } ``` -This would force the SDK to use workloads version `8.0.201`, and would error if that version was not installed. +This would force the SDK to use workload set version `8.0.201`, and would error if that version was not installed. -We will support side-by-side workload version installations. If 8.0.202 is installed, we would support running `dotnet workload install --version 8.0.201` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workloads version. An earlier workloads version would only be used if it was specified in a global.json file. +We will support side-by-side workload version installations. If 8.0.202 is installed, we would support running `dotnet workload install --version 8.0.201` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workload set version. An earlier workload set version would only be used if it was specified in a global.json file. -NOTE: We may not implement the global.json workloads version support in the same release as the rest of the changes described in this design. +NOTE: We may not implement the global.json workload set version support in the same release as the rest of the changes described in this design. ## Development build behavior -Currently, when the .NET SDK looks for workload updates, it looks for the latest available version of each individual manifest. We will change this so that it will look for the latest "workloads version", which is a package that we create and publish. +Currently, when the .NET SDK looks for workload updates, it looks for the latest available version of each individual manifest. We will change this so that it will look for the latest "workload set version", which is a package that we create and publish. -However, for development builds of the .NET SDK, we may want to retain the old behavior of updating each manifest individually. This would allow the latest workloads to be used with a development build even if they hadn't flowed into the installer build and we hadn't created a "workloads version" package. +However, for development builds of the .NET SDK, we may want to retain the old behavior of updating each manifest individually. This would allow the latest workloads to be used with a development build even if they hadn't flowed into the installer build and we hadn't created a workload set version package. We will add a `--update-manifests-separately` parameter to the `dotnet workload update` and `install` commands, which will opt in to the old update behavior. We may also make this the default behavior for development builds, though there may not be a good way to tell whether a given build is a "development" build or a build that will be released as an official public preview. If development builds aren't signed the same way as full preview builds are, we may be able to key off of that. \ No newline at end of file From a1a63b5d7ed86182880222d26b534d54dccd6895 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Wed, 17 May 2023 13:30:20 -0400 Subject: [PATCH 05/13] Add start of implementation design --- ...lify-workload-versioning-implementation.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 accepted/2023/simplify-workload-versioning-implementation.md diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md new file mode 100644 index 000000000..0bf35057c --- /dev/null +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -0,0 +1,77 @@ +## Implementation + +Creating a workload set version should be a lightweight process, which involves creating a NuGet packages and corresponding installers to deliver the mapping from the workload set version to the workload manifest versions. The .NET SDK assets and installers, workload manifests, and workload packs should already be built and should not need to be created when building a workload set. + +### Current workload release schedule + +Releases of .NET (both the SDK and runtime) typically align with Visual Studio releases and/or patch Tuesday. A .NET release needs to be built and validated ahead of the deadline to insert into Visual Studio or to be submitted to Microsoft Update. It takes time to build and validate a release of .NET, so we allow for roughly 2 weeks in the schedule for a release to build, validate, and if necessary fix any critical issues and re-build. This means that components that ship in the .NET SDK need to be inserted into it roughly 2 weeks before the Visual Studio or Microsoft Update deadline, which is itself some time before the actual release. + +Workloads such as ios, android, and maui which are built outside of the .NET Runtime do not currently have to meet the .NET SDK insertion deadlines. This is because they are inserted into Visual Studio separately from the .NET SDK. So when Visual Studio is installed, it installs the .NET SDK as well as the workloads that were inserted into Visual Studio. + +Non Visual Studio installs of the .NET SDK do not include workloads themselves in the .NET SDK package or installer. In this case the .NET SDK only hase workload manifests. Because the final versions of workloads are not inserted into the .NET SDK, these manifests (which we call "baseline manifests") are not the final versions and do not refer to the workloads that are actually released. When installing a workload, the default behavior of the .NET SDK is to first look for the latest workload manifests and install them before installing the workload. So as long as the workload packages are released publicly (on NuGet.org) on release day, installing a workload for a non-VS install of the .NET SDK will also install the right workload versions, even though the baseline manifest versions built into that SDK are not the final release versions. + +### Release schedule updates + +Workload sets will be created in a separate repo (possibly dotnet/workloadversions). This repo will have the versions for all the workload manifests, either via Arcade dependency flow (for workloads that build as part of .NET) or via manual updates. The build will create a json file in the rollback file format from these versions, and create NuGet packages, MSIs, and (in the future) other installers to deliver this json file as a workload set. + +To insert a final workloads into Visual Studio, the versions in the workloadversions repo should be updated and a new build of that repo produced. Then the updated workload set artifacts should be inserted into Visual Studio together with the final workloads. + +We intend that building the workloadversions repo should be quick (~1 hour) so that it should not have much schedule impact on when workloads can be inserted into Visual Studio. + +### Workload set packages + +The Workload set NuGet package for file-based installs will be named `Microsoft.NET.Workloads.`. Similar to the workload manifest NuGet packages the json file will be included in a `data` subfolder in the NuGet package. The version number of the package will be the version of the workload set. + +### Workload set disk layout + +Workload version information will be stored in `dotnet\sdk-manifests\\workloadsets\`, for example `dotnet\sdk-manifests\8.0.200\workloadsets\8.0.201`. In this folder will be one or more `.json` files which will specify the versions of each manifest that should be used. These `.json` files will use the same format as rollback files currently use. For example, the following file would specify that the android manifest should be version 33.0.4 from the 8.0.200 feature band, while the mono toolchain manifest should be version 8.0.3 from the 8.0.100 band: + +```json +{ + "microsoft.net.sdk.android": "33.0.4/8.0.200", + "microsoft.net.workload.mono.toolchain": "8.0.3/8.0.100" +} +``` + +### Workload manifest layout + +The layout of the workload manifests on disk will change. Currently, they go in a path which includes the feature band but not the version number of the manifest, such as `dotnet\sdk-manifests\8.0.200\microsoft.net.sdk.android`. This will change to also include the manifest version in the path, for example `dotnet\sdk-manifests\8.0.200\microsoft.net.sdk.android\33.0.4`. This will allow multiple workload sets to be installed side-by-side, and for a global.json file to select which one should be used. + +Currently, the manifests for each feature band update in-place, and the corresponding MSIs are upgradeable. With the new design, each manifest version will not have an MSI upgrade relationship to other versions of that manifest. This should also help fix issues that occur when we try to roll back to manifests prior to those installed by Visual Studio. Now, the prior versions of the manifest can be installed side-by-side with the versions installed by Visual Studio, and the `workloadsets` files (possibly together with glebal.json) will determine which manifests are used. + +### Baseline workload sets + +The final workload set for a release will be produced in the workloadversions repo. However, the installer repo will also produce a workload set for the baseline workload manifests that are included in the standalone SDK. The version assigned to this workload set will depend on whether the SDK being built is using a "stabilized" version number. If so, then the baseline workload set version will be `-baseline.`, for example `8.0.201-baseline.23919.2`. Otherwise, the workload set version will use the same version number as the .NET SDK, for example `8.0.100-preview.4.23221.6`. The baseline workload set will be included in the layout for file-based builds of the .NET SDK, and packaged up as an MSI which is bundled in the standalone installer. + +### Workload behavior when no workload patch information is installed + +In some cases, internal builds of Visual Studio may not have workload set information. In this case, when the SDK needs to display a workload set version, one will be generated using the .NET SDK feature band, the pre-release specifier `manifests`, and a hash of the manifest IDs and versions. For example, `8.0.200-manifests.9e7f4b93`. + +### Workload resolver updates + +Because the `workloadsets` folder will be next to the other workload manifest folders, the resolver will be updated to ignore this folder when looking for workload manifests. Older versions of the resolver should not be impacted, because they will not be looking in the newer version band folder. This does mean that the resolver changes should go into Visual Studio as soon as possible so that full Framework MSBuild can work with the new manifests. + +### Workload manifest garbage collection + + +### Rolling back and saving workload versions + + +### Managing workload set GC roots + + +### Workload updates + +Currently, the .NET SDK tries to keep workloads updated. By default, it will update to newer workload manifests whenever you install a workload. It will also check once a day to see if updated workloads are available, and if they are it will print out a message suggesting you update workloads when you run basic commands such as `dotnet build`. + +With the new system, instead of checking for updates to each manifest separately, the SDK will check for a new workload patch version. When updating to that workload patch, the patch will be downloaded and installed, and then the manifest versions from the patch will be downloaded and installed. + +### Visual Studio installed SDKs + +When the .NET SDK is installed by Visual Studio, it is usually best if the workloads are also installed via Visual Studio. That way they will get updated together with updates to Visual Studio and the .NET SDK. If the .NET SDK is installed by Visual Studio, and someone installs workloads from the command line (for example via `dotnet workload install`), then the workloads will stop working when Visual Studio updates the .NET SDK. + +Because of this, we will change CLI workload commands such as `dotnet workload install`, `dotnet workload restore`, and `dotnet workload update` to check if the current .NET SDK has been installed (only) by Visual Studio. If so, those commands will generate an error recommending that the workloads be installed using the Visual Studio installer. The commands will also provide an option to override the error and allow installing workloads from the command line if needed. + +### Telemetry + +We would like to better understand whether people are using SDKs installed by Visual Studio, standalone installs, or both. We will add this information to the telemetry that we send. To support this we will update the .NET SDK MSI to conditionally include a `.vs` or `.standalone` file in the SDK folder to identify how it was installed. From 88d50cc4d088ced92df7ea713639b1f2da4931ca Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Tue, 30 May 2023 19:43:57 -0400 Subject: [PATCH 06/13] Add details about how side-by-side workload sets will be managed --- ...lify-workload-versioning-implementation.md | 53 +++++++++++++++++-- accepted/2023/simplify-workload-versioning.md | 14 ++++- 2 files changed, 61 insertions(+), 6 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index 0bf35057c..e8276ee96 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -24,7 +24,7 @@ The Workload set NuGet package for file-based installs will be named `Microsoft. ### Workload set disk layout -Workload version information will be stored in `dotnet\sdk-manifests\\workloadsets\`, for example `dotnet\sdk-manifests\8.0.200\workloadsets\8.0.201`. In this folder will be one or more `.json` files which will specify the versions of each manifest that should be used. These `.json` files will use the same format as rollback files currently use. For example, the following file would specify that the android manifest should be version 33.0.4 from the 8.0.200 feature band, while the mono toolchain manifest should be version 8.0.3 from the 8.0.100 band: +Workload version information will be stored in `dotnet\sdk-manifests\\workloadsets\`, for example `dotnet\sdk-manifests\8.0.200\workloadsets\8.0.201`. In this folder will be one or more files ending in `.workloadset.json` which will specify the versions of each manifest that should be used. These `.json` files will use the same format as rollback files currently use. For example, the following file would specify that the android manifest should be version 33.0.4 from the 8.0.200 feature band, while the mono toolchain manifest should be version 8.0.3 from the 8.0.100 band: ```json { @@ -47,17 +47,62 @@ The final workload set for a release will be produced in the workloadversions re In some cases, internal builds of Visual Studio may not have workload set information. In this case, when the SDK needs to display a workload set version, one will be generated using the .NET SDK feature band, the pre-release specifier `manifests`, and a hash of the manifest IDs and versions. For example, `8.0.200-manifests.9e7f4b93`. +### Rolling back and pinning workload versions + +The following commands can be used to select workload versions which don't correspond to the latest workload set: + +- `dotnet workload update --from-rollback` +- `dotnet workload update --from-history` +- `dotnet workload update --version` + +When these commands run, they need to "pin" the workload versions so that the resolver will load the selected workload versions. The information about what versions are pinned needs to be stored somewhere. We will call the folder where it is stored the "workload install state folder". + +- For MSI-based installs, it will be stored in `%PROGRAMDATA%\dotnet\workloads\\InstallState` +- For file-based installs, it will be stored in `dotnet\metadata\workloads\\InstallState` + +If the workload versions are "pinned", there will be a `default.json` file in this folder. The file will specify either a workload set version, or versions for each individual manifest: + +```json +{ + "workloadSet": "8.0.102" +} +``` + +```json +{ + "manifests": + { + "microsoft.net.sdk.android": "33.0.46/7.0.100", + "microsoft.net.sdk.ios": "16.4.7054/7.0.100", + // etc. + } +} +``` + +Normally the file should use a workload set version if possible, and only include manifest versions when a rollback file is used. However, if there are manifests that are installed that are not part of the workload set (for example for Tizen), then the json file should include the workload set version and then the manifest versions for manifests that are not part of the workload set. + +When workload versions are pinned, running `dotnet workload install` will not automatically do a workload update. + +If `dotnet workload update` is run, then any pinned workload versions will be removed (the json file will be deleted), as the intent of the command is to update to the latest version. + ### Workload resolver updates Because the `workloadsets` folder will be next to the other workload manifest folders, the resolver will be updated to ignore this folder when looking for workload manifests. Older versions of the resolver should not be impacted, because they will not be looking in the newer version band folder. This does mean that the resolver changes should go into Visual Studio as soon as possible so that full Framework MSBuild can work with the new manifests. -### Workload manifest garbage collection +The workload resolver should first look for a workload set version from a global.json file. If not specified via global.json, then it should look in the workload install state folder for a workload set or workload manifest versions to use. If not specified via the install state folder, then the resolver should use the latest workload set version installed for the feature band. Finally, if no workload set is installed, the resolver should use the latest version of each manifest for the feature band. + +### Installing workload set from global.json +`dotnet workload update` - If workload set is specified in global.json, this should install that workload set. Also should register this global.json path in workload install state folder as a GC root. -### Rolling back and saving workload versions +- `dotnet workload restore` - global.json aware, same as update +- `dotnet workload install` - global.json aware +- `dotnet workload clean` - should look through all records of global.json, see if they are still there and up-to-date, delete as necessary before GC +- `dotnet workload roots` - (not sure about command name) List all registered global.json files and versions of workload sets they pin +When installing a workload, it is installed only for the active workload set version. If you need it installed for other versions, you need to go to the corresponding folder with the global.json and run a workload restore or install command. The alternative would be to install the workload for all workload sets, which doesn't seem desirable. -### Managing workload set GC roots +### Workload set / workload manifest garbage collection ### Workload updates diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index 5cb2d0b7b..4d040eef0 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -151,10 +151,20 @@ We will add support for specifying the workload set version in global.json. For This would force the SDK to use workload set version `8.0.201`, and would error if that version was not installed. -We will support side-by-side workload version installations. If 8.0.202 is installed, we would support running `dotnet workload install --version 8.0.201` to install that version of the workloads. After installing the earlier version, the .NET SDK would still by default use the latest installed workload set version. An earlier workload set version would only be used if it was specified in a global.json file. - NOTE: We may not implement the global.json workload set version support in the same release as the rest of the changes described in this design. +## Side by side workload sets + +The workload `restore`, `update`, and `install` commands will be updated to check if there is a global.json file specifying a workload set version applying to the current folder. If so, these commands will install that workload set version side-by-side with any other installed workload set. They will also record the path to the global.json file and the workload set version it specified. This record will be used as a root during garbage collection to prevent the side-by-side workload set (and its packs) from being removed. + +The `dotnet workload clean` command will read these records and check to see if the global.json file referenced still exists and specifies the same workload set version. If not, that record will be removed before garbage collection, allowing the workload set, its manifests, and packs to possibly be removed. The `dotnet workload clean --all` command uninstalls all workloads, and will also delete all of these records. + +We will add a `dotnet workload roots` command (though hopefully we can come up with a better name). This will list all of the current records of global.json paths and workload set versions that the SDK has. This can help understand why workload assets may not be uninstalled / cleaned up as expected. + +## Workload list updates + +`dotnet workload list` currently displays the workload ID, the manifest version, and the installation source. We will change it to display the workload set version instead of the manifest version. Additionally, we will add a column for the "status". If all the packs for the workload are installed, this will be "OK". If any packs are missing, the status will be "Missing packs". This can happen if the workload set is updated to a new baseline workload set, or if there are side-by-side workload sets installed but a workload is not installed for all of those workload sets. + ## Development build behavior Currently, when the .NET SDK looks for workload updates, it looks for the latest available version of each individual manifest. We will change this so that it will look for the latest "workload set version", which is a package that we create and publish. From 5233e7b6d87c64d91bcd9c1bbbd661c579a3ed87 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Wed, 31 May 2023 10:46:35 -0400 Subject: [PATCH 07/13] Fix typos --- accepted/2023/simplify-workload-versioning-implementation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index e8276ee96..d59a9df8f 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -1,6 +1,6 @@ ## Implementation -Creating a workload set version should be a lightweight process, which involves creating a NuGet packages and corresponding installers to deliver the mapping from the workload set version to the workload manifest versions. The .NET SDK assets and installers, workload manifests, and workload packs should already be built and should not need to be created when building a workload set. +Creating a workload set version should be a lightweight process, which involves creating NuGet packages and corresponding installers to deliver the mapping from the workload set version to the workload manifest versions. The .NET SDK assets and installers, workload manifests, and workload packs should already be built and should not need to be created when building a workload set. ### Current workload release schedule @@ -37,7 +37,7 @@ Workload version information will be stored in `dotnet\sdk-manifests\ Date: Sun, 4 Jun 2023 09:48:46 -0400 Subject: [PATCH 08/13] WIP --- accepted/2023/simplify-workload-versioning-implementation.md | 2 ++ accepted/2023/simplify-workload-versioning.md | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index d59a9df8f..d8e2e73c9 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -85,6 +85,8 @@ When workload versions are pinned, running `dotnet workload install` will not au If `dotnet workload update` is run, then any pinned workload versions will be removed (the json file will be deleted), as the intent of the command is to update to the latest version. +TODO: Error if global.json specifies a workload set that isn't installed should recommend running `dotnet workload restore`. + ### Workload resolver updates Because the `workloadsets` folder will be next to the other workload manifest folders, the resolver will be updated to ignore this folder when looking for workload manifests. Older versions of the resolver should not be impacted, because they will not be looking in the newer version band folder. This does mean that the resolver changes should go into Visual Studio as soon as possible so that full Framework MSBuild can work with the new manifests. diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index 4d040eef0..b208d5fdb 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -159,8 +159,12 @@ The workload `restore`, `update`, and `install` commands will be updated to chec The `dotnet workload clean` command will read these records and check to see if the global.json file referenced still exists and specifies the same workload set version. If not, that record will be removed before garbage collection, allowing the workload set, its manifests, and packs to possibly be removed. The `dotnet workload clean --all` command uninstalls all workloads, and will also delete all of these records. +Question: What should clean do if a path is not accessible (ie on a thumb drive that was removed)? + We will add a `dotnet workload roots` command (though hopefully we can come up with a better name). This will list all of the current records of global.json paths and workload set versions that the SDK has. This can help understand why workload assets may not be uninstalled / cleaned up as expected. +Question: Should we support `dotnet workload install --version`? This could help in the scenario where you switch back and forth between branches that pin different workload set versions, but don't want the churn of uninstalling and reinstalling workloads each time. + ## Workload list updates `dotnet workload list` currently displays the workload ID, the manifest version, and the installation source. We will change it to display the workload set version instead of the manifest version. Additionally, we will add a column for the "status". If all the packs for the workload are installed, this will be "OK". If any packs are missing, the status will be "Missing packs". This can happen if the workload set is updated to a new baseline workload set, or if there are side-by-side workload sets installed but a workload is not installed for all of those workload sets. From 1288eaf3fddccf5662d55690ba7a7e56271f634b Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Mon, 10 Jul 2023 13:47:42 -0400 Subject: [PATCH 09/13] Update global.json property name --- accepted/2023/simplify-workload-versioning-implementation.md | 2 +- accepted/2023/simplify-workload-versioning.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index d8e2e73c9..408fc54cc 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -64,7 +64,7 @@ If the workload versions are "pinned", there will be a `default.json` file in th ```json { - "workloadSet": "8.0.102" + "workloadVersion": "8.0.102" } ``` diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index b208d5fdb..8452a0d9a 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -144,7 +144,7 @@ We will add support for specifying the workload set version in global.json. For ```json { "sdk": { - "workloadsVersion": "8.0.201" + "workloadVersion": "8.0.201" } } ``` From bf382a0f52e25e7b69851e9c0fe5e23024b53a12 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Tue, 11 Jul 2023 12:10:46 -0400 Subject: [PATCH 10/13] Update wording of command output --- accepted/2023/simplify-workload-versioning.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index 8452a0d9a..b68060c1a 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -79,12 +79,12 @@ We will also update `dotnet --info` and `dotnet workload --info` to display the .NET SDK: Version: 8.0.201 Commit: - Workload set version: 8.0.201 + Workload version: 8.0.201 > dotnet workload --info - Workload set version: 8.0.201 + Workload version: 8.0.201 [wasm-tools] Installation Source: SDK 8.0.100-preview.4 Manifest Version: 8.0.0-preview.4.23181.9/8.0.100-preview.4 From e3dd128446ad1b5dc3abd05f92ecd8490905f783 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Mon, 8 Jan 2024 13:35:45 -0500 Subject: [PATCH 11/13] Add new proposal for workload set versions, and some garbage collection details --- ...implify-workload-versioning-implementation.md | 5 +++++ accepted/2023/simplify-workload-versioning.md | 16 ++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index 408fc54cc..66ec9d917 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -106,6 +106,10 @@ When installing a workload, it is installed only for the active workload set ver ### Workload set / workload manifest garbage collection +- Garbage collection occurs within the scope of a feature band, reference counts are used across feature bands. +- Baseline workload sets should not be garbage collected. They will be identified by convention as they should have a `baseline.workloadset.json` file +- Manifests installed with the SDK need to have feature band reference counts to prevent other feature bands from garbage collecting them. This should happen normally for MSI-based installs, but file-based installs will need to ship with the appropriate marker files with the path `metadata/workloads/InstalledManifests/v1////`. + ### Workload updates @@ -122,3 +126,4 @@ Because of this, we will change CLI workload commands such as `dotnet workload i ### Telemetry We would like to better understand whether people are using SDKs installed by Visual Studio, standalone installs, or both. We will add this information to the telemetry that we send. To support this we will update the .NET SDK MSI to conditionally include a `.vs` or `.standalone` file in the SDK folder to identify how it was installed. + diff --git a/accepted/2023/simplify-workload-versioning.md b/accepted/2023/simplify-workload-versioning.md index b68060c1a..ed1dc1193 100644 --- a/accepted/2023/simplify-workload-versioning.md +++ b/accepted/2023/simplify-workload-versioning.md @@ -34,6 +34,22 @@ Currently, we use a 3 part version number for the .NET SDK, for example 8.0.100. We will create the concept of a "workload set" which has a version number that encapsulates all the workload manifest versions that are released together. A workload set is essentially a mapping from the workload set version to the different versions for each workload manifest. For public releases, the workload set version will use the same version number as the .NET SDK. For example, when .NET SDK 8.0.101 releases, there will be a corresponding workload set version 8.0.101 that corresponds to the versions of the workloads that were released together with that .NET SDK. +## Interim workload set versions + +For workload sets released between releases of the .NET SDK, we will use 4-part version numbers. For example, workload set 8.0.203 would be released together with .NET SDK 8.0.203. If we want to release another workload set before the 8.0.204 SDK release, then we can designate that workload set as 8.0.203.1 (and 8.0.203.2 if there's another interim workload set, etc.). This is a departure from Semantic Versioning, but appears to be the best way to have an understandable version number for interim workload set releases. + +Note that the (possibly 4-part) workload set version will be what is displayed by the .NET CLI and can be specified in places such as the command line and global.json files, but it won't be used as a literal package verison for any NuGet packages. NuGet package versions will remain semantically versions. For the NuGet packages that represent the workload set, there will be a mapping from the workload set version to both the package ID and version, where the feature band is part of the package ID, freeing up space in the version for the additional interim release number. + +Some examples of possible workload set versions: + +- `8.0.203` - Regular monthly release +- `8.0.203.1` - Interim workload set release +- `8.0.204-preview.1` - Preview interim release of changes that are expected to be incuded in 8.0.204 +- `8.0.203.1-preview.1` - Preview interim release of changes that are expected to be released in a workload set before 8.0.204 +- `9.0.100-preview.2` - Workload set for .NET SDK preview release +- `8.0.201-servicing.23015` - Daily build of dotnet/workload-versions repo +- `8.0.201-ci.maui.23015` - Workload set corresponding to daily build of maui repo + ## Baseline workload versions .NET SDK workloads have similar shipping deadlines as the .NET SDK. That means that the .NET SDK itself can't include final versions of workloads which are built outside of the .NET build, as building and signing off on those workloads happens in parallel with .NET build and signoff. So the .NET SDK ships with *baseline* workload manifests, which are mainly used to enable the .NET SDK to list which workloads are available, and to know if a project requires a workload which isn't installed. By default, when a workload is installed via the .NET CLI, the .NET SDK will first look for and update to the latest workload manifests for the current feature band before installing workloads. From 16c553b7905e60a83c0c87acaeb3fd192221dca9 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 15 Feb 2024 16:28:14 -0500 Subject: [PATCH 12/13] Add details about workload set package version mapping --- ...lify-workload-versioning-implementation.md | 19 +++++++++++++++++++ accepted/2023/simplify-workload-versioning.md | 2 -- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index 66ec9d917..c158e8889 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -22,6 +22,23 @@ We intend that building the workloadversions repo should be quick (~1 hour) so t The Workload set NuGet package for file-based installs will be named `Microsoft.NET.Workloads.`. Similar to the workload manifest NuGet packages the json file will be included in a `data` subfolder in the NuGet package. The version number of the package will be the version of the workload set. +### Workload set package versions + +The (possibly 4-part) workload set version will be what is displayed by the .NET CLI and can be specified in places such as the command line and global.json files, but it won't be used as a literal package verison for any NuGet packages. NuGet package versions will remain semantic versions. For the NuGet packages that represent the workload set, there will be a mapping from the workload set version to both the package ID and version, where the feature band is part of the package ID, freeing up space in the version for the additional interim release number. + +Since the minor version of the SDK version (and hence workload set version) is expected to remain 0, for the workload set package versions we will omit that part of the version and shift the other parts over. For example, for workload set 8.0.203, the workload set package version would be 8.203.0. For workload set 8.0.203.1, the workload set package version would be 8.203.1. Because the SDK feature band is part of the workload set package ID, this mapping will still work even if we do end up having a minor release of the SDK in the future. + +| SDK Version | Workload set version | Workload set Package ID | Workload set package version | +|-------------|----------------------|-------------------------|------------------------------| +| 8.0.200 | 8.0.200 | Microsoft.Net.Workloads.8.0.200 | 8.200.0 | +| 8.0.201 | 8.0.201 | Microsoft.Net.Workloads.8.0.200 | 8.201.0 | +| 8.0.203 | 8.0.203.1 | Microsoft.Net.Workloads.8.0.200 | 8.203.1 | +| 9.0.100-preview.2 | 9.0.100-preview.2 | Microsoft.Net.Workloads.9.0.100-preview.2 | 9.100.0 | +| 8.0.201 | 8.0.201.1-preview | Microsoft.Net.Workloads.8.0.200 | 8.201.1-preview | +| 8.0.201 | 8.0.201.1-preview.2 | Microsoft.Net.Workloads.8.0.200 | 8.201.1-preview.2 | +| 8.0.201-servicing.23015 | 8.0.201-servicing.23015 | Microsoft.Net.Workloads.8.0.200 | 8.201.0-servicing.23015 | + + ### Workload set disk layout Workload version information will be stored in `dotnet\sdk-manifests\\workloadsets\`, for example `dotnet\sdk-manifests\8.0.200\workloadsets\8.0.201`. In this folder will be one or more files ending in `.workloadset.json` which will specify the versions of each manifest that should be used. These `.json` files will use the same format as rollback files currently use. For example, the following file would specify that the android manifest should be version 33.0.4 from the 8.0.200 feature band, while the mono toolchain manifest should be version 8.0.3 from the 8.0.100 band: @@ -33,6 +50,8 @@ Workload version information will be stored in `dotnet\sdk-manifests\ Date: Fri, 1 Mar 2024 13:48:39 -0500 Subject: [PATCH 13/13] Update workload set package version mapping --- accepted/2023/simplify-workload-versioning-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2023/simplify-workload-versioning-implementation.md b/accepted/2023/simplify-workload-versioning-implementation.md index c158e8889..558689761 100644 --- a/accepted/2023/simplify-workload-versioning-implementation.md +++ b/accepted/2023/simplify-workload-versioning-implementation.md @@ -33,7 +33,7 @@ Since the minor version of the SDK version (and hence workload set version) is e | 8.0.200 | 8.0.200 | Microsoft.Net.Workloads.8.0.200 | 8.200.0 | | 8.0.201 | 8.0.201 | Microsoft.Net.Workloads.8.0.200 | 8.201.0 | | 8.0.203 | 8.0.203.1 | Microsoft.Net.Workloads.8.0.200 | 8.203.1 | -| 9.0.100-preview.2 | 9.0.100-preview.2 | Microsoft.Net.Workloads.9.0.100-preview.2 | 9.100.0 | +| 9.0.100-preview.2 | 9.0.100-preview.2.39041 | Microsoft.Net.Workloads.9.0.100-preview.2 | 9.100.0-preview.2.39041 | | 8.0.201 | 8.0.201.1-preview | Microsoft.Net.Workloads.8.0.200 | 8.201.1-preview | | 8.0.201 | 8.0.201.1-preview.2 | Microsoft.Net.Workloads.8.0.200 | 8.201.1-preview.2 | | 8.0.201-servicing.23015 | 8.0.201-servicing.23015 | Microsoft.Net.Workloads.8.0.200 | 8.201.0-servicing.23015 |