From 08a6d3a2890bb6c02daf451de82f89849623ac24 Mon Sep 17 00:00:00 2001 From: Violet Hynes Date: Tue, 12 Nov 2024 09:15:52 -0500 Subject: [PATCH] Product usage reporting docs (#28858) * First draft of product usage reporting docs * Table data, fix issues * Changelog --- changelog/28858.txt | 3 + website/content/docs/configuration/index.mdx | 51 +++++-- .../content/docs/configuration/reporting.mdx | 42 ++++++ .../license/product-usage-reporting.mdx | 131 ++++++++++++++++++ website/data/docs-nav-data.json | 18 +++ 5 files changed, 231 insertions(+), 14 deletions(-) create mode 100644 changelog/28858.txt create mode 100644 website/content/docs/configuration/reporting.mdx create mode 100644 website/content/docs/enterprise/license/product-usage-reporting.mdx diff --git a/changelog/28858.txt b/changelog/28858.txt new file mode 100644 index 000000000000..422b9d650609 --- /dev/null +++ b/changelog/28858.txt @@ -0,0 +1,3 @@ +```release-note:feature +**Product Usage Reporting**: Added product usage reporting, which collects anonymous, numerical, non-sensitive data about Vault feature usage, and adds it to the existing utilization reports. +``` \ No newline at end of file diff --git a/website/content/docs/configuration/index.mdx b/website/content/docs/configuration/index.mdx index 0f8d06b6c407..e4d65457e378 100644 --- a/website/content/docs/configuration/index.mdx +++ b/website/content/docs/configuration/index.mdx @@ -13,6 +13,14 @@ The format of this file is [HCL](https://github.com/hashicorp/hcl) or JSON. An example configuration is shown below: + + +For multi-node clusters, replace the loopback address with a valid, routable IP address for each Vault node in your network. + +Refer to the [Vault HA clustering with integrated storage tutorial](/vault/tutorials/raft/raft-storage) for a complete scenario. + + + ```hcl ui = true cluster_addr = "https://127.0.0.1:8201" @@ -66,15 +74,19 @@ to specify where the configuration is. - `user_lockout` `([UserLockout][user-lockout]: nil)` – Configures the user-lockout behaviour for failed logins. For more information, please see the - [user lockout configuration documentation](/vault/docs/configuration/user-lockout). + [user lockout configuration documentation](/vault/docs/configuration/user-lockout). - `seal` `([Seal][seal]: nil)` – Configures the seal type to use for auto-unsealing, as well as for [seal wrapping][sealwrap] as an additional layer of data protection. -- `cluster_name` `(string: )` – Specifies the identifier for the - Vault cluster. If omitted, Vault will generate a value. When connecting to - Vault Enterprise, this value will be used in the interface. +- `reporting` `([Reporting][reporting]: nil)` - + Configures options relating to license reporting in Vault. + +- `cluster_name` `(string: )` – Specifies a human-readable + identifier for the Vault cluster. If omitted, Vault will generate a value. + The cluster name is included as a label in some [telemetry metrics](/vault/docs/internals/telemetry/metrics/). + The cluster name is safe to update on an existing Vault cluster. - `cache_size` `(string: "131072")` – Specifies the size of the read cache used by the physical storage subsystem. The value is in number of entries, so the @@ -115,10 +127,14 @@ to specify where the configuration is. sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault)) ``` - ~> Note: Since each plugin runs as a separate process, you need to do the same + + + Since each plugin runs as a separate process, you need to do the same for each plugin in your [plugins directory](/vault/docs/plugins/plugin-architecture#plugin-directory). + + If you use a Linux distribution with a modern version of systemd, you can add the following directive to the "[Service]" configuration section: @@ -169,10 +185,10 @@ to specify where the configuration is. maximum request duration allowed before Vault cancels the request. This can be overridden per listener via the `max_request_duration` value. -- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal -mutex locks that should be monitored for potential deadlocks. Currently supported values +- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal +mutex locks that should be monitored for potential deadlocks. Currently supported values include `statelock`, `quotas` and `expiration` which will cause "POTENTIAL DEADLOCK:" -to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this +to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this can have a negative effect on performance due to the tracking of each lock attempt. - `raw_storage_endpoint` `(bool: false)` – Enables the `sys/raw` endpoint which @@ -210,12 +226,20 @@ can have a negative effect on performance due to the tracking of each lock attem Supported values (in order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. This can also be specified via the `VAULT_LOG_LEVEL` environment variable. - ~> Note: On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level, + + + On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level, overriding (even if specified) both the CLI flag and environment variable. - ~> Note: Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular, + + + + + Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular, secrets/auth plugins are currently not updated dynamically. + + - `log_format` - Equivalent to the [`-log-format` command-line flag](/vault/docs/commands/server#_log_format). - `log_file` - Equivalent to the [`-log-file` command-line flag](/vault/docs/commands/server#_log_file). @@ -236,9 +260,6 @@ can have a negative effect on performance due to the tracking of each lock attem When `imprecise_lease_role_tracking` is set to true and a new role-based quota is enabled, subsequent lease counts start from 0. `imprecise_lease_role_tracking` affects role-based lease count quotas, but reduces latencies when not using role based quotas. -- `request_limiter` `([Request Limiter][request-limiter]: )` – Allows - operators to enable Vault's Request Limiter functionality. - ### High availability parameters The following parameters are used on backends that support [high availability][high-availability]. @@ -283,6 +304,9 @@ The following parameters are only used with Vault Enterprise provided via the environment variable `VAULT_LICENSE_PATH`, or the license itself can be provided in the environment variable `VAULT_LICENSE`. +- `administrative_namespace_path` `(string: "")` - Specifies the absolute path + to the Vault namespace to be used as an [Administrative namespace](/vault/docs/enterprise/namespaces/create-admin-namespace). + [storage-backend]: /vault/docs/configuration/storage [listener]: /vault/docs/configuration/listener [seal]: /vault/docs/configuration/seal @@ -291,4 +315,3 @@ The following parameters are only used with Vault Enterprise [sentinel]: /vault/docs/configuration/sentinel [high-availability]: /vault/docs/concepts/ha [plugins]: /vault/docs/plugins -[request-limiter]: /vault/docs/concepts/request-limiter diff --git a/website/content/docs/configuration/reporting.mdx b/website/content/docs/configuration/reporting.mdx new file mode 100644 index 000000000000..4978d6dd3654 --- /dev/null +++ b/website/content/docs/configuration/reporting.mdx @@ -0,0 +1,42 @@ +--- +layout: docs +page_title: Reporting - Configuration +description: |- + The reporting stanza specifies various parameters for tuning reporting and licensing related values. +--- + +# `replication` stanza + +@include 'alerts/enterprise-only.mdx' + +The `reporting` stanza specifies various parameters for tuning replication related values. + +Please see the pages relating to +[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +and [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting) +for more information regarding the license reporting. + +```hcl +reporting { + snapshot_retention_time = 9600 + disable_product_usage_reporting = false + license { + enabled = true + } +} +``` + +## `reporting` parameters + +- `snapshot_retention_time` `(duration: 9600h)` - The retention time for manual reporting snapshots in hours. Defaults to 9600 (400 days). +Uses [duration format strings](/vault/docs/concepts/duration-format). +- `disable_product_usage_reporting` `(boolean: false)` - Determines whether [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting) +is enabled. + +## `license` parameters + +- `enabled` `(boolean: true)` - Toggles automatic reporting of license utilization. See the +[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +page for more details. +- `billing_start_timestamp` `(timestamp)` - The start timestamp for billing for license reporting (manual and automated). +Defaults to the license start timestamp. diff --git a/website/content/docs/enterprise/license/product-usage-reporting.mdx b/website/content/docs/enterprise/license/product-usage-reporting.mdx new file mode 100644 index 000000000000..860d2bc28c35 --- /dev/null +++ b/website/content/docs/enterprise/license/product-usage-reporting.mdx @@ -0,0 +1,131 @@ +--- +layout: docs +page_title: Product usage reporting +description: >- + Learn what anonymous usage data HashiCorp collects as part of Enterprise utilization reporting. Enable or disable collection. +--- + +# Product usage reporting + +@include 'alerts/enterprise-only.mdx' + +HashiCorp collects usage data about how Vault clusters are being used. This data is not +used for billing or and is numerical only, and no sensitive information of +any nature is being collected. The data is GDPR compliant and is collected as part of +the [license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +process. If automated reporting is enabled, this data will be collected automatically. +If automated reporting is disabled, then this will be collected as part of the manual reports. + +## Opt out + +While none of the collected usage metrics are sensitive in any way, if you are still concerned +about these usage metrics being reported, then you can opt-out of them being collected. + +If you are considering opting out because you’re worried about the data, we +strongly recommend that you review the [usage metrics list](#usage-metrics-list) +before opting out. If you have concerns with any of the automatically-reported +data please bring them to your account manager. + +You have two options to opt out of product usage collection: + +- HCL configuration (recommended) +- Environment variable (requires restart) + + +#### HCL configuration + +Opting out in your product's configuration file doesn't require a system +restart, and is the method we recommend. Add the following block to your server +configuration file (e.g. `vault-config.hcl`). + +```hcl +reporting { + disable_product_usage_reporting = true +} +``` + + + + When you have a cluster, each node must have the reporting stanza in its + configuration to be consistent. In the event of leadership change, nodes will + use its server configuration to determine whether or not to opt-out the + product usage collection. Inconsistent configuration between nodes will change the + reporting status upon active unseal. + + + + +You will find the following entries in the server log. + + + + ``` + [DEBUG] activity: there is no reporting agent configured, skipping counts reporting + ``` + + + +#### Environment variable + +If you need to, you can also opt out using an environment variable, which will +provide a startup message confirming that you have product usage data collection. +This option requires a system restart. + + + + If the reporting stanza exists in the configuration file, the + `OPTOUT_PRODUCT_USAGE_REPORTING` value overrides the configuration. + + + +Set the following environment variable. + +```shell-session +$ export OPTOUT_PRODUCT_USAGE_REPORTING=true +``` + +Now, restart your [Vault servers](/vault/docs/commands/server) from the shell +where you set the environment variable. + +You will find the following entries in the server log. + + + + ``` + [DEBUG] core: product usage reporting disabled + ``` + + + +If your configuration file and environment variable differ, the environment +variable setting will take precedence. + +## Usage metrics list + +HashiCorp collects the following product usage metrics as part of the `metrics` part of the +[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads). +All of these metrics are numerical, and contain no sensitive values or additional metadata: + +| Metric Name | Description | +|--------------------------------------------|--------------------------------------------------------------------------| +| `vault.namespaces.count` | Total number of namespaces. | +| `vault.leases.count` | Total number of leases within Vault. | +| `vault.quotas.ratelimit.count` | Total number of rate limit quotas within Vault. | +| `vault.quotas.leasecount.count` | Total number of lease count quotas within Vault. | +| `vault.kv.version1.secrets.count` | Total number of KVv1 secrets within Vault. | +| `vault.kv.version2.secrets.count` | Total number of KVv2 secrets within Vault. | +| `vault.kv.version1.secrets.namespace.max` | The highest number of KVv1 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version2.secrets.namespace.max` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version1.secrets.namespace.min` | The lowest number of KVv1 secrets in a namespace in Vault, e.g. `2`. | +| `vault.kv.version2.secrets.namespace.min` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv1 secrets in namespaces in Vault, e.g. `52.8`. | +| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv2 secrets in namespaces in Vault, e.g. `52.8`. | + +## Usage metadata list + +HashiCorp collects the following product usage metadata as part of the `metadata` part of the +[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads): + +| Metadata Name | Description | +|----------------------|----------------------------------------------------------------------| +| `replication_status` | Replication status of this cluster, e.g. `perf-disabled,dr-disabled` | diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index a5d713ba3a2c..65e1d7e4bcf6 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -387,6 +387,15 @@ }, "path": "configuration/replication" }, + { + "title": "reporting", + "badge": { + "text": "ENTERPRISE", + "type": "outlined", + "color": "neutral" + }, + "path": "configuration/reporting" + }, { "title": "seal", "routes": [ @@ -440,6 +449,11 @@ }, { "title": "sentinel", + "badge": { + "text": "ENTERPRISE", + "type": "outlined", + "color": "neutral" + }, "path": "configuration/sentinel" }, { @@ -2700,6 +2714,10 @@ "title": "Manual license utilization reporting", "path": "enterprise/license/manual-reporting" }, + { + "title": "Product usage reporting", + "path": "enterprise/license/product-usage-reporting" + }, { "title": "FAQ", "path": "enterprise/license/faq"