Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.17 Manual Backport: Release notes and docs for product usage data #28917

Open
wants to merge 2 commits into
base: release/1.17.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions changelog/28858.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:feature
**Product Usage Reporting**: Added product usage reporting, which collects anonymous, numerical, non-sensitive data about Vault feature usage, and adds it to the existing utilization reports.
```
4 changes: 4 additions & 0 deletions website/content/docs/configuration/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@ to specify where the configuration is.
auto-unsealing, as well as for
[seal wrapping][sealwrap] as an additional layer of data protection.

- `reporting` `([Reporting][reporting]: nil)` -
Configures options relating to license reporting in Vault.

- `cluster_name` `(string: <generated>)` – Specifies a human-readable
identifier for the Vault cluster. If omitted, Vault will generate a value.
The cluster name is included as a label in some [telemetry metrics](/vault/docs/internals/telemetry/metrics/).
Expand Down Expand Up @@ -309,6 +312,7 @@ The following parameters are only used with Vault Enterprise

[storage-backend]: /vault/docs/configuration/storage
[listener]: /vault/docs/configuration/listener
[reporting]: /vault/docs/configuration/reporting
[seal]: /vault/docs/configuration/seal
[sealwrap]: /vault/docs/enterprise/sealwrap
[telemetry]: /vault/docs/configuration/telemetry
Expand Down
42 changes: 42 additions & 0 deletions website/content/docs/configuration/reporting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
layout: docs
page_title: Reporting - Configuration
description: |-
The reporting stanza specifies various parameters for tuning reporting and licensing related values.
---

# `replication` stanza

@include 'alerts/enterprise-only.mdx'

The `reporting` stanza specifies various parameters for tuning replication related values.

Please see the pages relating to
[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
and [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting)
for more information regarding the license reporting.

```hcl
reporting {
snapshot_retention_time = 9600
disable_product_usage_reporting = false
license {
enabled = true
}
}
```

## `reporting` parameters

- `snapshot_retention_time` `(duration: 9600h)` - The retention time for manual reporting snapshots in hours. Defaults to 9600 (400 days).
Uses [duration format strings](/vault/docs/concepts/duration-format).
- `disable_product_usage_reporting` `(boolean: false)` - Determines whether [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting)
is enabled.

## `license` parameters

- `enabled` `(boolean: true)` - Toggles automatic reporting of license utilization. See the
[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
page for more details.
- `billing_start_timestamp` `(timestamp)` - The start timestamp for billing for license reporting (manual and automated).
Defaults to the license start timestamp.
131 changes: 131 additions & 0 deletions website/content/docs/enterprise/license/product-usage-reporting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
layout: docs
page_title: Product usage reporting
description: >-
Learn what anonymous usage data HashiCorp collects as part of Enterprise utilization reporting. Enable or disable collection.
---

# Product usage reporting

@include 'alerts/enterprise-only.mdx'

HashiCorp collects usage data about how Vault clusters are being used. This data is not
used for billing or and is numerical only, and no sensitive information of
any nature is being collected. The data is GDPR compliant and is collected as part of
the [license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
process. If automated reporting is enabled, this data will be collected automatically.
If automated reporting is disabled, then this will be collected as part of the manual reports.

## Opt out

While none of the collected usage metrics are sensitive in any way, if you are still concerned
about these usage metrics being reported, then you can opt-out of them being collected.

If you are considering opting out because you’re worried about the data, we
strongly recommend that you review the [usage metrics list](#usage-metrics-list)
before opting out. If you have concerns with any of the automatically-reported
data please bring them to your account manager.

You have two options to opt out of product usage collection:

- HCL configuration (recommended)
- Environment variable (requires restart)


#### HCL configuration

Opting out in your product's configuration file doesn't require a system
restart, and is the method we recommend. Add the following block to your server
configuration file (e.g. `vault-config.hcl`).

```hcl
reporting {
disable_product_usage_reporting = true
}
```

<Warning>

When you have a cluster, each node must have the reporting stanza in its
configuration to be consistent. In the event of leadership change, nodes will
use its server configuration to determine whether or not to opt-out the
product usage collection. Inconsistent configuration between nodes will change the
reporting status upon active unseal.

</Warning>


You will find the following entries in the server log.

<CodeBlockConfig hideClipboard>

```
[DEBUG] activity: there is no reporting agent configured, skipping counts reporting
```

</CodeBlockConfig>

#### Environment variable

If you need to, you can also opt out using an environment variable, which will
provide a startup message confirming that you have product usage data collection.
This option requires a system restart.

<Note>

If the reporting stanza exists in the configuration file, the
`OPTOUT_PRODUCT_USAGE_REPORTING` value overrides the configuration.

</Note>

Set the following environment variable.

```shell-session
$ export OPTOUT_PRODUCT_USAGE_REPORTING=true
```

Now, restart your [Vault servers](/vault/docs/commands/server) from the shell
where you set the environment variable.

You will find the following entries in the server log.

<CodeBlockConfig hideClipboard>

```
[DEBUG] core: product usage reporting disabled
```

</CodeBlockConfig>

If your configuration file and environment variable differ, the environment
variable setting will take precedence.

## Usage metrics list

HashiCorp collects the following product usage metrics as part of the `metrics` part of the
[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads).
All of these metrics are numerical, and contain no sensitive values or additional metadata:

| Metric Name | Description |
|--------------------------------------------|--------------------------------------------------------------------------|
| `vault.namespaces.count` | Total number of namespaces. |
| `vault.leases.count` | Total number of leases within Vault. |
| `vault.quotas.ratelimit.count` | Total number of rate limit quotas within Vault. |
| `vault.quotas.leasecount.count` | Total number of lease count quotas within Vault. |
| `vault.kv.version1.secrets.count` | Total number of KVv1 secrets within Vault. |
| `vault.kv.version2.secrets.count` | Total number of KVv2 secrets within Vault. |
| `vault.kv.version1.secrets.namespace.max` | The highest number of KVv1 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version2.secrets.namespace.max` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version1.secrets.namespace.min` | The lowest number of KVv1 secrets in a namespace in Vault, e.g. `2`. |
| `vault.kv.version2.secrets.namespace.min` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv1 secrets in namespaces in Vault, e.g. `52.8`. |
| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv2 secrets in namespaces in Vault, e.g. `52.8`. |

## Usage metadata list

HashiCorp collects the following product usage metadata as part of the `metadata` part of the
[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads):

| Metadata Name | Description |
|----------------------|----------------------------------------------------------------------|
| `replication_status` | Replication status of this cluster, e.g. `perf-disabled,dr-disabled` |
33 changes: 18 additions & 15 deletions website/content/docs/release-notes/1.16.1.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,24 @@ description: |-

## Important changes

| Version | Change |
|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1.16.0+ | [Existing clusters do not show the current Vault version in UI by default](/vault/docs/upgrading/upgrade-to-1.16.x#default-policy-changes) |
| 1.16.0+ | [Default LCQ enabled when upgrading pre-1.9](/vault/docs/upgrading/upgrade-to-1.16.x#default-lcq-pre-1.9-upgrade) |
| 1.16.0+ | [External plugin environment variables take precedence over server variables](/vault/docs/upgrading/upgrade-to-1.16.x#external-plugin-variables) |
| 1.16.0+ | [LDAP auth entity alias names no longer include upndomain](/vault/docs/upgrading/upgrade-to-1.16.x#ldap-auth-entity-alias-names-no-longer-include-upndomain) |
| 1.16.0+ | [Secrets Sync now requires a one-time flag to operate](/vault/docs/upgrading/upgrade-to-1.16.x#secrets-sync-now-requires-setting-a-one-time-flag-before-use) |
| 1.16.0+ | [Azure secrets engine role creation failing](/vault/docs/upgrading/upgrade-to-1.16.x#azure-secrets-engine-role-creation-failing) |
| 1.16.1 - 1.16.3 | [New nodes added by autopilot upgrades provisioned with the wrong version](/vault/docs/upgrading/upgrade-to-1.15.x#new-nodes-added-by-autopilot-upgrades-provisioned-with-the-wrong-version) |
| 1.15.8+ | [Autopilot upgrade for Vault Enterprise fails](/vault/docs/upgrading/upgrade-to-1.15.x#autopilot) |
| 1.16.5 | [Listener stops listening on untrusted upstream connection with particular config settings](/vault/docs/upgrading/upgrade-to-1.16.x#listener-proxy-protocol-config) |
| 1.16.3 - 1.16.6 | [Vault standby nodes not deleting removed entity-aliases from in-memory database](/vault/docs/upgrading/upgrade-to-1.16.x#dangling-entity-alias-in-memory) |
| 0.7.0+ | [Duplicate identity groups created](/vault/docs/upgrading/upgrade-to-1.16.x#duplicate-identity-groups-created-when-concurrent-requests-sent-to-the-primary-and-pr-secondary-cluster) | |
| Known Issue (0.7.0+) | [Manual entity merges fail](/vault/docs/upgrading/upgrade-to-1.16.x#manual-entity-merges-sent-to-a-pr-secondary-cluster-are-not-persisted-to-storage)
| Known Issue (1.16.7-1.16.8) | [Some values in the audit logs not hmac'd properly](/vault/docs/upgrading/upgrade-to-1.16.x#client-tokens-and-token-accessors-audited-in-plaintext) |
| Version | Change |
|-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1.16.0+ | [Existing clusters do not show the current Vault version in UI by default](/vault/docs/upgrading/upgrade-to-1.16.x#default-policy-changes) |
| 1.16.0+ | [Default LCQ enabled when upgrading pre-1.9](/vault/docs/upgrading/upgrade-to-1.16.x#default-lcq-pre-1.9-upgrade) |
| 1.16.0+ | [External plugin environment variables take precedence over server variables](/vault/docs/upgrading/upgrade-to-1.16.x#external-plugin-variables) |
| 1.16.0+ | [LDAP auth entity alias names no longer include upndomain](/vault/docs/upgrading/upgrade-to-1.16.x#ldap-auth-entity-alias-names-no-longer-include-upndomain) |
| 1.16.0+ | [Secrets Sync now requires a one-time flag to operate](/vault/docs/upgrading/upgrade-to-1.16.x#secrets-sync-now-requires-setting-a-one-time-flag-before-use) |
| 1.16.0+ | [Azure secrets engine role creation failing](/vault/docs/upgrading/upgrade-to-1.16.x#azure-secrets-engine-role-creation-failing) |
| 1.16.1 - 1.16.3 | [New nodes added by autopilot upgrades provisioned with the wrong version](/vault/docs/upgrading/upgrade-to-1.15.x#new-nodes-added-by-autopilot-upgrades-provisioned-with-the-wrong-version) |
| 1.15.8+ | [Autopilot upgrade for Vault Enterprise fails](/vault/docs/upgrading/upgrade-to-1.15.x#autopilot) |
| 1.16.5 | [Listener stops listening on untrusted upstream connection with particular config settings](/vault/docs/upgrading/upgrade-to-1.16.x#listener-proxy-protocol-config) |
| 1.16.3 - 1.16.6 | [Vault standby nodes not deleting removed entity-aliases from in-memory database](/vault/docs/upgrading/upgrade-to-1.16.x#dangling-entity-alias-in-memory) |
| 0.7.0+ | [Duplicate identity groups created](/vault/docs/upgrading/upgrade-to-1.16.x#duplicate-identity-groups-created-when-concurrent-requests-sent-to-the-primary-and-pr-secondary-cluster) | |
| Known Issue (0.7.0+) | [Manual entity merges fail](/vault/docs/upgrading/upgrade-to-1.16.x#manual-entity-merges-sent-to-a-pr-secondary-cluster-are-not-persisted-to-storage) |
| Known Issue (1.16.7-1.16.8) | [Some values in the audit logs not hmac'd properly](/vault/docs/upgrading/upgrade-to-1.16.x#client-tokens-and-token-accessors-audited-in-plaintext) |
| New default (1.16.13) | [Vault product usage metrics reporting](/vault/docs/upgrading/upgrade-to-1.6.x#product-usage-reporting) |
| Deprecation (1.16.13) | [`default_report_months` is deprecated for the `sys/internal/counters` API](/vault/docs/upgrading/upgrade-to-1.16.x#activity-log-changes) |


## Vault companion updates

Expand Down
Loading
Loading