Skip to content

Commit

Permalink
Product usage reporting docs (#28858)
Browse files Browse the repository at this point in the history
* First draft of product usage reporting docs

* Table data, fix issues

* Changelog
  • Loading branch information
VioletHynes committed Nov 14, 2024
1 parent f5e0fbc commit 08a6d3a
Show file tree
Hide file tree
Showing 5 changed files with 231 additions and 14 deletions.
3 changes: 3 additions & 0 deletions changelog/28858.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:feature
**Product Usage Reporting**: Added product usage reporting, which collects anonymous, numerical, non-sensitive data about Vault feature usage, and adds it to the existing utilization reports.
```
51 changes: 37 additions & 14 deletions website/content/docs/configuration/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@ The format of this file is [HCL](https://github.com/hashicorp/hcl) or JSON.

An example configuration is shown below:

<Note>

For multi-node clusters, replace the loopback address with a valid, routable IP address for each Vault node in your network.

Refer to the [Vault HA clustering with integrated storage tutorial](/vault/tutorials/raft/raft-storage) for a complete scenario.

</Note>

```hcl
ui = true
cluster_addr = "https://127.0.0.1:8201"
Expand Down Expand Up @@ -66,15 +74,19 @@ to specify where the configuration is.

- `user_lockout` `([UserLockout][user-lockout]: nil)`
Configures the user-lockout behaviour for failed logins. For more information, please see the
[user lockout configuration documentation](/vault/docs/configuration/user-lockout).
[user lockout configuration documentation](/vault/docs/configuration/user-lockout).

- `seal` `([Seal][seal]: nil)` – Configures the seal type to use for
auto-unsealing, as well as for
[seal wrapping][sealwrap] as an additional layer of data protection.

- `cluster_name` `(string: <generated>)` – Specifies the identifier for the
Vault cluster. If omitted, Vault will generate a value. When connecting to
Vault Enterprise, this value will be used in the interface.
- `reporting` `([Reporting][reporting]: nil)` -
Configures options relating to license reporting in Vault.

- `cluster_name` `(string: <generated>)` – Specifies a human-readable
identifier for the Vault cluster. If omitted, Vault will generate a value.
The cluster name is included as a label in some [telemetry metrics](/vault/docs/internals/telemetry/metrics/).
The cluster name is safe to update on an existing Vault cluster.

- `cache_size` `(string: "131072")` – Specifies the size of the read cache used
by the physical storage subsystem. The value is in number of entries, so the
Expand Down Expand Up @@ -115,10 +127,14 @@ to specify where the configuration is.
sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault))
```

~> Note: Since each plugin runs as a separate process, you need to do the same
<Note>

Since each plugin runs as a separate process, you need to do the same
for each plugin in your [plugins
directory](/vault/docs/plugins/plugin-architecture#plugin-directory).

</Note>

If you use a Linux distribution with a modern version of systemd, you can add
the following directive to the "[Service]" configuration section:

Expand Down Expand Up @@ -169,10 +185,10 @@ to specify where the configuration is.
maximum request duration allowed before Vault cancels the request. This can
be overridden per listener via the `max_request_duration` value.

- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal
mutex locks that should be monitored for potential deadlocks. Currently supported values
- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal
mutex locks that should be monitored for potential deadlocks. Currently supported values
include `statelock`, `quotas` and `expiration` which will cause "POTENTIAL DEADLOCK:"
to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this
to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this
can have a negative effect on performance due to the tracking of each lock attempt.

- `raw_storage_endpoint` `(bool: false)` – Enables the `sys/raw` endpoint which
Expand Down Expand Up @@ -210,12 +226,20 @@ can have a negative effect on performance due to the tracking of each lock attem
Supported values (in order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`.
This can also be specified via the `VAULT_LOG_LEVEL` environment variable.
~> Note: On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level,
<Note>
On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level,
overriding (even if specified) both the CLI flag and environment variable.
~> Note: Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular,
</Note>
<Note>
Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular,
secrets/auth plugins are currently not updated dynamically.

</Note>

- `log_format` - Equivalent to the [`-log-format` command-line flag](/vault/docs/commands/server#_log_format).

- `log_file` - Equivalent to the [`-log-file` command-line flag](/vault/docs/commands/server#_log_file).
Expand All @@ -236,9 +260,6 @@ can have a negative effect on performance due to the tracking of each lock attem
When `imprecise_lease_role_tracking` is set to true and a new role-based quota is enabled, subsequent lease counts start from 0.
`imprecise_lease_role_tracking` affects role-based lease count quotas, but reduces latencies when not using role based quotas.

- `request_limiter` `([Request Limiter][request-limiter]: <none>)` – Allows
operators to enable Vault's Request Limiter functionality.
### High availability parameters

The following parameters are used on backends that support [high availability][high-availability].
Expand Down Expand Up @@ -283,6 +304,9 @@ The following parameters are only used with Vault Enterprise
provided via the environment variable `VAULT_LICENSE_PATH`, or the license
itself can be provided in the environment variable `VAULT_LICENSE`.
- `administrative_namespace_path` `(string: "")` - Specifies the absolute path
to the Vault namespace to be used as an [Administrative namespace](/vault/docs/enterprise/namespaces/create-admin-namespace).
[storage-backend]: /vault/docs/configuration/storage
[listener]: /vault/docs/configuration/listener
[seal]: /vault/docs/configuration/seal
Expand All @@ -291,4 +315,3 @@ The following parameters are only used with Vault Enterprise
[sentinel]: /vault/docs/configuration/sentinel
[high-availability]: /vault/docs/concepts/ha
[plugins]: /vault/docs/plugins
[request-limiter]: /vault/docs/concepts/request-limiter
42 changes: 42 additions & 0 deletions website/content/docs/configuration/reporting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
layout: docs
page_title: Reporting - Configuration
description: |-
The reporting stanza specifies various parameters for tuning reporting and licensing related values.
---

# `replication` stanza

@include 'alerts/enterprise-only.mdx'

The `reporting` stanza specifies various parameters for tuning replication related values.

Please see the pages relating to
[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
and [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting)
for more information regarding the license reporting.

```hcl
reporting {
snapshot_retention_time = 9600
disable_product_usage_reporting = false
license {
enabled = true
}
}
```

## `reporting` parameters

- `snapshot_retention_time` `(duration: 9600h)` - The retention time for manual reporting snapshots in hours. Defaults to 9600 (400 days).
Uses [duration format strings](/vault/docs/concepts/duration-format).
- `disable_product_usage_reporting` `(boolean: false)` - Determines whether [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting)
is enabled.

## `license` parameters

- `enabled` `(boolean: true)` - Toggles automatic reporting of license utilization. See the
[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
page for more details.
- `billing_start_timestamp` `(timestamp)` - The start timestamp for billing for license reporting (manual and automated).
Defaults to the license start timestamp.
131 changes: 131 additions & 0 deletions website/content/docs/enterprise/license/product-usage-reporting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
layout: docs
page_title: Product usage reporting
description: >-
Learn what anonymous usage data HashiCorp collects as part of Enterprise utilization reporting. Enable or disable collection.
---

# Product usage reporting

@include 'alerts/enterprise-only.mdx'

HashiCorp collects usage data about how Vault clusters are being used. This data is not
used for billing or and is numerical only, and no sensitive information of
any nature is being collected. The data is GDPR compliant and is collected as part of
the [license utilization reporting](/vault/docs/enterprise/license/utilization-reporting)
process. If automated reporting is enabled, this data will be collected automatically.
If automated reporting is disabled, then this will be collected as part of the manual reports.

## Opt out

While none of the collected usage metrics are sensitive in any way, if you are still concerned
about these usage metrics being reported, then you can opt-out of them being collected.

If you are considering opting out because you’re worried about the data, we
strongly recommend that you review the [usage metrics list](#usage-metrics-list)
before opting out. If you have concerns with any of the automatically-reported
data please bring them to your account manager.

You have two options to opt out of product usage collection:

- HCL configuration (recommended)
- Environment variable (requires restart)


#### HCL configuration

Opting out in your product's configuration file doesn't require a system
restart, and is the method we recommend. Add the following block to your server
configuration file (e.g. `vault-config.hcl`).

```hcl
reporting {
disable_product_usage_reporting = true
}
```

<Warning>

When you have a cluster, each node must have the reporting stanza in its
configuration to be consistent. In the event of leadership change, nodes will
use its server configuration to determine whether or not to opt-out the
product usage collection. Inconsistent configuration between nodes will change the
reporting status upon active unseal.

</Warning>


You will find the following entries in the server log.

<CodeBlockConfig hideClipboard>

```
[DEBUG] activity: there is no reporting agent configured, skipping counts reporting
```

</CodeBlockConfig>

#### Environment variable

If you need to, you can also opt out using an environment variable, which will
provide a startup message confirming that you have product usage data collection.
This option requires a system restart.

<Note>

If the reporting stanza exists in the configuration file, the
`OPTOUT_PRODUCT_USAGE_REPORTING` value overrides the configuration.

</Note>

Set the following environment variable.

```shell-session
$ export OPTOUT_PRODUCT_USAGE_REPORTING=true
```

Now, restart your [Vault servers](/vault/docs/commands/server) from the shell
where you set the environment variable.

You will find the following entries in the server log.

<CodeBlockConfig hideClipboard>

```
[DEBUG] core: product usage reporting disabled
```

</CodeBlockConfig>

If your configuration file and environment variable differ, the environment
variable setting will take precedence.

## Usage metrics list

HashiCorp collects the following product usage metrics as part of the `metrics` part of the
[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads).
All of these metrics are numerical, and contain no sensitive values or additional metadata:

| Metric Name | Description |
|--------------------------------------------|--------------------------------------------------------------------------|
| `vault.namespaces.count` | Total number of namespaces. |
| `vault.leases.count` | Total number of leases within Vault. |
| `vault.quotas.ratelimit.count` | Total number of rate limit quotas within Vault. |
| `vault.quotas.leasecount.count` | Total number of lease count quotas within Vault. |
| `vault.kv.version1.secrets.count` | Total number of KVv1 secrets within Vault. |
| `vault.kv.version2.secrets.count` | Total number of KVv2 secrets within Vault. |
| `vault.kv.version1.secrets.namespace.max` | The highest number of KVv1 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version2.secrets.namespace.max` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version1.secrets.namespace.min` | The lowest number of KVv1 secrets in a namespace in Vault, e.g. `2`. |
| `vault.kv.version2.secrets.namespace.min` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. |
| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv1 secrets in namespaces in Vault, e.g. `52.8`. |
| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv2 secrets in namespaces in Vault, e.g. `52.8`. |

## Usage metadata list

HashiCorp collects the following product usage metadata as part of the `metadata` part of the
[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads):

| Metadata Name | Description |
|----------------------|----------------------------------------------------------------------|
| `replication_status` | Replication status of this cluster, e.g. `perf-disabled,dr-disabled` |
18 changes: 18 additions & 0 deletions website/data/docs-nav-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,15 @@
},
"path": "configuration/replication"
},
{
"title": "<code>reporting</code>",
"badge": {
"text": "ENTERPRISE",
"type": "outlined",
"color": "neutral"
},
"path": "configuration/reporting"
},
{
"title": "<code>seal</code>",
"routes": [
Expand Down Expand Up @@ -440,6 +449,11 @@
},
{
"title": "<code>sentinel</code>",
"badge": {
"text": "ENTERPRISE",
"type": "outlined",
"color": "neutral"
},
"path": "configuration/sentinel"
},
{
Expand Down Expand Up @@ -2700,6 +2714,10 @@
"title": "Manual license utilization reporting",
"path": "enterprise/license/manual-reporting"
},
{
"title": "Product usage reporting",
"path": "enterprise/license/product-usage-reporting"
},
{
"title": "FAQ",
"path": "enterprise/license/faq"
Expand Down

0 comments on commit 08a6d3a

Please sign in to comment.