Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.x](backport #4501) Add reference to prefer-ilm setting for apm lifeycle known bug #4515

Merged
merged 1 commit into from
Nov 8, 2024
Merged

[8.x](backport #4501) Add reference to prefer-ilm setting for apm lifeycle known bug #4515

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 16 additions & 52 deletions docs/en/observability/apm/known-issues.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,11 @@ to `true` by following the {observability-guide}/apm-ilm-how-to.html[updated gui
[discrete]
== Upgrading to v8.15.x may cause ingestion to fail

_Elastic Stack versions: 8.15.0+_
_Elastic Stack versions: 8.15.0, 8.15.1, 8.15.2, 8.15.3_ +
_Fixed in Elastic Stack version 8.15.4_

// The conditions in which this issue occurs
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version.
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version prior to 8.15.4.
The issue does _not_ occur when creating a _new_ cluster using any 8.15.x version, or when upgrading
from 8.12.2 to 8.13.x or 8.14.x and then to 8.15.x.

Expand Down Expand Up @@ -106,63 +107,26 @@ stream lifecycle configuration, such existing data streams become unmanaged for
lifecycle configurations.

// How to fix it
Upgrading to 8.15.1 should fix any new indices created for the data stream. However,
indices created in version 8.15.0 would remain unmanaged if the default ILM policy is
used. One of the following approaches can be adopted to fix the unmanaged indices:

. Manually delete the indices when they are no longer needed.
. Explicitly configure APM data streams with the default data stream lifecycle config.
Using this approach would migrate all data streams to use data stream lifecycles,
which should be equivalent to the default ILM policies:
Upgrading to 8.15.1 resolves the lifecycle issue for any new indices created for APM data streams.
However, indices created in version 8.15.0 will remain unmanaged if the default ILM policy is in place.
To fix these unmanaged indices, consider one of the following approaches:

. Manually delete the unmanaged indices when they are no longer needed.
. Explicitly configure APM data streams to use the default data stream lifecycle configuration.
This approach migrates all affected data streams to use data stream lifecycles,
maintaining behavior equivalent to the default ILM policies.
Apply this fix only to data streams that have unmanaged indices due to missing default ILM policies.
+
[source,txt]
----
PUT _data_stream/traces-apm-*/_lifecycle
PUT _data_stream/{{data_stream_name}}-{{data_stream_namespace}}/_lifecycle
{
"data_retention": "10d"
}

PUT _data_stream/traces-apm.rum*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/traces-apm.sampled*/_lifecycle
{
"data_retention": "1h"
}

PUT _data_stream/metrics-apm.*.1m-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.*.10m-*/_lifecycle
{
"data_retention": "180d"
}

PUT _data_stream/metrics-apm.*.60m-*/_lifecycle
{
"data_retention": "390d"
}

PUT _data_stream/metrics-apm.internal-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.app.*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/logs-apm.*/_lifecycle
{
"data_retention": "10d"
"data_retention": <data_retention_period>
}
----

Default `<data_retention_period>` for each data stream is available in {observability-guide}/apm-ilm-how-to.html[this guide].

// Link to fix if it exists
This issue is fixed in 8.15.1 (https://github.com/elastic/elasticsearch/pull/112432[elastic/elasticsearch#112432]).

Expand Down