Skip to content

Data Retention Regulator Change Log

Jump to bottom
Dmytro Vyazelenko edited this page Jun 27, 2025 · 11 revisions

1.2.4 - 2025-06-27

  • Bump Aeron to 1.48.4.
  • Bump Agrona to 2.2.4.
  • Bump Shadow to 8.3.7.
  • Bump JUnit to 5.13.2.
  • Bump Checkstyle to 10.26.0.

1.2.3 - 2025-06-20

  • Bump Aeron to 1.48.3.
  • Bump Agrona to 2.2.3.
  • Bump Gradle to 8.14.2.

1.2.2 - 2025-06-12

  • Bump Aeron to 1.48.2.
  • Bump JUnit to 5.13.1.

1.2.1 - 2025-06-06

1.2.0 - 2025-06-03

1.1.0 - 2025-05-09

  • Breaking (behaviour): Changed RetainEnoughDataForRecoveryPolicy and MatchingLogRecordingPolicy to fall back to Aeron's system properties for clusterId, clusterDirName, and standbyDirName if not set programmatically or via explicit policy properties.

  • Breaking (behaviour): Changed the DataRetentionRegulator#main entrypoint to make property value precedence more sane. Now, system properties take precedence over property values defined in properties files, and property values defined in "later" files (w.r.t. the command line arguments) take precedence over property values defined in "earlier files.

  • Breaking (API): Changed the signature of DataRetentionRegulator#execute to return a boolean indicating whether the data retention regulation was successfully completed or not. It returns false, e.g., if backing up a segment file was not complete or if an archive operation failed. This allows the caller to retry the operation.

  • Breaking (API, behaviour): DataRetentionRegulator.Context has had some significant changes:

    • #archiveContext now defaults to null. It is initialised to use the local control channels of the archive using the specified archiveDirName (which are read from its mark file) in the DataRetentionRegulator.Context#conclude method.
    • #aeronDirectoryName has been removed, as it is now discovered from the archive's mark file.
    • it no longer extends CommonContext.

  • Added support for automatic retries in the DataRetentionRegulator#main entrypoint (until successful execution) when the aeron.data.retention.execute.timeout system property is set to a positive value. The default is 0, which will behave as before (i.e., no retries). When set to a positive value, the regulator will retry the operation every aeron.data.retention.execute.retry.period (which defaults to 10 seconds) until the operation is successful or the timeout is reached. Both system properties accept time unit suffixes (ns, us, ms, and s).

  • Fixed a bug where MatchingLogRecordingPolicy did propagate an evaluation context to the inner policy; therefore, the inner policy might fail due to missing dependencies, e.g., the epochClock in DeleteSegmentsAfterPeriodPolicy.

  • Added a retentionPeriod overload to both DeleteSegmentsAfterPeriodPolicy.Context and DeleteRecordingsAfterPeriodPolicy.Context that accepts amount and unit simultaneously, to avoid the need to call two methods to set the retention period.

  • Improved javadoc for DeleteSegmentsAfterPeriodPolicy.Context

  • Made the retention behaviour of DeleteSegmentsAfterPeriodPolicy configurable via a mode. Opting into retentionMode(RETENTION_UNTIL_DELETION) makes it possible to compose the policy with others using RetainMaxOfPolicy in a way that will keep at least the retention period of data. Previously, and when using mode NO_RETENTION, the policy would only "cover" recordings when they had segments due for deletion, i.e., segments last modified before the retention period. When using the new RETENTION_UNTIL_DELETION mode, the policy will "cover" all recordings, i.e., the policy will ensure data is kept for the retention period by proposing a recording start position based on the earliest segment that was modified within the retention period.

  • Support providing a backing up strategy via the aeron.data.retention.backing.up.strategy system property, where the value should be the name of a class with a no-arg constructor that implements SegmentFileBackingUpStrategy.

  • Upgrade Aeron to 1.47.5.

  • Upgrade to Aeron Standby 1.47.5.

  • Upgrade to Agrona 2.1.0

1.0.0 - 2025-03-14

  • Breaking: groupId changed to io.aeron.premium.drr: 
    <groupId>io.aeron.premium.drr</groupId>
<artifactId>aeron-data-retention-regulator</artifactId>
  • Upgrade Aeron to 1.47.4.
  • Upgrade to Aeron Standby 1.47.4.

0.11.0 - 2025-01-31

Breaking changes

  • The aeron-data-retention-regulator-x.y.z.jar artifact is now a regular thin jar, the fat jar now lives in a new artifact aeron-data-retention-regulator-x.y.z-all.jar.
  • Upgrade Agrona to 2.0.1 - requires additional command line argument.

Changelog

  • Upgrade Aeron to 1.47.2.
  • The jars now contain the README and the CHANGELOG: 
    aeron-data-retention-regulator/README.md
aeron-data-retention-regulator/CHANGELOG.md

0.10.0 - 2024-11-01

  • Fix a bug where RecordingMetadata would be incorrectly populated (introduced in 0.7.0):
    • maxRecordedPosition would be correct only for stopped recordings, -1 for active ones
    • startTimestamp would be the maxRecordedPosition
    • stopTimestamp would be the startTimestamp. Which would have the following effects:
      • RetainLengthPolicy would always retain all active recordings
      • DeleteSegmentsAfterPeriodPolicy would not delete any segments for active recordings
      • DeleteRecordingsAfterPeriodPolicy would delete after period calculated from recording start instead of stop, unless recording is active, then it would fail to delete
  • Upgrade Aeron to 1.46.7.
  • Upgrade Agrona to 1.23.1.

0.9.0 - 2024-08-30

Breaking changes

  • Java 17 or newer is now required.

Changelog

  • JavaDoc jars are now published.
  • Upgrade to Aeron 1.46.0
  • Upgrade to Agrona 1.23.0

0.8.0 - 2024-08-30

  • Upgrade to Aeron 1.45.0
  • Upgrade to Agrona 1.22.0

0.7.0 - 2024-08-29

Non-breaking changes

  • Make testing policy trees easier in downstream projects without extensive mocking:
    • Allow creation of SegmentFileBoundary
    • Allow creation of RecordingMetadata
  • Make dry run output more precise by avoiding the calculation of new start positions before the existing start position in the built-in policies.

0.6.0 - 2024-08-14

Non-breaking changes

  • Introduced a new composite policy:
    • RetainMinOfPolicy: combines sub-policies by retaining the minimum of its inner policies.
  • Fixed a bug where DeleteRecordingsAfterPeriodPolicy gave the wrong retention instruction for live recordings. Thankfully, the purge would always fail, as the recording was live. A test has been added to prevent regressions.

0.5.0 (2024-08-01)

Non-breaking changes

  • Introduce a "dry run" mode, where the tool does not delete any data, but logs what it would have done. This can be useful for testing policies.

  • Change the default controlResponseStreamId we use from 20 to 20240712 to reduce the chances of collisions with other AeronArchive clients.

0.4.0 - 2024-07-24

Non-breaking changes

  • Make it easier to configure the controlReponseStreamId on the AeronArchive.Context that the DRR uses.
  • Avoid attempts to detach partial (i.e., not complete) segment files in DeleteSegmentsAfterPeriodPolicy if they have "expired".

0.3.0 - 2024-07-19

Breaking changes

  • Renamed DeleteAfterPeriodPolicy to the more-specific DeleteRecordingsAfterPeriodPolicy

Non-breaking changes

  • Introduced new policies:
    • DeleteSegmentsAfterPeriodPolicy: detaches and deletes recording segment files that have not been modified for some period of time.
    • MatchingLogRecordingPolicy: applies an inner policy to the Consensus Module's log recording.

0.2.0 - 2024-07-19

Noteworthy

Initial release with the following built-in data retention policies:

  • Leaf policies:
    • RetainEnoughDataForRecoveryPolicy: keeps the last N snapshots and the log following the earliest of these snapshots available for cluster node recovery.
    • RetainLengthPolicy: keeps at least the last N bytes of data. It will detach and delete recording segments before the last N bytes.
    • DeleteAfterPeriodPolicy: deletes recordings that stopped some period of time ago.
  • Composite policies:
    • MatchingStreamsPolicy: applies an inner policy to recordings that match a channel URI and/or stream identifier.
    • RetainMaxOfPolicy: safely combines policies, e.g., to retain X snapshots worth of cluster data and Y bytes of data per recording, by retaining the maximum of its inner policies.
    • CascadePolicy: accepts a sequence of policies. It will try each policy in turn and apply the first policy that "covers" a recording, i.e., that has an opinion on what the extent of the recording should be.
Clone this wiki locally