Streaming telemetry implementation #971
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Create TaskRunMetrics class for individual task timing and error tracking - Enhance TasksRunner to capture start/end times and errors for each task - Add addTaskTelemetry method to TelemetryProcessor - Update MetadataDumper to pass telemetry processor to TasksRunner - Add comprehensive tests for new telemetry functionality - Maintain backward compatibility with existing telemetry system This enhancement provides granular visibility into individual task execution performance and error details, complementing the existing overall runtime metrics.
- Add EventType.METADATA for streaming telemetry events - Implement builder pattern for ClientTelemetry - Add TelemetryWriteStrategy interface with process/flush methods - Implement DiskTelemetryWriteStrategy and NoOpTelemetryWriteStrategy - Inject FileSystem into TelemetryProcessor for ZIP file writing - Update TelemetryProcessor to support immediate event processing - Add setFileSystem() method to enable ZIP-based telemetry writing
kaxuna
requested review from
shevek-google and
vladislav-sidorovich
as code owners
Code Coverage Report
|
misolt
reviewed
Sep 19, 2025
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Show resolved
Hide resolved
misolt
reviewed
Sep 19, 2025
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Outdated
Show resolved
Hide resolved
misolt
reviewed
Sep 19, 2025
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Outdated
Show resolved
Hide resolved
misolt
reviewed
Sep 19, 2025
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Show resolved
Hide resolved
...src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/ClientTelemetry.java
Outdated
Show resolved
Hide resolved
...src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/ClientTelemetry.java
Outdated
Show resolved
Hide resolved
dumper/app/src/main/java/com/google/edwmigration/dumper/application/dumper/TasksRunner.java
Outdated
Show resolved
Hide resolved
...src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/ClientTelemetry.java
Outdated
Show resolved
Hide resolved
| DUMPER_RUN_METRICS | ||
| DUMPER_RUN_START, | ||
| DUMPER_RUN_END, | ||
| METADATA, |
Member
There was a problem hiding this comment.
Let's use event name.
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
|
|
||
| /** | ||
| * @author kakha keep immutable. TaskRunner is multi-threaded, so we need to make it thread-safe. |
Member
There was a problem hiding this comment.
Let's add java doc to describe the meaning of the class.
.../src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/TaskRunMetrics.java
Outdated
Show resolved
Hide resolved
.../src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/TaskRunMetrics.java
Outdated
Show resolved
Hide resolved
.../src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/TaskRunMetrics.java
Outdated
Show resolved
Hide resolved
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Outdated
Show resolved
Hide resolved
.../main/java/com/google/edwmigration/dumper/application/dumper/DiskTelemetryWriteStrategy.java
Show resolved
Hide resolved
|
This PR is stale because it has been open 35 days with no activity. Remove "stale" label or comment or this will be closed in 14 days. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.