Skip to content

[exporterhelper] Refactor persistent storage size backup #12890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dmitryax opened this issue Apr 18, 2025 · 3 comments · May be fixed by #13047 or #13126
Open

[exporterhelper] Refactor persistent storage size backup #12890

dmitryax opened this issue Apr 18, 2025 · 3 comments · May be fixed by #13047 or #13126
Assignees
@malus2077
Labels
area:exporter help wanted Good issue for contributors to OpenTelemetry Service to pick up

Comments

@dmitryax
Copy link
Member

Currently we have this logic to persist the current queue size every 10 writes to ensure it can be restored after a crash. The problem that it's not reliable and can lead to sizing problems.

Instead, we should write all the information every time under one key: start/stop/inprogres/size.
@dmitryax dmitryax mentioned this issue Apr 17, 2025
Open
@dmitryax dmitryax changed the title [exporterhelper] Updade perisistent storage size counting [exporterhelper] Refactor persistent storage size backup Apr 18, 2025
@dmitryax dmitryax added help wanted Good issue for contributors to OpenTelemetry Service to pick up area:exporter labels Apr 18, 2025
@malus2077
Copy link
Contributor

malus2077 commented May 3, 2025 •
edited
Loading

Hi! I'd like to work on this issue. Could you please assign it to me? Thanks!

@sincejune
Copy link
Contributor

@malus2077 Assigned to you.

@bogdandrutu bogdandrutu mentioned this issue May 16, 2025
Open
@malus2077 malus2077 linked a pull request May 17, 2025 that will close this issue
Open
@malus2077
Copy link
Contributor

I’ve opened PR #13047 to address this. Please have a look when you get a chance.

@malus2077 malus2077 mentioned this issue May 21, 2025
Merged
@bogdandrutu bogdandrutu mentioned this issue May 21, 2025
Closed
malus2077 added a commit to malus2077/opentelemetry-collector that referenced this issue May 28, 2025
@malus2077
[chore] Add unified queueState struct
e39a096 
- Add QueueMetadata protobuf definition
- Implement sizer type conversion utilities
- Update persistent queue to support unified metadata
- Add build rules for internal protobuf generation

Addresses: open-telemetry#12890
github-merge-queue bot pushed a commit that referenced this issue May 28, 2025
@malus2077
[chore] Add unified queueState struct (#13067)
279ccad 
Adds a single key structure to store information about the persistent
queue. #12890
@malus2077 malus2077 linked a pull request Jun 2, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@malus2077 malus2077

Labels
area:exporter help wanted Good issue for contributors to OpenTelemetry Service to pick up
Projects
None yet
Milestone
No milestone
3 participants
@dmitryax @sincejune @malus2077