Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][v2][storage] Create v2 query service to operate on otlp data model #6343

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

mahadzaryab1
Copy link
Collaborator

@mahadzaryab1 mahadzaryab1 commented Dec 11, 2024

Which problem is this PR solving?

Description of the changes

  • Implement a v2 version of the query service that operates on the OTLP data model. This PR will be followed up by a series of PRs where this this new query service will be updated with the existing handlers. Once all the handlers have been migrated to use this query service, we can remove the old one.

How was this change tested?

Checklist

Copy link

codecov bot commented Dec 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 82 lines in your changes missing coverage. Please review.

Project coverage is 95.90%. Comparing base (ba6228f) to head (d31a5e5).

Files with missing lines Patch % Lines
cmd/query/app/querysvc/query_service_v2.go 0.00% 82 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6343      +/-   ##
==========================================
- Coverage   96.27%   95.90%   -0.38%     
==========================================
  Files         367      368       +1     
  Lines       20978    21060      +82     
==========================================
  Hits        20197    20197              
- Misses        597      679      +82     
  Partials      184      184              
Flag Coverage Δ
badger_v1 8.54% <0.00%> (-0.41%) ⬇️
badger_v2 1.75% <0.00%> (-0.09%) ⬇️
cassandra-4.x-v1-manual 14.20% <0.00%> (-0.67%) ⬇️
cassandra-4.x-v2-auto 1.69% <0.00%> (-0.09%) ⬇️
cassandra-4.x-v2-manual 1.69% <0.00%> (-0.09%) ⬇️
cassandra-5.x-v1-manual 14.20% <0.00%> (-0.67%) ⬇️
cassandra-5.x-v2-auto 1.69% <0.00%> (-0.09%) ⬇️
cassandra-5.x-v2-manual 1.69% <0.00%> (-0.09%) ⬇️
elasticsearch-6.x-v1 17.71% <0.00%> (-0.85%) ⬇️
elasticsearch-7.x-v1 17.79% <0.00%> (-0.84%) ⬇️
elasticsearch-8.x-v1 17.95% <0.00%> (-0.85%) ⬇️
elasticsearch-8.x-v2 1.74% <0.00%> (-0.10%) ⬇️
grpc_v1 10.12% <0.00%> (-0.49%) ⬇️
grpc_v2 7.73% <0.00%> (-0.36%) ⬇️
kafka-v1 8.88% <0.00%> (-0.42%) ⬇️
kafka-v2 1.75% <0.00%> (-0.09%) ⬇️
memory_v2 1.75% <0.00%> (-0.09%) ⬇️
opensearch-1.x-v1 17.84% <0.00%> (-0.85%) ⬇️
opensearch-2.x-v1 17.84% <0.00%> (-0.84%) ⬇️
opensearch-2.x-v2 1.75% <0.00%> (-0.09%) ⬇️
tailsampling-processor 0.44% <0.00%> (-0.03%) ⬇️
unittests 94.76% <0.00%> (-0.38%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mahadzaryab1 mahadzaryab1 changed the title [WIP][v2][storage] Create v2 query service to operate on otel data model [WIP][v2][storage] Create v2 query service to operate on otlp data model Dec 12, 2024
mahadzaryab1 added a commit that referenced this pull request Dec 23, 2024
…e on otlp format (#6396)

## Which problem is this PR solving?
- Towards #6344 

## Description of the changes
- Implemented a function `StandardAdjusters` that returns a list of
adjusters to be applied on ptrace.Traces
- This will be used by the v2 query service in
#6343

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `npm run lint` and `npm run test`

---------

Signed-off-by: Mahad Zaryab <[email protected]>
Comment on lines +147 to +154
func (qs QueryServiceV2) Adjust(tracesIter iter.Seq[[]ptrace.Traces]) {
tracesIter(func(traces []ptrace.Traces) bool {
for _, trace := range traces {
qs.options.Adjuster.Adjust(trace)
}
return true
})
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro Did I understand correctly in that this is what we wanted here? Or did you mean that we should change the underlying adjusters themselves to work on Seq?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you understood correctly. However, because adjusting needs to re-arrange the data I think this func should return Seq[ptrace.Traces] where each item is fully adjusted trace.

Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1 mahadzaryab1 added the changelog:new-feature Change that should be called out as new feature in CHANGELOG label Dec 24, 2024
@mahadzaryab1
Copy link
Collaborator Author

@yurishkuro Should this be classified as a new feature or a minor feature?

Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

cmd/query/app/querysvc/query_service_v2.go Outdated Show resolved Hide resolved
cmd/query/app/querysvc/query_service_v2.go Outdated Show resolved Hide resolved
func (qs QueryServiceV2) Adjust(tracesIter iter.Seq[[]ptrace.Traces]) {
tracesIter(func(traces []ptrace.Traces) bool {
for _, trace := range traces {
qs.options.Adjuster.Adjust(trace)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you cannot assume here that trace is a full trace, you need to clump consecutive chunks if they are for the same trace ID. We can implement a helper function for that that will take Seq[[]ptrace.Traces] and return Seq[ptrace.Traces] where each item is a full trace. Similar to what I was suggesting in https://github.com/jaegertracing/jaeger/pull/6388/files#r1894703201

Comment on lines +147 to +154
func (qs QueryServiceV2) Adjust(tracesIter iter.Seq[[]ptrace.Traces]) {
tracesIter(func(traces []ptrace.Traces) bool {
for _, trace := range traces {
qs.options.Adjuster.Adjust(trace)
}
return true
})
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you understood correctly. However, because adjusting needs to re-arrange the data I think this func should return Seq[ptrace.Traces] where each item is fully adjusted trace.

@yurishkuro
Copy link
Member

This is more like a refactoring, not a new feature.

@mahadzaryab1 mahadzaryab1 added changelog:bugfix-or-minor-feature and removed changelog:new-feature Change that should be called out as new feature in CHANGELOG labels Dec 25, 2024
// The iterator is single-use: once consumed, it cannot be used again.
func (qs QueryServiceV2) GetTraces(
ctx context.Context,
traceIDs ...tracestore.GetTraceParams) iter.Seq2[[]ptrace.Traces, error] {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
traceIDs ...tracestore.GetTraceParams) iter.Seq2[[]ptrace.Traces, error] {
traceIDs ...tracestore.GetTraceParams,
) iter.Seq2[[]ptrace.Traces, error] {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants