Skip to content

[v2][storage] Create v2 query service to operate on otlp data model #6343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Dec 31, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ad0de07
Create Query Service V2 To Operate on OTEL Data Model
mahadzaryab1 Dec 11, 2024
24278bb
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 24, 2024
41f1d51
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 24, 2024
a33ab6c
Address Build Failures And Feedback
mahadzaryab1 Dec 24, 2024
ffad629
Fix ArchiveTrace And GetTraces To Operate On Iterators
mahadzaryab1 Dec 24, 2024
1d721ac
Change Adjuster To Work On Seq
mahadzaryab1 Dec 24, 2024
a6df503
Fix Error Capture
mahadzaryab1 Dec 24, 2024
60a875a
Address Feedback From PR Review
mahadzaryab1 Dec 25, 2024
d31a5e5
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 25, 2024
2c7a848
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 29, 2024
d32ba48
Run Linter
mahadzaryab1 Dec 29, 2024
ecab080
Update Query Service To Perform Adjustments
mahadzaryab1 Dec 30, 2024
9ee2d38
Fix Signature of ArchiveTrace Function
mahadzaryab1 Dec 30, 2024
57ef9e7
Add Some Unit Tests
mahadzaryab1 Dec 30, 2024
40d9a68
Add Missing License
mahadzaryab1 Dec 30, 2024
32491f5
Adjust Archive Trace
mahadzaryab1 Dec 30, 2024
3d6af5f
Aggregate Traces
mahadzaryab1 Dec 30, 2024
ab95ab7
Remove TODO Comment
mahadzaryab1 Dec 30, 2024
6737ca9
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 30, 2024
005d8d7
Use SpanIter
mahadzaryab1 Dec 30, 2024
7171785
Add Proceed Check
mahadzaryab1 Dec 30, 2024
e5a4b1b
Address Feedback From PR Review
mahadzaryab1 Dec 30, 2024
f887d09
Create Receive Traces Helper Function
mahadzaryab1 Dec 30, 2024
959d2d4
Add Unit Tests For Get Traces
mahadzaryab1 Dec 30, 2024
d468249
Fix Lint
mahadzaryab1 Dec 30, 2024
f6d157e
Add Unit Tests For Find Traces
mahadzaryab1 Dec 30, 2024
d2f1c95
Add Test For Archive Trace Writer Error
mahadzaryab1 Dec 31, 2024
6f69d63
Add Test For Archive Trace Writer Success
mahadzaryab1 Dec 31, 2024
9250285
Write Test For Get Traces In Archive Storage
mahadzaryab1 Dec 31, 2024
1e0c5b3
Move Tests To Separate Package To Simplify Naming
mahadzaryab1 Dec 31, 2024
c14fc4f
Fix Comment
mahadzaryab1 Dec 31, 2024
73fbb19
Add Test For Error In Get Trace
mahadzaryab1 Dec 31, 2024
5020f89
Use Flatten With Errors
mahadzaryab1 Dec 31, 2024
e29eafe
Add Test For Error In Get Trace Reader
mahadzaryab1 Dec 31, 2024
2c716bf
Cleanup Tests
mahadzaryab1 Dec 31, 2024
28e5fd0
Combine Tests For Brevity
mahadzaryab1 Dec 31, 2024
5ae23f2
Fix Lint
mahadzaryab1 Dec 31, 2024
a79bcac
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 31, 2024
5636427
Move Query Service To V2 Package
mahadzaryab1 Dec 31, 2024
529bcda
Fix Lint
mahadzaryab1 Dec 31, 2024
806a265
Drop V2 Suffix
mahadzaryab1 Dec 31, 2024
f163d55
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions cmd/query/app/querysvc/query_service.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,6 @@ import (
"github.com/jaegertracing/jaeger/storage_v2/v1adapter"
)

var errNoArchiveSpanStorage = errors.New("archive span storage was not configured")

const (
defaultMaxClockSkewAdjust = time.Second
)

// QueryServiceOptions has optional members of QueryService
type QueryServiceOptions struct {
ArchiveSpanReader spanstore.Reader
Expand Down
140 changes: 140 additions & 0 deletions cmd/query/app/querysvc/query_service_v2.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
// Copyright (c) 2024 The Jaeger Authors.
// SPDX-License-Identifier: Apache-2.0

package querysvc

import (
"context"
"errors"
"time"

"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/ptrace"

"github.com/jaegertracing/jaeger/cmd/query/app/querysvc/adjuster"
"github.com/jaegertracing/jaeger/model"
"github.com/jaegertracing/jaeger/pkg/iter"
"github.com/jaegertracing/jaeger/storage/spanstore"
"github.com/jaegertracing/jaeger/storage_v2/depstore"
"github.com/jaegertracing/jaeger/storage_v2/tracestore"
)

var errNoArchiveSpanStorage = errors.New("archive span storage was not configured")

const (
defaultMaxClockSkewAdjust = time.Second
)

// QueryServiceOptions has optional members of QueryService
type QueryServiceOptionsV2 struct {
ArchiveTraceReader tracestore.Reader
ArchiveTraceWriter tracestore.Writer
Adjuster adjuster.Adjuster
}

// StorageCapabilities is a feature flag for query service
type StorageCapabilitiesV2 struct {
ArchiveStorage bool `json:"archiveStorage"`
// TODO: Maybe add metrics Storage here
// SupportRegex bool
// SupportTagFilter bool
Copy link
Collaborator Author

@mahadzaryab1 mahadzaryab1 Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro do you have the context for these items? do we still want to do them?

}

// QueryService contains span utils required by the query-service.
type QueryServiceV2 struct {
traceReader tracestore.Reader
dependencyReader depstore.Reader
options QueryServiceOptionsV2
}

// NewQueryService returns a new QueryService.
func NewQueryServiceV2(
traceReader tracestore.Reader,
dependencyReader depstore.Reader,
options QueryServiceOptionsV2,
) *QueryServiceV2 {
qsvc := &QueryServiceV2{
traceReader: traceReader,
dependencyReader: dependencyReader,
options: options,
}

if qsvc.options.Adjuster == nil {
qsvc.options.Adjuster = adjuster.Sequence(adjuster.StandardAdjusters(defaultMaxClockSkewAdjust)...)
}
return qsvc

Check warning on line 65 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L55-L65

Added lines #L55 - L65 were not covered by tests
}

// GetTrace is the queryService implementation of tracestore.Reader.GetTrace
func (qs QueryServiceV2) GetTraces(ctx context.Context, traceIDs ...tracestore.GetTraceParams) iter.Seq2[[]ptrace.Traces, error] {
traceIter := qs.traceReader.GetTraces(ctx, traceIDs...)
_, err := iter.FlattenWithErrors(traceIter)
if errors.Is(err, spanstore.ErrTraceNotFound) {
if qs.options.ArchiveTraceReader == nil {
return func(yield func([]ptrace.Traces, error) bool) {
yield(nil, err)
}

Check warning on line 76 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L69-L76

Added lines #L69 - L76 were not covered by tests
}
traceIter = qs.options.ArchiveTraceReader.GetTraces(ctx, traceIDs...)

Check warning on line 78 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L78

Added line #L78 was not covered by tests
}
return traceIter

Check warning on line 80 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L80

Added line #L80 was not covered by tests
}

// GetServices is the queryService implementation of tracestore.Reader.GetServices
func (qs QueryServiceV2) GetServices(ctx context.Context) ([]string, error) {
return qs.traceReader.GetServices(ctx)

Check warning on line 85 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L84-L85

Added lines #L84 - L85 were not covered by tests
}

// GetOperations is the queryService implementation of tracestore.Reader.GetOperations
func (qs QueryServiceV2) GetOperations(
ctx context.Context,
query tracestore.OperationQueryParameters,
) ([]tracestore.Operation, error) {
return qs.traceReader.GetOperations(ctx, query)

Check warning on line 93 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L92-L93

Added lines #L92 - L93 were not covered by tests
}

// FindTraces is the queryService implementation of tracestore.Reader.FindTraces
func (qs QueryServiceV2) FindTraces(ctx context.Context, query tracestore.TraceQueryParams) iter.Seq2[[]ptrace.Traces, error] {
return qs.traceReader.FindTraces(ctx, query)

Check warning on line 98 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L97-L98

Added lines #L97 - L98 were not covered by tests
}

// ArchiveTrace is the queryService utility to archive traces.
func (qs QueryServiceV2) ArchiveTrace(ctx context.Context, traceID pcommon.TraceID) error {
if qs.options.ArchiveTraceWriter == nil {
return errNoArchiveSpanStorage
}
var err error
traces, err := iter.FlattenWithErrors(qs.GetTraces(ctx, tracestore.GetTraceParams{TraceID: traceID}))
if err != nil {
return err
}
for _, trace := range traces {
err = errors.Join(err, qs.options.ArchiveTraceWriter.WriteTraces(ctx, trace))
}
return err

Check warning on line 114 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L102-L114

Added lines #L102 - L114 were not covered by tests
}

// Adjust applies adjusters to the trace.
func (qs QueryServiceV2) Adjust(trace ptrace.Traces) {
qs.options.Adjuster.Adjust(trace)

Check warning on line 119 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L118-L119

Added lines #L118 - L119 were not covered by tests
}

// GetDependencies implements depstore.Reader.GetDependencies
func (qs QueryServiceV2) GetDependencies(ctx context.Context, endTs time.Time, lookback time.Duration) ([]model.DependencyLink, error) {
return qs.dependencyReader.GetDependencies(ctx, depstore.QueryParameters{
StartTime: endTs.Add(-lookback),
EndTime: endTs,
})

Check warning on line 127 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L123-L127

Added lines #L123 - L127 were not covered by tests
}

// GetCapabilities returns the features supported by the query service.
func (qs QueryServiceV2) GetCapabilities() StorageCapabilities {
return StorageCapabilities{
ArchiveStorage: qs.options.hasArchiveStorage(),
}

Check warning on line 134 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L131-L134

Added lines #L131 - L134 were not covered by tests
}

// hasArchiveStorage returns true if archive storage reader/writer are initialized.
func (opts *QueryServiceOptionsV2) hasArchiveStorage() bool {
return opts.ArchiveTraceReader != nil && opts.ArchiveTraceWriter != nil

Check warning on line 139 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L138-L139

Added lines #L138 - L139 were not covered by tests
}
Loading