Add E2E test V2 Strategic testing framework for more maintainability #2904
Conversation
📝 WalkthroughWalkthroughThis pull request consists of extensive refactoring and feature enhancements across the codebase. New testing frameworks and sample files have been added; documentation and OpenAPI/Swagger files have been improved with detailed summaries, status code tables, and troubleshooting guidance. In addition, several new utilities and abstractions have been introduced (for example, a Cycle iterator and a noise generator), while error‐handling logic across network, runner, and server components has been streamlined using helper methods such as errors.IsNot and errors.IsAny. Dependency versions, Makefiles, Dockerfiles, and Kubernetes configuration files have also been updated. Changes
Sequence Diagram(s)Below is a high-level sequence diagram for the new noise generator feature: sequenceDiagram
participant C as Caller
participant NG as noiseGenerator
participant M as Modifier Function
C->>NG: Call New(data, num, opts)
NG-->>C: Return noiseGenerator instance
C->>NG: Call Mod()
NG-->>C: Return Modifier Function (closure)
C->>M: Invoke modifier(vector, index)
M-->>C: Return modified vector with noise
Possibly related PRs
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
[WARNING:INTCFG] Changes in |
github-actions
bot
added
area/discoverer
area/manager/index
area/filter/ingress
area/filter/egress
area/internal
type/ci
area/makefile
area/agent/sidecar
area/tools/cli/loadtest
area/gateway/lb
area/gateway/filter
area/agent/core/faiss
area/agent/core/ngt
area/gateway/mirror
area/index/job/correction
area/index/job/creation
area/index/job/save
area/index/job/readreplica
labels
|
[CHATOPS:HELP] ChatOps commands.
|
kpango
removed
type/feature
kpango
force-pushed
the
feature/test-e2e/add-e2e-test-v2-strategic-testing
branch
4 times, most recently
from
8401150 to
799102b
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 7
🔭 Outside diff range comments (2)
internal/iter/iter.go (1)
1-153: 💡 Verification agent🧩 Analysis chain
Add unit tests to verify functionality and improve coverage.
This is a new file with complex functionality, but there are no corresponding tests visible in the diff. Consider adding comprehensive unit tests to verify the behavior of each method, particularly:
NewCyclewith various input parameters- Cycle behavior with different array sizes and offset values
- Context cancellation handling
- Transformation function application
- Edge cases (empty arrays, nil arrays, etc.)
🏁 Script executed:
#!/bin/bash # Check if unit tests exist for the new iter package echo "Looking for test files for internal/iter/iter.go..." fd "iter_test.go" internal/iterLength of output: 144
Action Required: Add Comprehensive Unit Tests for Cycle Iterator Implementation
The file
internal/iter/iter.godefines a Cycle iterator with multiple complex behaviors (cyclic indexing, context handling, and an optional transformation function). However, our verification indicates that no unit tests exist for this functionality. To ensure stability and prevent regressions, please add comprehensive tests covering at least the following scenarios:
Initialization Tests:
ValidateNewCyclewith various inputs, including valid inputs,nilarrays, and empty arrays.Behavioral Tests:
Confirm that the iterator correctly manages cyclic indexing with different array sizes and offset values.Transformation Function:
Verify that the optional transformation function (modFunc) is applied correctly on-the-fly.Context Handling:
Ensure that context cancellation is properly handled in theValue,Seq2,Indexes, andForEachmethods.Edge Cases:
Test boundary conditions, including edge cases where the iterator might behave unexpectedly.Adding these unit tests will help maintain code quality and reliability as the functionality evolves.
.gitfiles (1)
1966-1976: 💡 Verification agent🧩 Analysis chain
Ensure Rust code follows project best practices for Option handling.
Based on a previous learning from PR #2807, Rust code should avoid using
unwrap()on Option types and instead handle None values explicitly. Ensure that the newly added Rust files:
rust/bin/agent/src/handler/object.rsrust/bin/meta/src/test_client.rsFollow this guidance to maintain robustness.
🏁 Script executed:
#!/bin/bash # Check for unwrap() calls in the new Rust files echo "Checking rust/bin/agent/src/handler/object.rs for unwrap() calls:" grep -n "unwrap()" rust/bin/agent/src/handler/object.rs echo -e "\nChecking rust/bin/meta/src/test_client.rs for unwrap() calls:" grep -n "unwrap()" rust/bin/meta/src/test_client.rsLength of output: 531
Action Required: Update unwrap() Usage in Rust Files
There are still instances of
unwrap()in the new code that violate our Option handling best practices (as per PR #2807). Please update the following:
- rust/bin/agent/src/handler/object.rs (line 44): Replace
hostname.to_str().unwrap()with proper error handling (e.g., by using a match statement or the?operator) to gracefully handle potentialNonevalues.- rust/bin/meta/src/test_client.rs (lines 31 & 32): Replace the calls to
unwrap()with explicit handling, ensuring that errors are managed appropriately.Implementing these changes will help maintain the robustness of our codebase.
♻️ Duplicate comments (8)
internal/file/file_test.go (1)
1523-1606:⚠️ Potential issueTestAbsolutePath needs implemented test cases
The test structure is sound, but it currently only has commented-out placeholders without actual test cases. Since this is testing a new function, it's critical to implement test cases that cover various scenarios.
Consider adding test cases for:
tests := []test{ - // TODO test cases - /* - { - name: "test_case_1", - args: args { - path:"", - }, - want: want{}, - checkFunc: defaultCheckFunc, - beforeFunc: func(t *testing.T, args args) { - t.Helper() - }, - afterFunc: func(t *testing.T, args args) { - t.Helper() - }, - }, - */ + { + name: "return empty string when path is empty", + args: args{ + path: "", + }, + want: want{ + want: "", + }, + }, + { + name: "return absolute path when path is relative", + args: args{ + path: "testdata/sample.txt", + }, + beforeFunc: func(t *testing.T, args args) { + t.Helper() + // Get the current working directory for comparison in checkFunc + }, + checkFunc: func(w want, got string) error { + // Since the absolute path depends on the current working directory, + // verify that the result starts with / (Unix) or contains :\ (Windows) + if !filepath.IsAbs(got) { + return errors.Errorf("got: \"%#v\" is not an absolute path", got) + } + return nil + }, + }, + { + name: "return same path when path is already absolute", + args: args{ + path: filepath.Join(os.TempDir(), "sample.txt"), + }, + want: want{ + want: filepath.Clean(filepath.Join(os.TempDir(), "sample.txt")), + }, + },internal/net/quic/conn_test.go (1)
18-108: 🛠️ Refactor suggestionTestNewConn implementation needed to validate QUIC connection creation
This test function is fully commented out with only a TODO placeholder. Given that this PR introduces a new E2E testing framework for better maintainability, having actual test implementations is crucial.
Implement at least basic test cases for these scenarios:
- Creating a connection with valid parameters
- Creating a connection with nil connection
- Error handling for invalid inputs
This will provide initial validation of the QUIC connection handling.
internal/test/data/vector/noise/noise.go (2)
78-106:⚠️ Potential issueAvoid potential divide-by-zero for all-zero-length vectors.
When the dataset contains vectors but all of them have zero length,countremains 0, leading tofloat64(count)in the denominator at line 105. This triggers a NaN or panic.Consider applying this fix to fall back safely:
--- a/noise.go +++ b/noise.go @@ -104,0 +105,4 @@ func() float32 { + if count == 0 { + log.Warn("All vectors in the dataset are zero-length. Falling back to noise level 0.01") + return 0.01 + }
119-123:⚠️ Potential issueCheck for 0 or 1 in required to avoid potential underflow.
IfminNoiseTableSizeis 0 or 1, thenrequired - 1might be 0, causingbits.Len64(0) = 0and leading to1 << 0 = 1. In a case whererequiredcould become 0, shifting could overflow.Suggest enforcing a minimum lower bound for
minNoiseTableSize:--- a/noise.go +++ b/noise.go @@ -119,6 +119,10 @@ func() int { if required < ng.minNoiseTableSize { required = ng.minNoiseTableSize } + // Prevent underflow or zero shift operand. + if required < 2 { + required = 2 + } return 1 << bits.Len64(required-1) }internal/net/grpc/status/status_test.go (1)
245-331:⚠️ Potential issueImplement test cases for the commented out TestIs function
The newly added
TestIstest function is completely commented out and lacks actual test cases. This test is essential for validating the functionality of the correspondingIsfunction that checks if an error corresponds to a specific gRPC status code.Consider uncommenting and implementing this test function with appropriate test cases to ensure the
Isfunction works as expected. At minimum, include test cases for:
- When the error is nil and code is valid
- When the error is a valid status error with the same code
- When the error is a valid status error with a different code
- When the error is a non-status error
-// func TestIs(t *testing.T) { +func TestIs(t *testing.T) { type args struct { err error code codes.Code } type want struct { want bool } type test struct { name string args args want want checkFunc func(want, bool) error beforeFunc func(*testing.T, args) afterFunc func(*testing.T, args) } defaultCheckFunc := func(w want, got bool) error { if !reflect.DeepEqual(got, w.want) { return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) } return nil } tests := []test{ - // TODO test cases - /* - { - name: "test_case_1", - args: args { - err:nil, - code:nil, - }, - want: want{}, - checkFunc: defaultCheckFunc, - beforeFunc: func(t *testing.T, args args) { - t.Helper() - }, - afterFunc: func(t *testing.T, args args) { - t.Helper() - }, - }, - */ + { + name: "nil error returns false", + args: args{ + err: nil, + code: codes.OK, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "non-status error returns false", + args: args{ + err: errors.New("regular error"), + code: codes.Internal, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "error with matching code returns true", + args: args{ + err: Error(codes.NotFound, "not found"), + code: codes.NotFound, + }, + want: want{ + want: true, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "error with different code returns false", + args: args{ + err: Error(codes.NotFound, "not found"), + code: codes.Internal, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, } for _, tc := range tests { test := tc t.Run(test.name, func(tt *testing.T) { tt.Parallel() defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) if test.beforeFunc != nil { test.beforeFunc(tt, test.args) } if test.afterFunc != nil { defer test.afterFunc(tt, test.args) } checkFunc := test.checkFunc if test.checkFunc == nil { checkFunc = defaultCheckFunc } got := Is(test.args.err, test.args.code) if err := checkFunc(test.want, got); err != nil { tt.Errorf("error = %v", err) } }) } -// } +}internal/servers/server/option_test.go (1)
3727-4066:⚠️ Potential issueImplement the commented-out gRPC option test functions
These GRPC option test functions (TestWithGRPCMaxConcurrentStreams, TestWithGRPCNumStreamWorkers, TestWithGRPCSharedWriteBuffer, and TestWithGRPCWaitForHandlers) are currently commented out and not implementing any actual test logic. As part of the E2E testing framework improvements, these tests should be properly implemented to ensure adequate test coverage of the gRPC server options.
Based on previous review comments, these tests should follow the same pattern as other option tests in the file. For example, this implementation pattern works for TestWithGRPCMaxConcurrentStreams:
-// func TestWithGRPCMaxConcurrentStreams(t *testing.T) { -// type args struct { -// size uint32 -// } -// type want struct { -// want Option -// } -// type test struct { -// ... -// } +func TestWithGRPCMaxConcurrentStreams(t *testing.T) { + type test struct { + name string + size uint32 + checkFunc func(opt Option) error + } + + tests := []test{ + { + name: "set success when size is more than 0", + size: 1024, + checkFunc: func(opt Option) error { + got := new(server) + opt(got) + + if len(got.grpc.opts) != 1 { + return errors.New("invalid param was set") + } + return nil + }, + }, + { + name: "not set when size is 0", + checkFunc: func(opt Option) error { + gopts := []grpc.ServerOption{ + grpc.ConnectionTimeout(10 * time.Second), + } + got := new(server) + got.grpc.opts = gopts + opt(got) + + if !reflect.DeepEqual(got.grpc.opts, gopts) { + return errors.New("invalid param was set") + } + return nil + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + opt := WithGRPCMaxConcurrentStreams(tt.size) + if err := tt.checkFunc(opt); err != nil { + t.Error(err) + } + }) + } +}internal/iter/iter.go (2)
52-59: 🛠️ Refactor suggestionReturn a no-op iterator for empty or nil slices.
Currently, the function returns
nilif the array is empty or nil. This could lead to nil pointer dereferences if callers don't check the return value before calling methods. Consider returning a valid but empty cycle instance instead.func NewCycle[S ~[]E, E any](array S, num, offset uint64, mod func(uint64, E) E) Cycle[S, E] { if array == nil { - return nil + array = make(S, 0) } size := uint64(len(array)) if size == 0 { - return nil + // Return an empty cycle that behaves safely + return &cycle[S, E]{ + start: 0, + num: 0, + size: 0, + offset: offset, + array: array, + modFunc: mod, + } } return &cycle[S, E]{
20-23:⚠️ Potential issueFix circular import issue with
"iter"package.There's a potential circular dependency here. You're importing a package named
"iter"within a package that's also namediter. This is likely trying to use the Go standard library's experimental iterator package, but importing it as a bare import path creates ambiguity.Two options to resolve this:
- Use a qualified import to disambiguate:
stdlib_iter "iter"- If using Go 1.22+, use the full path:
"golang.org/x/exp/iter"import ( "context" - "iter" + stdlib_iter "iter" )
🧹 Nitpick comments (27)
apis/swagger/v1/vald/remove.swagger.json (1)
50-51: Expanded Details for the/remove/multipleEndpoint
The revisedsummarynow clearly labels the endpoint as multi-remove and emphasizes a single request operation. The addeddescriptionincludes an HTML notice for gRPC message size limitations and a detailed error resolution table, which improves clarity. Please verify that the HTML content renders correctly in your API documentation tools.apis/swagger/v1/vald/filter.swagger.json (2)
177-178: Detailed Summary and Description for MultiUpdateObject EndpointThe revised summary now clearly states that multiple objects can be updated in one request. The additional description with an HTML notice regarding the gRPC message size limitation is informative. However, please verify that the HTML markup (
<div class="notice">...) renders as expected in your API documentation tools. If not, consider switching to markdown formatting.
242-243: Detailed Summary and Description for MultiUpsertObject EndpointThe updated summary precisely indicates that multiple objects can be upserted in one request. The accompanying description again uses an HTML notice to warn about gRPC message size limitations. Please ensure that this HTML markup is supported by your documentation generator or consider alternative formatting for consistency.
apis/swagger/v1/vald/insert.swagger.json (3)
18-18: Review /insert Description Formatting
The added detailed description with a markdown table clearly outlines error codes and troubleshooting steps. Please ensure that the markdown table renders correctly and consistently within your API documentation tool.
50-50: Clarify MultiInsert Summary Formatting
The summary for the MultiInsert endpoint now indicates its purpose clearly. Consider revising the emphasis on the numeral (the use of "1 request")—for instance, replacing it with "a single request" could improve readability and consistency.
51-51: Ensure Consistent HTML and Markdown Rendering in MultiInsert Description
The description mixes an HTML notice (using<div class="notice">) with markdown content. Verify that your documentation viewer renders this hybrid format correctly and that it aligns stylistically with other endpoint descriptions.apis/swagger/v1/vald/upsert.swagger.json (2)
17-18: Enhanced Documentation for the /upsert Endpoint
The revisedsummaryanddescriptionnow provide a very detailed explanation including status codes and troubleshooting guidance. Please verify that the markdown table and line breaks render correctly in your Swagger UI.
50-52: Enhanced Documentation for the /upsert/multiple Endpoint
The updatedsummaryclearly explains the MultiUpsert functionality and thedescriptionoffers comprehensive troubleshooting information, including a notice on gRPC message size limits. Ensure the embedded HTML snippet (i.e.<div class="notice">) renders as expected in your API documentation viewer.internal/net/quic/conn_test.go (1)
16-16: Add specific timeline or task reference to "NOT IMPLEMENTED" commentTo improve clarity and tracking, enhance the "NOT IMPLEMENTED BELOW" comment with more specific information such as:
- // NOT IMPLEMENTED BELOW + // NOT IMPLEMENTED BELOW - Will be implemented in follow-up PR #XXXXor
- // NOT IMPLEMENTED BELOW + // NOT IMPLEMENTED BELOW - TODO(username): Implement by YYYY-MM-DDThis provides better context for other developers and helps with project management.
apis/docs/v1/docs.md (2)
1638-1639: Enhance Grammar in Filter Service Overview
In the sentence "Vald Filter Gateway forward user request to user-defined ingress/egress filter components allowing user to run custom logic," consider using "forwards user requests" to improve grammatical clarity.
1202-1203: Correct Typographical Error in Upsert Section
There is a typo in the phrase "Represent mthe ultiple upsert request." Please change it to "Represent the multiple upsert request."pkg/index/job/deletion/service/deleter_test.go (2)
16-17: File structure suggests incomplete implementationThe "NOT IMPLEMENTED BELOW" comment indicates that this file contains test stubs rather than functioning tests. Consider either removing this comment when implementing the tests or providing more specific TODO comments within each test function.
1-527: Implement a comprehensive E2E test for the deletion workflowThis file provides individual test stubs but lacks an end-to-end test that validates the complete deletion workflow. Consider adding a test that simulates a realistic deletion scenario from start to finish.
// Add this function after implementing the required mocks func TestE2EDeleteWorkflow(t *testing.T) { // Setup test environment ctx := context.Background() mockClient := &mockClient{ mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { return &mockRemoveClient{ mockRemove: func(ctx context.Context, req *payload.Remove_Request, opts ...grpc.CallOption) (*payload.Object_Location, error) { return &payload.Object_Location{ Name: req.GetId(), }, nil }, }, nil }, } // Create deleter with test configuration deleter, err := New( WithTargetIndexID("test-index"), WithConcurrency(2), ) if err != nil { t.Fatalf("failed to create deleter: %v", err) } // Set mock client and target addresses idx := deleter.(*index) idx.client = mockClient idx.targetAddrs = []string{"addr1", "addr2"} // Execute deletion workflow err = idx.Start(ctx) if err != nil { t.Fatalf("deletion workflow failed: %v", err) } }internal/sync/errgroup/group_test.go (1)
1172-1288: Commented-out test code ready for removalThis commented-out test function for
Test_group_TryGois superseded by the newTest_group_execfunction above. Since you've fully migrated to the new approach with atomic limits, this code can be safely removed rather than kept as commented code.-// func Test_group_TryGo(t *testing.T) { -// type args struct { -// f func() error -// } -// type fields struct { -// egctx context.Context -// cancel context.CancelCauseFunc -// limit atomic.Int64 -// sem *semaphore.Weighted -// emap map[string]struct{} -// errs []error -// } -// type want struct { -// want bool -// } -// type test struct { -// name string -// args args -// fields fields -// want want -// checkFunc func(want, bool) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } -// defaultCheckFunc := func(w want, got bool) error { -// if !reflect.DeepEqual(got, w.want) { -// return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) -// } -// return nil -// } -// tests := []test{ -// // TODO test cases -// /* -// { -// name: "test_case_1", -// args: args { -// f:nil, -// }, -// fields: fields { -// egctx:nil, -// cancel:nil, -// limit:nil, -// sem:nil, -// emap:nil, -// errs:nil, -// }, -// want: want{}, -// checkFunc: defaultCheckFunc, -// beforeFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// afterFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// }, -// */ -// -// -// // TODO test cases -// /* -// func() test { -// return test { -// name: "test_case_2", -// args: args { -// f:nil, -// }, -// fields: fields { -// egctx:nil, -// cancel:nil, -// limit:nil, -// sem:nil, -// emap:nil, -// errs:nil, -// }, -// want: want{}, -// checkFunc: defaultCheckFunc, -// beforeFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// afterFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// } -// }(), -// */ -// } -// -// -// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// g := &group{ -// egctx: test.fields.egctx, -// cancel: test.fields.cancel, -// limit: test.fields.limit, -// sem: test.fields.sem, -// emap: test.fields.emap, -// errs: test.fields.errs, -// } -// -// -// got := g.TryGo(test.args.f) -// if err := checkFunc(test.want, got); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// }internal/params/params_test.go (1)
562-800: Consider cleaning up commented-out test code.Large sections of commented-out test code for
Test_parser_RestoreandTest_parser_Overrideremain in the file. If these functions are planned for future implementation, consider adding a TODO comment with a tracking issue. If they're no longer needed, it would be cleaner to remove them entirely.apis/swagger/v1/vald/search.swagger.json (1)
83-84: Improved multi-ID search documentation for/linearsearch/id/multiple.
The summary now succinctly explains the endpoint’s purpose while the description warns about gRPC message size limitations and provides a comprehensive error troubleshooting table.internal/net/grpc/option_test.go (1)
20-105: Consider implementing the TODO test cases.All test functions across the file contain "TODO test cases" comments. While the function structure and naming are now improved, the actual test implementations are still missing.
// TODO test cases /* { name: "test_case_1", args: args { dur:"", }, want: want{}, ... */Consider implementing these test cases to ensure proper test coverage for the gRPC options.
Would you like me to generate sample test cases for any of these functions?
Also applies to: 107-190, 192-275, 277-360, 362-445, 447-530, 532-3344
go.mod (3)
3-18: Go Version and Cloud Dependency Upgrades
The Go version is now updated to 1.24.1 and several Cloud-related modules (e.g.cloud.google.com/go,bigquery,compute,iam,kms,monitoring,pubsub,secretmanager,storage, andtrace) have been bumped to newer versions. Please ensure that these changes are fully compatible with your codebase and deployment environments.
37-39: TOML and Cloud SQL-Proxy Version Bumps
Thegithub.com/BurntSushi/tomllibrary is updated to v1.5.0 and the Cloud SQL Proxy dependency to v1.37.6. These kinds of changes can sometimes introduce subtle API or behavior modifications. It is advisable to run any dependent tests and confirm that configuration parsing and proxy functionality remain intact.
352-424: First Require Block: Direct Dependency Updates
In the firstrequireblock, many direct dependencies have been updated (including updates for cloud libraries, AWS SDK v1, several Google libraries, Kubernetes client libraries, and more). It is critical to run a full build and execute the test suite to ensure that none of these changes have inadvertently broken dependency expectations or introduced conflicts. Consider runninggo mod tidyand verifying that no unexpected version conflicts appear.internal/test/data/vector/noise/noise.go (1)
180-181: Eliminate repeated cloning of the noise table for large data sets.
Cloningng.noiseTableevery timeMod()is called can be expensive, especially for a large table.One small optimization is to store the table pointer in the
noiseGeneratorstruct for reuse:--- a/noise.go +++ b/noise.go @@ -180,8 +180,8 @@ func (ng *noiseGenerator) Mod() Func { - noiseTable := slices.Clone(ng.noiseTable) - tableSize := uint64(len(noiseTable)) + tableSize := uint64(len(ng.noiseTable)) return func(i uint64, vec []float32) (res []float32) { - res = slices.Clone(vec) + res = slices.Clone(vec) // preserve original vector n := uint64(len(res)) baseIdx := i * n for j := uint64(0); j < n; j++ {internal/errors/k8s.go (1)
43-45: Inconsistent capitalization in error messagesThe error messages for
ErrUndefinedNamespaceandErrUndefinedServicestart with an uppercase "Undefined" while all other error messages use lowercase.For consistency with other error messages in the codebase:
- ErrUndefinedNamespace = New("Undefined namespace") - ErrUndefinedService = New("Undefined service") + ErrUndefinedNamespace = New("undefined namespace") + ErrUndefinedService = New("undefined service")internal/iter/iter.go (5)
96-98: Add documentation to explain the Seq method's purpose.This method lacks documentation. Consider adding a comment to explain that it's a convenience method that delegates to
Values.+// Seq returns an iterator sequence that yields the values (without indexes). +// This is an alias for Values() for compatibility with the standard iterator pattern. func (c *cycle[_, E]) Seq(ctx context.Context) stdlib_iter.Seq[E] { return c.Values(ctx) }
117-130: Parameter receiver should be a pointer for consistency.The
Indexesmethod uses a value receiver (c cycle[_, _]) while other methods use pointer receivers. For consistency and to avoid unnecessary copying, consider using a pointer receiver here as well.-func (c cycle[_, _]) Indexes(ctx context.Context) stdlib_iter.Seq[uint64] { +func (c *cycle[_, _]) Indexes(ctx context.Context) stdlib_iter.Seq[uint64] {
132-143: Add documentation for ForEach method.The
ForEachmethod lacks documentation. Add a comment explaining its purpose and behavior.+// ForEach executes the provided function for each element in the cycle, +// stopping if the context is cancelled or if the function returns false. func (c cycle[_, E]) ForEach(ctx context.Context, fn func(uint64, E) bool) {Also, for consistency with other methods, consider using a pointer receiver:
-func (c cycle[_, E]) ForEach(ctx context.Context, fn func(uint64, E) bool) { +func (c *cycle[_, E]) ForEach(ctx context.Context, fn func(uint64, E) bool) {
145-147: Add documentation for Raw method.The
Rawmethod lacks documentation. Add a comment explaining its purpose.+// Raw returns the original slice that the cycle was created with. func (c cycle[S, _]) Raw() S {For consistency with other methods, consider using a pointer receiver:
-func (c cycle[S, _]) Raw() S { +func (c *cycle[S, _]) Raw() S {
149-152: Consistent pointer usage for method receivers.For consistency with other methods, consider using a pointer receiver for the
Lenmethod:-func (c cycle[_, _]) Len() uint64 { +func (c *cycle[_, _]) Len() uint64 {
🛑 Comments failed to post (7)
internal/config/config_test.go (1)
999-1003:
⚠️ Potential issueBehavior change in handling nonexistent files.
The test case was modified to expect an empty result when accessing a nonexistent file path instead of returning the file path itself. This represents a significant change in error handling behavior that could affect how the system manages missing configuration files.
Make sure this behavior change is intended and consistent with other parts of the codebase that may rely on the previous behavior. This might require additional testing to ensure that code consuming the
GetActualValuefunction properly handles the new empty return value for nonexistent files.pkg/index/job/deletion/service/options_test.go (1)
1-15:
⚠️ Potential issueTest file consists entirely of commented-out code.
This entire test file consists of commented-out code, which doesn't contribute to actual test coverage. Since this PR aims to improve the E2E testing framework's maintainability, it's important to have working tests rather than just templates.
Please uncomment and properly implement all the test functions with appropriate test cases, or remove this file if it's not ready for inclusion in the PR.
internal/timeutil/time_test.go (1)
183-349:
⚠️ Potential issueThe test structure is well-designed but needs implementation.
The commented-out test structure follows a good pattern with comprehensive check functions, parallel test execution, and goroutine leak detection. However, these tests need to be uncommented and filled with actual test cases to provide proper test coverage for the
DurationStringtype.According to past review comments, this issue was supposedly addressed in commit 662eee0, but the tests still appear to be commented out in this version. Please ensure these tests are fully implemented before merging.
apis/docs/v1/docs.md (1)
1636-1644:
⚠️ Potential issueVerify and Correct Method Naming in Filter Service
In the Filter service table, the "MultiSearchObject" row’s description refers to "StreamSearchObject RPC," while the "StreamSearchObject" row refers to "MultiSearchObject RPC." This naming mismatch is potentially confusing. Please verify the intended behavior and swap the descriptions if necessary so that each description accurately matches its method name.pkg/agent/core/ngt/service/ngt.go (1)
1475-1478: 💡 Verification agent
🧩 Analysis chain
Good defensive programming to prevent nil pointer dereference.
Adding this nil check is essential to prevent a potential panic when
statsis nil but no error was returned fromGetGraphStatistics. This properly logs a warning and returns gracefully instead of attempting to access fields of a nil pointer.However, static analysis shows this code path isn't covered by tests. Consider adding a test case that verifies the behavior when
statsis nil:
🏁 Script executed:
#!/bin/bash # Check for existing test coverage of the nil stats case rg -A 3 "Test.*loadStatistics" --type goLength of output: 300
Action Required: Add test coverage for nil stats case
The nil-check is a good defensive programming practice to prevent a nil pointer dereference. However, static analysis indicates that this code path isn’t actively covered by tests. A search in the test file (
pkg/agent/core/ngt/service/ngt_test.go) reveals that the intended test (Test_ngt_loadStatistics) appears to be commented out or incomplete.
- File:
pkg/agent/core/ngt/service/ngt.go(lines 1475-1478) – The nil check is correctly implemented.- Recommendation: Add a test case in
pkg/agent/core/ngt/service/ngt_test.goto simulate a scenario wherestatsis nil. The test should verify that a warning gets logged and the function returns nil gracefully..gitfiles (2)
1986-1990: 💡 Verification agent
🧩 Analysis chain
Verify QBG algorithm implementation follows Rust best practices.
The new QBG algorithm implementation in Rust should follow the same best practices for error handling as the rest of the codebase. Check for proper error handling patterns instead of using
unwrap().
🏁 Script executed:
#!/bin/bash # Check for unwrap() calls in the QBG implementation echo "Checking rust/libs/algorithms/qbg/src/lib.rs for unwrap() calls:" grep -n "unwrap()" rust/libs/algorithms/qbg/src/lib.rsLength of output: 1880
Attention: The QBG algorithm in Rust currently uses multiple
unwrap()calls that bypass proper error handling.
- In
rust/libs/algorithms/qbg/src/lib.rs, the output indicatesunwrap()is used on lines 379, 397, 448, 454, 457, 462, 463, 468, 473, 477, 498, 523, 528, 533, 537, 557, 622, 628, 631, 636, 637, 642, 647, 651, 657, 665, and 667.- Replace these occurrences with robust error handling (for example, using the
?operator or implementing proper error propagation) to align with Rust best practices followed elsewhere in the codebase.Please refactor accordingly to ensure that errors are handled gracefully without unexpected panics.
1274-1276: 💡 Verification agent
🧩 Analysis chain
Add "quic" to the project dictionary to avoid false CSpell warnings.
The CSpell spell checker is flagging "quic" as an unknown word. Since QUIC is a valid networking protocol name used in
internal/net/quic/conn_test.goandinternal/net/quic/listener_test.go, consider adding it to the project's dictionary in.cspell.json.
🏁 Script executed:
#!/bin/bash # Check if quic is already in the dictionary grep -q '"quic"' .cspell.json || echo "QUIC is not in the dictionary. It should be added."Length of output: 142
Action Required: Add "quic" to the project dictionary.
The verification confirmed that the word "quic" is not listed in the project's
.cspell.jsonfile, which causes false CSpell warnings in the QUIC-related files (internal/net/quic/conn_test.go,internal/net/quic/listener.go, andinternal/net/quic/listener_test.go). Please update.cspell.jsonto include"quic"in the allowed words. This change will suppress the false positives from the CSpell checker.
There was a problem hiding this comment.
Actionable comments posted: 5
🔭 Outside diff range comments (1)
pkg/index/job/deletion/service/deleter_test.go (1)
1-528: 💡 Verification agent🧩 Analysis chain
Important: This test file contains commented-out code and is not active.
The file sets up a comprehensive test framework for the deletion service, but all code is currently commented out with
//and marked with TODOs, making these tests non-functional. This appears to be a skeleton/template for future test implementation.To make this file functional, either:
- Uncomment the test code and implement the TODOs
- Add necessary imports at the top of the file:
import ( "context" "reflect" "testing" "github.com/vdaas/vald/internal/errors" "github.com/vdaas/vald/internal/net/grpc" "github.com/vdaas/vald/apis/grpc/v1/payload" "github.com/vdaas/vald/apis/grpc/v1/vald" "github.com/vdaas/vald/pkg/discoverer" "go.uber.org/goleak" )
- Create and implement a mock client as suggested in previous reviews:
type mockClient struct { mockStartClient func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) mockClose func() error } func (m *mockClient) StartClient(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { if m.mockStartClient != nil { return m.mockStartClient(ctx, addr, targetIndexID, opts...) } return nil, errors.New("mock not implemented") } func (m *mockClient) Close() error { if m.mockClose != nil { return m.mockClose() } return nil }
🏁 Script executed:
#!/bin/bash # Check if there are other test files that might contain the actual implementation # of these tests that have been moved or refactored echo "Checking for related test implementations in the codebase..." rg -t go "TestNew.*pkg/index/job/deletion/service" --stats rg -t go "Test_delDuplicateAddrs" --stats rg -t go "Test_index_StartClient" --stats rg -t go "Test_index_Start" --stats rg -t go "Test_index_doDeleteIndex" --stats # Check if there are any mock client implementations elsewhere echo "Checking for mock client implementations..." rg -t go "type mock.*Client struct" --statsLength of output: 2750
Attention: Activate the Deletion Service Test Suite
- The file
pkg/index/job/deletion/service/deleter_test.gocurrently contains only commented-out code and multiple TODOs, meaning that these tests are effectively non-functional.- Action Items:
- Uncomment & Implement: Either uncomment the existing test skeleton and implement the missing test cases, or remove the file if redundant with working tests elsewhere.
- Import Updates: Ensure that all necessary imports are included at the top of the file. For example:
import ( "context" "reflect" "testing" "github.com/vdaas/vald/internal/errors" "github.com/vdaas/vald/internal/net/grpc" "github.com/vdaas/vald/apis/grpc/v1/payload" "github.com/vdaas/vald/apis/grpc/v1/vald" "github.com/vdaas/vald/pkg/discoverer" "go.uber.org/goleak" )- Implement Mocks: As no mock client implementations were found in the codebase (per the verification), consider adding a mock client similar to:
type mockClient struct { mockStartClient func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) mockClose func() error } func (m *mockClient) StartClient(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { if m.mockStartClient != nil { return m.mockStartClient(ctx, addr, targetIndexID, opts...) } return nil, errors.New("mock not implemented") } func (m *mockClient) Close() error { if m.mockClose != nil { return m.mockClose() } return nil }- Review Related Tests: Validate against similar test implementations in other packages (e.g., in
pkg/index/job/save/service/indexer_test.goandpkg/index/job/creation/service/indexer_test.go) to ensure consistency in testing practices.
♻️ Duplicate comments (8)
apis/swagger/v1/vald/update.swagger.json (1)
83-83:⚠️ Potential issueIncomplete Documentation for
/update/timestampEndpoint.
Thesummarystill contains "TODO" placeholders for the status code and troubleshooting sections. To maintain consistency with the other endpoints and ensure comprehensive guidance, please complete these sections with the appropriate details.internal/test/mock/server_stream.go (1)
40-52:⚠️ Potential issueFix variable shadowing in Recv method.
There's a variable shadowing issue on line 46 where
err :=shadows the named return parametererr. This could lead to unexpected behavior since the shadowed variable is local to the if-block.Apply this fix:
func (m *StreamInsertServerMock) Recv() (res *payload.Insert_Request, err error) { if m != nil { if m.RecvFunc != nil { return m.RecvFunc() } res = new(payload.Insert_Request) - err := m.ServerStream.RecvMsg(res) + err = m.ServerStream.RecvMsg(res) if err != nil { return nil, err } return res, nil } return nil, nil }The improved error handling approach is good, but like the
Sendmethod, these code paths need test coverage to verify their behavior, particularly:
- When
mis nil- When
RecvFuncis defined- When falling back to
ServerStream.RecvMsg- Error handling when
RecvMsgfails🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by testsinternal/params/option.go (1)
27-28:⚠️ Potential issueUnsafe access to os.Args[0] in package initialization
Using
filepath.Base(os.Args[0])directly in package-level variables is risky asos.Argsmight be empty in certain contexts (tests, specific environments).Consider one of these safer alternatives:
- Create a helper function with a fallback:
-var defaultOptions = []Option{ - WithName(filepath.Base(os.Args[0])), +func getDefaultName() string { + if len(os.Args) > 0 { + return filepath.Base(os.Args[0]) + } + return "vald-app" // or another appropriate fallback +} + +var defaultOptions = []Option{ + WithName(getDefaultName()),
- Initialize name at runtime instead of compile-time
internal/net/quic/listener_test.go (1)
14-15: 🛠️ Refactor suggestionMissing necessary imports for test implementation
The package declaration is present, but when uncommented, this test file will require imports for several packages used in the test code:
package quic +import ( + "context" + "net" + "reflect" + "testing" + "crypto/tls" + + "github.com/cockroachdb/errors" + "go.uber.org/goleak" + "github.com/quic-go/quic-go" +)internal/test/data/vector/noise/noise.go (2)
79-106:⚠️ Potential issuePrevent division by zero when computing average standard deviation.
If all vectors are empty (so
len(data)is non-zero, butlen(vec)is zero in each iteration),countremains 0, leading to division by zero at line 105. Provide a fallback ifcountis 0 to avoid runtime errors.Consider this diff:
--- a/noise.go +++ b/noise.go @@ -103,6 +103,11 @@ func() float32 { } return float32(totalStd/float64(count)) * ng.noiseLevelFactor }() +if count == 0 { + // Fallback for edge case when all vectors have length 0 + noiseLevel = 0.01 + log.Warnf("No valid vectors found. Falling back to default noise level 0.01") +}
108-123:⚠️ Potential issueAvoid underflow with
bits.Len64(required-1).When
requiredis 0 or 1, subtracting 1 can produce an invalid shift parameter. Confirmrequired >= 2or enforce a minimum value.For instance:
--- a/noise.go +++ b/noise.go @@ -117,6 +117,10 @@ func() int { required /= ng.noiseTableDivisionFactor // Ensure the noise table size is at least minNoiseTableSize. if required < ng.minNoiseTableSize { + if ng.minNoiseTableSize < 2 { + ng.minNoiseTableSize = 2 + } required = ng.minNoiseTableSize } return 1 << bits.Len64(required-1)internal/iter/iter.go (1)
51-67: 🛠️ Refactor suggestionReturn a no-op cycle instead of
nil, and add test coverage.Currently, the function returns
nilwhen the array is empty ornil, forcing callers to handlenilchecks. Consider returning a valid no-op cycle to eliminate the risk ofnilpointer dereferences and promote safer usage. Additionally, lines 52-67 lack test coverage per static analysis; ensure that boundary cases (empty slice, non-empty slice, largeoffset) are tested.Here's an example of a no-op cycle approach:
func NewCycle[S ~[]E, E any](array S, num, offset uint64, mod func(uint64, E) E) Cycle[S, E] { if array == nil { - return nil + // return a no-op cycle + return &cycle[S, E]{array: make(S, 0), num: 0, size: 0} } size := uint64(len(array)) if size == 0 { - return nil + // return a no-op cycle + return &cycle[S, E]{array: array, num: 0, size: 0} } ... }🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by testsinternal/servers/server/option_test.go (1)
3727-4066:⚠️ Potential issueImplement the commented-out gRPC option test functions
These test functions for gRPC options (TestWithGRPCMaxConcurrentStreams, TestWithGRPCNumStreamWorkers, TestWithGRPCSharedWriteBuffer, TestWithGRPCWaitForHandlers) are still commented out despite previous review comments indicating they should be implemented.
These tests are important for ensuring proper test coverage of the gRPC option features that are part of the new E2E strategic testing framework.
Either:
- Implement these tests by uncommenting and completing the test cases, or
- Add explicit TODO comments with ticket references if they're planned for future implementation
🧹 Nitpick comments (24)
apis/swagger/v1/vald/remove.swagger.json (2)
17-18: Extended Summary with Detailed Error Codes
The updated summary now embeds a detailed table of status codes and troubleshooting instructions. While this enriches the documentation, consider whether all this detail might be better suited to the description field so that the summary remains concise.
83-84: Detailed Information for RemoveByTimestamp Endpoint
The updated summary and description for the/remove/timestampendpoint include comprehensive details on how timestamp-based removals work and potential troubleshooting steps. Ensure that the formatting (with HTML tags and markdown tables) is compatible with your API documentation tooling.internal/core/algorithm/usearch/usearch_test.go (1)
670-670: Synchronization design change in commented test code.The change from
mu:sync.RWMutex{}tomu:nilin these commented-out test cases indicates a shift in the synchronization approach. When these tests are eventually implemented, ensure this aligns with how thread safety is handled in the actualusearchimplementation.Consider documenting the reason for this change with a comment to explain the synchronization strategy being adopted. This would help future developers understand the design decision when implementing these tests.
Also applies to: 698-698, 801-801, 832-832, 932-932, 960-960, 1065-1065, 1097-1097, 1200-1200, 1231-1231, 1340-1340, 1372-1372, 1475-1475, 1506-1506, 1602-1602, 1630-1630
apis/swagger/v1/vald/upsert.swagger.json (1)
17-18: Enhanced Summary and Troubleshooting for the Upsert Endpoint
The updatedsummaryclearly explains the purpose of the Upsert RPC and includes a detailed table of status codes along with a troubleshooting section. This additional context improves usability and helps developers quickly understand expected responses and potential issues. Consider adding header rows to the table (if supported by your UI) for even greater clarity.apis/swagger/v1/vald/filter.swagger.json (1)
177-178: Additional Notice for Multi Update Object EndpointThe newly added description warns about gRPC message size limitations. This notice is useful for developers; however, consider whether linking to further documentation or providing guidance on mitigation strategies might further enhance clarity.
apis/swagger/v1/vald/insert.swagger.json (1)
50-51: Enhanced Documentation for/insert/multipleEndpoint
The new summary clearly describes the multi-insert operation, and the accompanying description adds valuable troubleshooting guidance—including an HTML notice about gRPC message size limitations and a status code table. Confirm that the embedded HTML renders as expected in your API documentation.internal/params/params.go (1)
139-149: Consider documenting concurrency implications of flag.CommandLine modificationsThe
Restore()andOverride()methods modify the globalflag.CommandLinevariable, which could cause issues if multiple parsers are used simultaneously or in concurrent scenarios.Consider adding documentation comments to warn users about potential issues:
+// Restore resets the global flag.CommandLine to the original default. +// Note: This modifies a global variable and should be used with caution in concurrent environments. func (p *parser) Restore() { if p.defaults != nil { flag.CommandLine = p.defaults } } +// Override replaces the global flag.CommandLine with this parser's flag set. +// Note: This modifies a global variable and should be used with caution in concurrent environments. func (p *parser) Override() { if p.f != nil { flag.CommandLine = p.f } }apis/docs/v1/docs.md (9)
1372-1377: Refine Mirror Service Overview Documentation
The new Mirror section now includes an "Overview" header and a table describing the Register RPC. Please verify that the method’s purpose and the listed status codes are accurate and that the formatting is consistent with similar sections.
1676-1682: Review Flush Service Overview
The addition of the "Overview" for the Flush service clarifies its responsibility for removing vectors. Please review the accompanying table to ensure that the status codes and troubleshooting details match the service’s behavior.
1691-1694: Enhance Index Service Documentation
The inclusion of an “Overview” header for the Index service improves clarity. Consider expanding the description slightly to detail its role in managing index details, ensuring consistency with other service documentation.
1712-1720: Correct Typographical Error in Insert Service Overview
The Insert service overview now clearly defines its responsibility; however, note that “Inset RPC” should be corrected to “Insert RPC” to avoid confusion. Please also confirm that the following table details accurately reflect the intended functionality.
1774-1790: Verify Remove Service Overview Consistency
The Remove service section now presents an overview of its function to remove indexed vectors. Ensure that the table entries and troubleshooting guidelines are accurate and consistent with established documentation standards.
1818-1838: Consolidate Search Service Documentation
The Search service section has been extensively updated with an overview, method details, and error troubleshooting. Please verify that each method’s description (e.g., Search, SearchByID, StreamSearch) is uniformly presented and that the status codes are correctly documented across the board.
1840-1895: Clarify Linear Search Service Descriptions
The documentation for LinearSearch and LinearSearchByID (including streaming variants) is detailed but dense. Consider reviewing the troubleshooting sections for completeness and ensuring consistency in error message formatting with the rest of the Search documentation.
1896-1904: Review Update Service Overview and Error Handling
The Update service section now includes an “Overview” and detailed method descriptions. Please confirm that the error codes and troubleshooting recommendations accurately reflect the current service behavior, and that the formatting is consistent with similar sections.
1934-1946: Verify Upsert Service Documentation Accuracy
The Upsert service overview now describes its dual role in updating existing vectors or inserting new ones. Please double-check that the table reflects accurate status codes and error conditions. Additionally, note any typographical errors (e.g., “mthe ultiple” should be corrected to “multiple”) in the related sections.internal/iter/iter.go (1)
72-77: Add unit tests to cover boundary conditions.The
Atmethod’s lines (72–77) remain uncovered by tests. Verify its behavior with various indexes (including out-of-range if relevant) and ensure your tests confirm that themodFuncis applied correctly.Shall I generate a basic test file for this function to help improve coverage?
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by testspkg/agent/core/faiss/service/faiss.go (1)
1246-1248: Improve error handling logic with double negation.The current implementation has a double negative (
!errors.IsNot) which makes the code harder to understand at first glance. Consider refactoring for clarity.- if !errors.IsNot(err, context.Canceled, context.DeadlineExceeded) { - err = nil - } + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + err = nil + }🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by testsinternal/net/grpc/stream.go (1)
80-98: Improved error handling with dedicated finalize functionThe new
finalizefunction encapsulates error aggregation and cleanup logic:
- Waits for all goroutines to complete
- Collects and deduplicates errors
- Properly formats gRPC status information
- Sets trace attributes for observability
This refactoring improves code organization and ensures consistent cleanup behavior.
Consider adding error context with the request count or other operation metrics in the error details to aid debugging.
internal/net/grpc/pool/pool.go (4)
34-35: Consider grouping gRPC imports.These imports from
internal/net/grpc/codesandinternal/net/grpc/statusare logically related to gRPC error handling. For consistency, you could group them together with the standardgrpcimport block.import ( "context" "fmt" "strconv" "sync/atomic" "time" "github.com/vdaas/vald/internal/backoff" "github.com/vdaas/vald/internal/errors" "github.com/vdaas/vald/internal/log" "github.com/vdaas/vald/internal/net" - "github.com/vdaas/vald/internal/net/grpc/codes" - "github.com/vdaas/vald/internal/net/grpc/status" "github.com/vdaas/vald/internal/strings" "github.com/vdaas/vald/internal/sync" "github.com/vdaas/vald/internal/sync/errgroup" "google.golang.org/grpc" "google.golang.org/grpc/connectivity" + + "github.com/vdaas/vald/internal/net/grpc/codes" + "github.com/vdaas/vald/internal/net/grpc/status" )
410-434: Potential concurrency mismatch with multiple IPs and pool size.When connecting to multiple IPs, if the number of IPs exceeds the pool size, you call
p.grow(...)first. This is correct for coverage, but be mindful that if the DNS returns more IPs on subsequent reconnections, the pool might keep growing. Ensure you have checks or limits to avoid unbounded growth in transient or round-robin DNS setups.
661-696: Reconsider 10ms TCP dial for IP “ping”.Although a quick dial attempt can serve as a functional “ping,” 10ms may be too strict in some slow or high-latency networks, potentially discarding viable IP addresses. If your environment might include such latencies, consider a slightly higher threshold to reduce false negatives.
- pingCtx, cancel := context.WithTimeout(ctx, 10*time.Millisecond) + pingCtx, cancel := context.WithTimeout(ctx, 50*time.Millisecond)
606-633: Efficiently tracking healthy connections with sync.Map.Using the global
metricsmap to store and update healthy connection counts per address is concise. Confirm that external components rely on stable keys and that no stale entries remain after an address changes (e.g., dynamic DNS). You might want to periodically prune or refresh outdated entries if addresses rotate often.internal/net/grpc/option_test.go (1)
532-3344: Overall test structure could be improvedThe test file contains significant code duplication across all test functions. Each test follows exactly the same pattern with similar boilerplate code.
Consider refactoring the tests to reduce duplication. Options include:
- A shared test generator function that creates test cases based on parameters
- Table-driven tests for similar option types (e.g., size-related options)
- Helper functions for common assertions and setup
// Example of a shared test generator approach: + func testOptionFunc(t *testing.T, name string, optionFunc func(args interface{}) Option, + argType interface{}, testValues []interface{}) { + for _, val := range testValues { + t.Run(fmt.Sprintf("%s_%v", name, val), func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + + got := optionFunc(val) + // Perform assertions + }) + } + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
apis/grpc/v1/agent/core/agent.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/agent/sidecar/sidecar.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/discoverer/discoverer.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/egress/egress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/ingress/ingress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/meta/meta.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/payload/payload.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/rpc/errdetails/error_details.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goexample/client/go.sumis excluded by!**/*.sumgo.sumis excluded by!**/*.sumrust/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (186)
.gitfiles(15 hunks).github/ISSUE_TEMPLATE/bug_report.md(1 hunks).github/ISSUE_TEMPLATE/security_issue_report.md(1 hunks).github/PULL_REQUEST_TEMPLATE.md(1 hunks).github/workflows/_docker-image.yaml(1 hunks).github/workflows/_release-pr.yaml(1 hunks).github/workflows/build-binaries.yaml(1 hunks).github/workflows/build-protobuf.yaml(1 hunks).github/workflows/chatops.yaml(5 hunks).github/workflows/codeql-analysis.yml(2 hunks).github/workflows/coverage.yaml(1 hunks).github/workflows/e2e-chaos.yaml(4 hunks).github/workflows/e2e-code-bench-agent.yaml(2 hunks).github/workflows/e2e-max-dim.yaml(1 hunks).github/workflows/e2e-profiling.yaml(1 hunks).github/workflows/e2e.yaml(8 hunks).github/workflows/format.yaml(3 hunks).github/workflows/fossa.yaml(1 hunks).github/workflows/helm-lint.yaml(3 hunks).github/workflows/helm.yaml(2 hunks).github/workflows/reviewdog-k8s.yaml(3 hunks).github/workflows/reviewdog.yaml(2 hunks).github/workflows/test-hack.yaml(1 hunks).github/workflows/unit-test-rust.yaml(1 hunks).github/workflows/unit-test.yaml(3 hunks).github/workflows/update-deps.yaml(1 hunks)Makefile(8 hunks)Makefile.d/e2e.mk(1 hunks)Makefile.d/functions.mk(1 hunks)Makefile.d/minikube.mk(1 hunks)Makefile.d/tools.mk(2 hunks)apis/docs/v1/docs.md(10 hunks)apis/swagger/v1/mirror/mirror.swagger.json(1 hunks)apis/swagger/v1/vald/filter.swagger.json(8 hunks)apis/swagger/v1/vald/flush.swagger.json(1 hunks)apis/swagger/v1/vald/index.swagger.json(5 hunks)apis/swagger/v1/vald/insert.swagger.json(2 hunks)apis/swagger/v1/vald/object.swagger.json(4 hunks)apis/swagger/v1/vald/remove.swagger.json(3 hunks)apis/swagger/v1/vald/search.swagger.json(8 hunks)apis/swagger/v1/vald/update.swagger.json(3 hunks)apis/swagger/v1/vald/upsert.swagger.json(2 hunks)charts/vald/values/dev.yaml(1 hunks)cmd/index/job/deletion/main_test.go(1 hunks)dockers/agent/core/agent/Dockerfile(1 hunks)dockers/agent/core/faiss/Dockerfile(1 hunks)dockers/agent/core/ngt/Dockerfile(1 hunks)dockers/agent/sidecar/Dockerfile(1 hunks)dockers/binfmt/Dockerfile(1 hunks)dockers/buildbase/Dockerfile(1 hunks)dockers/buildkit/Dockerfile(1 hunks)dockers/buildkit/syft/scanner/Dockerfile(1 hunks)dockers/ci/base/Dockerfile(1 hunks)dockers/dev/Dockerfile(1 hunks)dockers/discoverer/k8s/Dockerfile(1 hunks)dockers/example/client/Dockerfile(1 hunks)dockers/gateway/filter/Dockerfile(1 hunks)dockers/gateway/lb/Dockerfile(1 hunks)dockers/gateway/mirror/Dockerfile(1 hunks)dockers/index/job/correction/Dockerfile(1 hunks)dockers/index/job/creation/Dockerfile(1 hunks)dockers/index/job/deletion/Dockerfile(1 hunks)dockers/index/job/readreplica/rotate/Dockerfile(1 hunks)dockers/index/job/save/Dockerfile(1 hunks)dockers/index/operator/Dockerfile(1 hunks)dockers/manager/index/Dockerfile(1 hunks)dockers/operator/helm/Dockerfile(1 hunks)dockers/tools/benchmark/job/Dockerfile(1 hunks)dockers/tools/benchmark/operator/Dockerfile(1 hunks)dockers/tools/cli/loadtest/Dockerfile(1 hunks)example/client/go.mod(2 hunks)example/client/go.mod.default(1 hunks)go.mod(13 hunks)hack/go.mod.default(4 hunks)internal/backoff/backoff.go(2 hunks)internal/backoff/backoff_test.go(1 hunks)internal/backoff/option.go(0 hunks)internal/backoff/option_test.go(0 hunks)internal/client/v1/client/discoverer/discover.go(1 hunks)internal/client/v1/client/discoverer/discover_test.go(5 hunks)internal/config/benchmark_test.go(4 hunks)internal/config/config.go(2 hunks)internal/config/config_test.go(1 hunks)internal/config/grpc.go(1 hunks)internal/config/index_deleter_test.go(1 hunks)internal/core/algorithm/usearch/usearch_test.go(16 hunks)internal/db/nosql/cassandra/option_test.go(8 hunks)internal/errors/errors.go(2 hunks)internal/errors/errors_test.go(1 hunks)internal/errors/k8s.go(1 hunks)internal/errors/params.go(1 hunks)internal/file/file.go(3 hunks)internal/file/file_test.go(1 hunks)internal/info/info_test.go(17 hunks)internal/iter/iter.go(1 hunks)internal/net/dialer_test.go(1 hunks)internal/net/grpc/client.go(13 hunks)internal/net/grpc/errdetails/errdetails.go(6 hunks)internal/net/grpc/metadata_test.go(1 hunks)internal/net/grpc/option.go(1 hunks)internal/net/grpc/option_test.go(46 hunks)internal/net/grpc/pool/option.go(1 hunks)internal/net/grpc/pool/option_test.go(6 hunks)internal/net/grpc/pool/pool.go(6 hunks)internal/net/grpc/pool/pool_bench_test.go(2 hunks)internal/net/grpc/server_test.go(2 hunks)internal/net/grpc/status/status.go(1 hunks)internal/net/grpc/status/status_test.go(1 hunks)internal/net/grpc/stream.go(4 hunks)internal/net/grpc/stream_test.go(4 hunks)internal/net/http/client/client.go(2 hunks)internal/net/http/client/client_test.go(3 hunks)internal/net/http/client/option_test.go(4 hunks)internal/net/net.go(1 hunks)internal/net/net_test.go(3 hunks)internal/net/quic/conn_test.go(1 hunks)internal/net/quic/listener_test.go(1 hunks)internal/params/option.go(2 hunks)internal/params/option_test.go(1 hunks)internal/params/params.go(4 hunks)internal/params/params_test.go(12 hunks)internal/rand/rand.go(1 hunks)internal/runner/runner.go(4 hunks)internal/servers/server/option_test.go(1 hunks)internal/servers/server/server.go(1 hunks)internal/servers/servers.go(3 hunks)internal/sync/errgroup/group.go(7 hunks)internal/sync/errgroup/group_test.go(16 hunks)internal/test/data/vector/gen.go(0 hunks)internal/test/data/vector/noise/noise.go(1 hunks)internal/test/data/vector/noise/noise_test.go(1 hunks)internal/test/data/vector/noise/option.go(1 hunks)internal/test/data/vector/noise/option_test.go(1 hunks)internal/test/mock/server_stream.go(1 hunks)internal/timeutil/time.go(3 hunks)internal/timeutil/time_test.go(1 hunks)k8s/agent/clusterrole.yaml(1 hunks)k8s/agent/clusterrolebinding.yaml(1 hunks)k8s/agent/daemonset.yaml(1 hunks)k8s/agent/deployment.yaml(1 hunks)k8s/agent/faiss/configmap.yaml(1 hunks)k8s/agent/hpa.yaml(1 hunks)k8s/agent/networkpolicy.yaml(1 hunks)k8s/agent/serviceaccount.yaml(1 hunks)k8s/agent/sidecar/configmap.yaml(1 hunks)k8s/agent/sidecar/svc.yaml(1 hunks)k8s/discoverer/daemonset.yaml(1 hunks)k8s/discoverer/deployment.yaml(1 hunks)k8s/discoverer/hpa.yaml(1 hunks)k8s/discoverer/networkpolicy.yaml(1 hunks)k8s/gateway/gateway/filter/configmap.yaml(1 hunks)k8s/gateway/gateway/filter/daemonset.yaml(1 hunks)k8s/gateway/gateway/filter/deployment.yaml(1 hunks)k8s/gateway/gateway/filter/hpa.yaml(1 hunks)k8s/gateway/gateway/filter/networkpolicy.yaml(1 hunks)k8s/gateway/gateway/filter/pdb.yaml(1 hunks)k8s/gateway/gateway/filter/priorityclass.yaml(1 hunks)k8s/gateway/gateway/filter/svc.yaml(1 hunks)k8s/gateway/gateway/lb/daemonset.yaml(1 hunks)k8s/gateway/gateway/lb/deployment.yaml(1 hunks)k8s/gateway/gateway/lb/networkpolicy.yaml(1 hunks)k8s/gateway/gateway/mirror/daemonset.yaml(1 hunks)k8s/gateway/gateway/mirror/deployment.yaml(1 hunks)k8s/gateway/gateway/mirror/networkpolicy.yaml(1 hunks)k8s/index/job/correction/networkpolicy.yaml(1 hunks)k8s/index/job/creation/networkpolicy.yaml(1 hunks)k8s/index/job/readreplica/rotate/clusterrole.yaml(1 hunks)k8s/index/job/readreplica/rotate/clusterrolebinding.yaml(1 hunks)k8s/index/job/readreplica/rotate/configmap.yaml(1 hunks)k8s/index/job/readreplica/rotate/networkpolicy.yaml(1 hunks)k8s/index/job/readreplica/rotate/serviceaccount.yaml(1 hunks)k8s/index/job/save/networkpolicy.yaml(1 hunks)k8s/index/operator/deployment.yaml(1 hunks)k8s/manager/index/daemonset.yaml(1 hunks)k8s/manager/index/deployment.yaml(1 hunks)k8s/manager/index/networkpolicy.yaml(1 hunks)pkg/agent/core/faiss/service/faiss.go(3 hunks)pkg/agent/core/ngt/service/ngt.go(5 hunks)pkg/agent/core/ngt/service/ngt_test.go(8 hunks)pkg/agent/core/ngt/service/option_test.go(8 hunks)pkg/discoverer/k8s/service/discover_test.go(4 hunks)pkg/gateway/lb/handler/grpc/aggregation_test.go(2 hunks)pkg/gateway/lb/handler/grpc/handler_test.go(2 hunks)pkg/gateway/mirror/service/mirror.go(2 hunks)pkg/index/job/deletion/config/config_test.go(1 hunks)pkg/index/job/deletion/service/deleter_test.go(1 hunks)
💤 Files with no reviewable changes (3)
- internal/backoff/option.go
- internal/backoff/option_test.go
- internal/test/data/vector/gen.go
✅ Files skipped from review due to trivial changes (35)
- k8s/discoverer/hpa.yaml
- k8s/index/job/correction/networkpolicy.yaml
- k8s/discoverer/networkpolicy.yaml
- k8s/gateway/gateway/filter/pdb.yaml
- k8s/gateway/gateway/filter/daemonset.yaml
- k8s/agent/hpa.yaml
- k8s/gateway/gateway/filter/networkpolicy.yaml
- k8s/gateway/gateway/lb/networkpolicy.yaml
- k8s/manager/index/networkpolicy.yaml
- k8s/index/job/readreplica/rotate/configmap.yaml
- k8s/gateway/gateway/filter/configmap.yaml
- k8s/manager/index/daemonset.yaml
- k8s/gateway/gateway/mirror/daemonset.yaml
- k8s/agent/daemonset.yaml
- k8s/index/job/creation/networkpolicy.yaml
- k8s/gateway/gateway/filter/hpa.yaml
- k8s/gateway/gateway/filter/svc.yaml
- k8s/discoverer/daemonset.yaml
- k8s/gateway/gateway/filter/deployment.yaml
- k8s/agent/clusterrole.yaml
- k8s/agent/sidecar/svc.yaml
- k8s/agent/deployment.yaml
- k8s/index/job/save/networkpolicy.yaml
- k8s/agent/clusterrolebinding.yaml
- k8s/index/job/readreplica/rotate/serviceaccount.yaml
- k8s/gateway/gateway/mirror/networkpolicy.yaml
- k8s/agent/faiss/configmap.yaml
- k8s/index/job/readreplica/rotate/clusterrole.yaml
- k8s/gateway/gateway/filter/priorityclass.yaml
- k8s/agent/serviceaccount.yaml
- k8s/agent/sidecar/configmap.yaml
- k8s/index/job/readreplica/rotate/clusterrolebinding.yaml
- k8s/agent/networkpolicy.yaml
- k8s/gateway/gateway/lb/daemonset.yaml
- k8s/index/job/readreplica/rotate/networkpolicy.yaml
🚧 Files skipped from review as they are similar to previous changes (103)
- dockers/buildbase/Dockerfile
- dockers/index/job/creation/Dockerfile
- dockers/buildkit/Dockerfile
- dockers/operator/helm/Dockerfile
- dockers/gateway/filter/Dockerfile
- dockers/index/operator/Dockerfile
- dockers/dev/Dockerfile
- .github/workflows/e2e-chaos.yaml
- dockers/index/job/correction/Dockerfile
- .github/workflows/e2e-profiling.yaml
- .github/workflows/test-hack.yaml
- .github/workflows/helm.yaml
- dockers/gateway/lb/Dockerfile
- internal/backoff/backoff_test.go
- internal/errors/params.go
- Makefile.d/tools.mk
- .github/workflows/e2e-max-dim.yaml
- .github/workflows/helm-lint.yaml
- Makefile.d/e2e.mk
- .github/workflows/_docker-image.yaml
- dockers/binfmt/Dockerfile
- .github/workflows/coverage.yaml
- dockers/agent/sidecar/Dockerfile
- .github/workflows/fossa.yaml
- dockers/tools/cli/loadtest/Dockerfile
- dockers/discoverer/k8s/Dockerfile
- dockers/index/job/save/Dockerfile
- internal/net/net.go
- dockers/agent/core/faiss/Dockerfile
- dockers/tools/benchmark/operator/Dockerfile
- .github/workflows/build-protobuf.yaml
- .github/workflows/unit-test.yaml
- .github/workflows/unit-test-rust.yaml
- dockers/agent/core/ngt/Dockerfile
- internal/net/grpc/option.go
- internal/config/benchmark_test.go
- .github/workflows/update-deps.yaml
- k8s/index/operator/deployment.yaml
- internal/net/grpc/pool/pool_bench_test.go
- Makefile.d/minikube.mk
- k8s/discoverer/deployment.yaml
- example/client/go.mod.default
- dockers/gateway/mirror/Dockerfile
- dockers/example/client/Dockerfile
- .github/ISSUE_TEMPLATE/security_issue_report.md
- dockers/agent/core/agent/Dockerfile
- dockers/manager/index/Dockerfile
- charts/vald/values/dev.yaml
- internal/net/grpc/status/status.go
- dockers/index/job/deletion/Dockerfile
- dockers/ci/base/Dockerfile
- internal/net/grpc/errdetails/errdetails.go
- internal/net/http/client/client_test.go
- cmd/index/job/deletion/main_test.go
- .github/workflows/build-binaries.yaml
- .github/workflows/_release-pr.yaml
- dockers/tools/benchmark/job/Dockerfile
- internal/backoff/backoff.go
- .github/workflows/e2e-code-bench-agent.yaml
- k8s/manager/index/deployment.yaml
- internal/net/http/client/option_test.go
- .github/workflows/reviewdog-k8s.yaml
- .github/PULL_REQUEST_TEMPLATE.md
- internal/test/data/vector/noise/noise_test.go
- internal/timeutil/time.go
- internal/net/grpc/metadata_test.go
- internal/net/net_test.go
- k8s/gateway/gateway/mirror/deployment.yaml
- internal/net/http/client/client.go
- pkg/index/job/deletion/config/config_test.go
- .github/workflows/reviewdog.yaml
- .github/workflows/format.yaml
- internal/errors/errors_test.go
- internal/db/nosql/cassandra/option_test.go
- pkg/agent/core/ngt/service/option_test.go
- internal/net/dialer_test.go
- k8s/gateway/gateway/lb/deployment.yaml
- apis/swagger/v1/mirror/mirror.swagger.json
- internal/rand/rand.go
- pkg/agent/core/ngt/service/ngt_test.go
- internal/client/v1/client/discoverer/discover_test.go
- internal/config/grpc.go
- dockers/buildkit/syft/scanner/Dockerfile
- internal/info/info_test.go
- internal/config/index_deleter_test.go
- dockers/index/job/readreplica/rotate/Dockerfile
- apis/swagger/v1/vald/object.swagger.json
- internal/net/grpc/server_test.go
- pkg/discoverer/k8s/service/discover_test.go
- apis/swagger/v1/vald/index.swagger.json
- .github/workflows/chatops.yaml
- internal/net/grpc/pool/option_test.go
- .github/ISSUE_TEMPLATE/bug_report.md
- internal/net/grpc/pool/option.go
- internal/test/data/vector/noise/option.go
- pkg/gateway/lb/handler/grpc/handler_test.go
- internal/config/config.go
- internal/sync/errgroup/group.go
- internal/params/params_test.go
- example/client/go.mod
- hack/go.mod.default
- internal/net/grpc/client.go
- Makefile
🧰 Additional context used
🧠 Learnings (1)
.gitfiles (1)
Learnt from: datelier
PR: vdaas/vald#2807
File: rust/bin/agent/src/handler/object.rs:36-38
Timestamp: 2025-03-28T08:07:14.230Z
Learning: In Rust code for Vald handlers, avoid using unwrap() on Option types. Instead, handle None values explicitly by returning appropriate error statuses to maintain robustness.
🧬 Code Definitions (4)
internal/servers/server/server.go (2)
internal/net/grpc/server.go (1)
ErrServerStopped(44-44)internal/file/file.go (1)
Join(555-565)
internal/net/grpc/stream.go (1)
internal/net/grpc/status/status.go (5)
Is(48-50)Errorf(129-131)ParseError(133-183)Code(41-41)Error(125-127)
internal/net/grpc/pool/pool.go (4)
internal/errors/errors.go (2)
Is(163-168)IsNot(182-192)internal/net/grpc/status/status.go (1)
Is(48-50)internal/net/grpc/codes/codes.go (2)
Canceled(26-26)DeadlineExceeded(29-29)internal/net/grpc/option.go (1)
Option(41-41)
pkg/agent/core/ngt/service/ngt.go (2)
internal/errors/errors.go (2)
IsNot(182-192)Join(254-319)internal/errors/agent.go (1)
ErrUncommittedIndexNotFound(98-98)
🪛 GitHub Check: codecov/patch
pkg/agent/core/faiss/service/faiss.go
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by tests
[warning] 1250-1250: pkg/agent/core/faiss/service/faiss.go#L1250
Added line #L1250 was not covered by tests
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by tests
[warning] 1268-1268: pkg/agent/core/faiss/service/faiss.go#L1268
Added line #L1268 was not covered by tests
internal/iter/iter.go
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests
[warning] 81-87: internal/iter/iter.go#L81-L87
Added lines #L81 - L87 were not covered by tests
[warning] 89-91: internal/iter/iter.go#L89-L91
Added lines #L89 - L91 were not covered by tests
[warning] 96-97: internal/iter/iter.go#L96-L97
Added lines #L96 - L97 were not covered by tests
[warning] 101-107: internal/iter/iter.go#L101-L107
Added lines #L101 - L107 were not covered by tests
[warning] 109-111: internal/iter/iter.go#L109-L111
Added lines #L109 - L111 were not covered by tests
[warning] 117-123: internal/iter/iter.go#L117-L123
Added lines #L117 - L123 were not covered by tests
[warning] 125-127: internal/iter/iter.go#L125-L127
Added lines #L125 - L127 were not covered by tests
[warning] 132-137: internal/iter/iter.go#L132-L137
Added lines #L132 - L137 were not covered by tests
[warning] 139-141: internal/iter/iter.go#L139-L141
Added lines #L139 - L141 were not covered by tests
[warning] 145-146: internal/iter/iter.go#L145-L146
Added lines #L145 - L146 were not covered by tests
[warning] 150-151: internal/iter/iter.go#L150-L151
Added lines #L150 - L151 were not covered by tests
internal/test/mock/server_stream.go
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by tests
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by tests
pkg/agent/core/ngt/service/ngt.go
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by tests
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
⏰ Context from checks skipped due to timeout of 90000ms (79)
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: build / build
🔇 Additional comments (181)
internal/params/option_test.go (1)
451-705: Implement or remove commented test functionsThe new commented test functions (
TestWithName,TestWithOverrideDefault, andTestWithArgumentFilters) are templates without actual test cases. Consider either implementing them with real test cases or removing them to avoid code clutter. Having proper tests for these functions would improve code coverage and ensure their correct behavior.internal/config/config_test.go (1)
999-1003: Test case updated to reflect new behavior for non-existent files.The test case now correctly reflects that
GetActualValueshould return an empty result when a file does not exist, rather than returning the file path itself. This change improves error handling for missing files by gracefully returning empty instead of potentially misleading information.internal/timeutil/time_test.go (1)
184-349: Implement the commented-out test functions for DurationStringBoth test functions (
TestDurationString_DurationandTestDurationString_DurationWithDefault) are structured well but remain commented out and lack actual test cases. These tests are essential to ensure the newDurationStringtype functions correctly.Please implement these tests with the following cases:
For
TestDurationString_Duration:
- Valid duration strings (e.g., "1s", "1m", "1h", "1h30m")
- Invalid duration formats
- Empty string handling
For
TestDurationString_DurationWithDefault:
- Valid duration string processing
- Fallback to default value for empty strings
- Fallback to default value for invalid strings
The test structure is already well-defined with proper error checking and goroutine leak detection.
apis/swagger/v1/vald/update.swagger.json (2)
17-18: Enhanced Documentation for/updateEndpoint.
The updatedsummaryanddescriptionnow provide a well-structured overview, including a comprehensive table of status codes and troubleshooting guidelines. This clear presentation helps API consumers understand potential error responses and their resolutions.
50-51: Improved Documentation for/update/multipleEndpoint.
The revised summary succinctly explains the multi-update operation, and the enhanced description effectively warns about gRPC message size limitations while providing detailed troubleshooting steps. This documentation upgrade improves clarity for users handling multiple updates in a single request.pkg/gateway/mirror/service/mirror.go (2)
143-143: Improved error handling witherrors.IsNotThe replacement of
!errors.Is(err, errors.ErrTargetNotFound)witherrors.IsNot(err, errors.ErrTargetNotFound)improves code readability by removing the negation operator. This change maintains the same functionality while making the code more explicit about the intention to check if an error is not of a specific type.
344-344: Consistent error handling pattern witherrors.IsNotSimilar to the previous change, replacing
!errors.Is(err, errors.ErrGRPCClientConnNotFound(addr))witherrors.IsNot(err, errors.ErrGRPCClientConnNotFound(addr))improves readability while maintaining identical behavior. This change is consistent with the project's effort to enhance error handling patterns throughout the codebase..github/workflows/e2e.yaml (8)
45-47: Credentials for e2e-stream-crud Container
The addition of thecredentialsblock ensures that the container uses the secure environment variables for authentication. Verify that the secretsPACKAGE_USERandPACKAGE_TOKENare correctly configured in your repository settings.
89-91: Credentials for e2e-stream-crud-for-operator Container
This change consistently adds container-level authentication. Confirm that the operator job indeed requires these credentials and that the corresponding GitHub secrets are set up properly.
156-158: Credentials for e2e-stream-crud-under-index-management-jobs
The credentials block here mirrors other container configurations, promoting a consistent security practice. Ensure that the secrets provided have the appropriate permissions for index management operations.
204-206: Credentials for e2e-stream-crud-skip-exist-check
The secure injection of credentials is maintained here. It would be beneficial to double-check that the execution context for this job requires these specific credentials without overexposing access rights.
248-250: Credentials for e2e-multiapis-crud Container
The uniform use of thecredentialssection helps standardize access across jobs. Confirm that the token’s permissions cover all multi-API operations and that there are no unexpected side effects from using the same credentials across different jobs.
288-290: Credentials for e2e-jobs Container
Adding container credentials here improves the security posture for job execution. Please ensure that these credentials are periodically rotated or audited as per your security policy.
325-327: Credentials for e2e-stream-crud-with-readreplica
The credentials injection is consistent with other jobs. Verify that the secret token’s scope includes all necessary operations for handling read-replica deployments, especially under dynamic scaling conditions.
378-381: Credentials for e2e-stream-crud-with-mirror
The newly added credentials help secure the mirror-based E2E tests. Ensure that the same security standards and secret management practices are applied here, matching the rest of the workflow configurations.pkg/gateway/lb/handler/grpc/aggregation_test.go (5)
37-39: Good addition of the new wantAttrs field to the want structThe want struct has been expanded to include expected attributes for validation, which aligns with the enhanced testing capabilities being added in the E2E testing framework.
46-46: Function signature properly updated to support attribute checkingThe checkFunc signature has been correctly modified to accommodate the new attribute validation parameter.
50-50: DefaultCheckFunc signature properly updatedThe function signature has been correctly updated to accept the new attribute parameter, maintaining consistency with the checkFunc signature change.
57-59: Good implementation of attribute validationThe implementation correctly validates that gotAttrs matches wantAttrs using reflect.DeepEqual, following the same pattern as the existing validation for other fields.
157-158: Correctly updated function call and error checkingThe code now properly captures and validates the new attribute return value from the aggregationSearch method.
internal/client/v1/client/discoverer/discover.go (1)
290-290: Improved error handling with consolidated condition.The change from using two separate
errors.Ischecks to a singleerrors.IsNotfunction makes the code more concise while maintaining the same logical behavior. This aligns well with the PR's goal of improving error handling capabilities in the internal packages.internal/test/mock/server_stream.go (1)
30-38: Robust nil handling added to Send method.The improved error handling prevents potential nil pointer dereferences, making the mock more resilient during testing. The hierarchical approach to handling different cases is clear and logical.
While this implementation is correct, the static analysis shows these new code paths aren't covered by tests. Consider adding test cases that explicitly verify:
- Behavior when
mis nil- Behavior when
SendFuncis defined- Fallback to
ServerStream.SendMsgwhenSendFuncis nil#!/bin/bash # Check existing test coverage for StreamInsertServerMock echo "Tests using StreamInsertServerMock:" rg -l "StreamInsertServerMock" --type go | grep "_test.go" echo "---" echo "Test calls to the Send method:" rg "StreamInsertServerMock.*Send" --type go | grep "_test.go"🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by testsapis/swagger/v1/vald/remove.swagger.json (1)
50-51: Enhanced Documentation for MultiRemove Endpoint
The revised summary and description for the/remove/multipleendpoint now provide clear guidance—including a notice about gRPC message size limitations and structured troubleshooting steps. Please verify that the embedded HTML (e.g., the<div class="notice">tag) and Markdown elements render correctly in your Swagger UI.internal/runner/runner.go (4)
180-180: Improved error handling clarity with errors.IsNotThe new
errors.IsNotcall simplifies the previously complex condition that checked if an error was not a context deadline or cancellation error. This makes the code more readable while maintaining the same logical behavior.
192-192: Consistent error handling pattern across lifecycle methodsThe use of
errors.IsNothere maintains consistency with the other lifecycle methods, improving the overall readability and maintainability of the codebase.
204-204: Uniform error filtering approachThis change follows the same pattern applied to other lifecycle methods, ensuring uniform treatment of expected context errors during shutdown.
213-213: Better error handling abstractionBy using
errors.IsNothere as well, the code successfully abstracts away the repetitive error checking logic across all shutdown phases, adhering to the DRY principle.This refactoring improves code readability while maintaining the same behavior of ignoring context cancellation and deadline exceeded errors during the shutdown sequence.
apis/swagger/v1/vald/upsert.swagger.json (1)
50-51: Improved Documentation for the MultiUpsert Endpoint
The revisedsummarysuccinctly describes the MultiUpsert operation, and the newdescriptioneffectively warns users about gRPC message size limitations while providing detailed troubleshooting steps. This enhancement ensures that users are better informed about potential constraints when handling multiple vectors in a single request.apis/swagger/v1/vald/filter.swagger.json (7)
17-17: Enhanced Summary for Insert Object EndpointThe updated summary now clearly defines the operation (inserting an object) and details the status codes that may be returned. This added clarity should help API consumers understand the endpoint's behavior.
49-49: Enhanced Summary for Multi Insert Object EndpointThe refined summary concisely explains that this endpoint handles multiple insert operations in a single request and includes a detailed status code table. This is a good improvement for API documentation.
81-81: Improved Summary for Search Object EndpointThe updated summary provides a more descriptive explanation of the search operation and lists all relevant status codes. This increased clarity will benefit users integrating with the API.
113-113: Detailed Summary for Multi Search Object EndpointBy including information about the bidirectional streaming RPC mechanism along with a clear breakdown of status codes, this summary significantly enhances the documentation for handling multiple search queries.
145-145: Clear Summary for Update Object EndpointThe updated summary succinctly describes the endpoint's purpose—updating a single vector—and provides a detailed status code table, which makes the API behavior much clearer.
210-210: Precise Summary for Upsert Object EndpointThe revised summary clearly states that the endpoint can both update an existing object and add a new one, while listing the pertinent status codes. This dual-purpose explanation is now much more informative.
242-243: Comprehensive Summary for Multi Upsert Object EndpointThe updated summary, together with the new descriptive notice, effectively communicates the functionality for handling multiple upsert operations and alerts users to gRPC message size limitations. This comprehensive documentation will aid in proper API usage.
internal/file/file.go (3)
259-259: Error handling improvement usingerrors.IsNotThe error handling has been modified to use
errors.IsNot(err, io.EOF)instead of the likely previous patternerr != nil && !errors.Is(err, io.EOF). This improves code readability and maintainability by encapsulating the error negation pattern.
374-391: Well-implemented path handling utility functionThe new
AbsolutePathfunction efficiently handles path normalization with proper edge cases:
- Returns empty string for empty input
- Uses current working directory for relative paths
- Falls back to filepath.Abs when needed
- Properly cleans the final path
The function follows a clear progressive approach to resolving paths, making it robust against different input scenarios.
564-564: Good refactoring to reduce duplicationThe modification of the
Joinfunction to use the newAbsolutePathfunction removes duplicated path resolution logic that was likely present before, improving maintainability and consistency.internal/file/file_test.go (1)
1523-1606: TestAbsolutePath needs implemented test casesThe test structure is sound, but it currently only has commented-out placeholders without actual test cases. Since this is testing a new function, it's critical to implement test cases that cover various scenarios.
Consider adding test cases for:
tests := []test{ - // TODO test cases - /* - { - name: "test_case_1", - args: args { - path:"", - }, - want: want{}, - checkFunc: defaultCheckFunc, - beforeFunc: func(t *testing.T, args args) { - t.Helper() - }, - afterFunc: func(t *testing.T, args args) { - t.Helper() - }, - }, - */ + { + name: "return empty string when path is empty", + args: args{ + path: "", + }, + want: want{ + want: "", + }, + }, + { + name: "return absolute path when path is relative", + args: args{ + path: "testdata/sample.txt", + }, + beforeFunc: func(t *testing.T, args args) { + t.Helper() + // Get the current working directory for comparison in checkFunc + }, + checkFunc: func(w want, got string) error { + // Since the absolute path depends on the current working directory, + // verify that the result starts with / (Unix) or contains :\ (Windows) + if !filepath.IsAbs(got) { + return errors.Errorf("got: \"%#v\" is not an absolute path", got) + } + return nil + }, + }, + { + name: "return same path when path is already absolute", + args: args{ + path: filepath.Join(os.TempDir(), "sample.txt"), + }, + want: want{ + want: filepath.Clean(filepath.Join(os.TempDir(), "sample.txt")), + }, + },Makefile.d/functions.mk (1)
134-154:❓ Verification inconclusive
New V2 E2E testing framework looks good but needs verification
The new
run-v2-e2e-crud-testfunction introduces a more maintainable, configuration-driven approach for E2E testing, which aligns well with the PR objectives. It's notably more streamlined than the original function, using a centralized config file instead of numerous individual parameters.However, I noticed the V1 function explicitly handles port-forwarding configurations (lines 188-192), while the V2 function doesn't appear to have equivalent functionality directly visible.
🏁 Script executed:
#!/bin/bash # Check if the new V2 E2E framework handles port-forwarding configurations echo "Checking for port-forwarding logic in V2 E2E framework..." # Check the test config file structure if [ -f "tests/v2/e2e/crud/config.go" ]; then echo "Examining config structure in tests/v2/e2e/crud/config.go" grep -A 10 "type Config" tests/v2/e2e/crud/config.go echo "Checking for port-forwarding related code:" grep -r "portforward" --include="*.go" tests/v2/e2e/ fi # Check if there's documentation explaining the transition from V1 to V2 echo "Checking for V2 E2E framework documentation..." find . -path "./tests/v2/e2e" -type f -name "*.md" | xargs catLength of output: 331
Attention: Port-forwarding configuration missing in the new V2 function
The newrun-v2-e2e-crud-testfunction adopts a centralized, configuration-driven approach for E2E testing, which is more maintainable and aligns well with the PR’s objectives. However, our checks indicate that the explicit port-forwarding logic present in the V1 implementation (lines 188–192) is not visible in the V2 framework—there’s no sign of "portforward" code in the E2E tests or any accompanying documentation addressing this transition. Please confirm if this omission is intentional or if additional adjustments are expected.
- Location under review:
Makefile.d/functions.mk, lines 134–154- Verification details: Verified absence of port-forwarding related code in the V2 framework and its documentation via repository grep/search.
.github/workflows/codeql-analysis.yml (8)
46-48: New 'analyze' Job with Dynamic Matrix ConfigurationA new job named
analyzeis added with a dynamicruns-onexpression that selectsmacos-latestwhen the language is Swift and defaults toubuntu-latestfor all others. Please verify that this conditional expression using&&and||works as expected in GitHub Actions.
50-51: Enabling CodeQL Experimental FeaturesThe environment variables
CODEQL_ENABLE_EXPERIMENTAL_FEATURESandCODEQL_ACTION_TEST_MODEare set for this job. Ensure these settings are documented and that enabling experimental features does not unintentionally affect production runs.
53-57: Setting Minimal Permissions for AnalysisThe job now explicitly sets permissions for
security-events,packages,actions, andcontents. Confirm that these permissions adhere to the principle of least privilege and match your security policies.
58-67: Matrix Strategy for Multi-Language AnalysisA matrix is defined to run the analysis for different languages (
actions,go, andrust) with their correspondingbuild-modesettings. This structure is effective for covering multiple environments. Double-check that these matrix configurations tie in correctly with your build process.
70-72: Container Credentials ConfigurationIn the container section, credentials are now provided (using
PACKAGE_USERandPACKAGE_TOKEN) to access the container image. Ensure that the secrets are securely managed and that the image tag derived fromdetect-ci-containeroutputs is correctly validated.
83-88: Updating CodeQL Initialization to Version 3The "Initialize CodeQL" step now uses
github/codeql-action/init@v3and passes dynamic values forlanguagesandbuild-mode. Please confirm that these changes are compatible with your overall CodeQL configuration.
89-91: Conditional Autobuild for GoThe autobuild step is conditionally executed only for Go (when
matrix.language == 'go') and usesgithub/codeql-action/autobuild@v3. Verify that this conditional logic meets your build requirements for Go and that no other language inadvertently triggers this step.
92-96: Performing CodeQL Analysis with Updated VersionThe analysis step now employs
github/codeql-action/analyze@v3and sets a category based on the language (e.g.,/language:go). Ensure that this categorization integrates seamlessly with your reporting and that the updated version does not disrupt previously established analysis routines.internal/net/quic/conn_test.go (6)
44-83: Replace placeholder comments with actual test casesThis section contains TODO placeholder comments instead of actual test implementations. This matches previous review comments that were flagged in other commits.
Implement at least these basic test cases for
TestNewConn:
- Create a connection with valid input
- Create a connection with nil input
- Test error handling
For example:
{ name: "successful_connection_creation", args: args{ ctx: context.Background(), conn: &mockQuicConnection{}, }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: nil, }, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "nil_connection_input", args: args{ ctx: context.Background(), conn: nil, }, want: want{ want: nil, err: errors.New("expected error message"), }, checkFunc: defaultCheckFunc, },
110-152: Implement test cases for connection closingThe
TestConn_Closefunction lacks actual test implementations. Testing connection closing is critical for ensuring proper resource management.Implement test cases that verify:
- Successful closing of both connection and stream
- Error handling when connection closing fails
- Error handling when stream closing fails
- Behavior when fields are nil
297-352: Add comprehensive test cases for QUIC connection dialingThe test for
qconn_dialQuicContextis particularly important as it likely handles connection caching and reconnection logic.Implement test cases that verify:
- Creating a new connection when not in cache
- Reusing cached connections
- Error handling for invalid addresses
- Error handling for TLS configuration errors
- Context cancellation handling
405-464: Implement mock QUIC connection for testingTo effectively test QUIC connections, you need mock implementations.
Create mock implementations for QUIC connections and streams:
type mockQuicConnection struct { closeFunc func() error openStreamFunc func(context.Context) (quic.Stream, error) // other required methods } func (m *mockQuicConnection) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } func (m *mockQuicConnection) OpenStream(ctx context.Context) (quic.Stream, error) { if m.openStreamFunc != nil { return m.openStreamFunc(ctx) } return &mockQuicStream{}, nil } // Implement other required methods type mockQuicStream struct { closeFunc func() error readFunc func([]byte) (int, error) writeFunc func([]byte) (int, error) } func (m *mockQuicStream) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } // Implement read/write methods
90-91: Add goleak import and setup proper leak detectionThe test uses
goleak.VerifyNone()but the import is missing.Import go.uber.org/goleak package and consider using the project's custom goleak wrapper if available. Also ensure proper setup for leak detection.
36-37: Verify imports match error function usageThe code uses
errors.Is()anderrors.Errorf()but standard library's errors package doesn't includeErrorf().Ensure you're importing the correct errors package. Based on function usage, this appears to be a custom errors package from the project (likely "github.com/vdaas/vald/internal/errors").
apis/swagger/v1/vald/flush.swagger.json (1)
17-17: Enhanced Summary for the/flushDELETE Endpoint
The updated summary now provides a structured overview with a detailed status code table. Please verify that the line breaks (\n) and table formatting are rendered correctly in your API documentation tool.apis/swagger/v1/vald/insert.swagger.json (1)
17-18: Typographical Error in/insertSummary
The summary still reads “Inset RPC is the method to add a new single vector.” Please update “Inset” to “Insert” to accurately reflect the endpoint’s functionality.apis/swagger/v1/vald/search.swagger.json (8)
17-18: Enhanced/linearsearchEndpoint Documentation
The updated summary and description now provide a detailed overview with a comprehensive status code table and troubleshooting guidance. Please ensure that the formatting (especially the use of line breaks and table markers) appears correctly in your rendered docs.
50-52: Improved/linearsearch/idEndpoint Summary
The revision clearly explains that this endpoint searches by a user-defined vector ID and highlights prerequisites (i.e. the vector must already be indexed). Verify that the line breaks (including the HTML<br>) and formatting are preserved in your API documentation.
83-84: Detailed/linearsearch/id/multipleEndpoint Documentation
The summary now explicitly mentions the multi-ID search functionality and includes an HTML notice for potential gRPC message size limitations. Confirm that the embedded HTML is compatible with your Swagger UI.
116-117: Comprehensive/linearsearch/multipleEndpoint Overview
The updated summary explains the multi-vector search capability with a clear status code table and troubleshooting section. Ensure that the formatting remains consistent with other endpoints.
149-150: Clear/searchEndpoint Overview
The enhanced summary and description for the/searchendpoint concisely articulate the search functionality and include a detailed status code table with troubleshooting tips.
182-184: Enhanced/search/idEndpoint Summary
The revised documentation clearly outlines that the search by ID requires the vector to be pre-indexed and details various error scenarios with suggested resolutions.
215-217: Detailed/search/id/multipleEndpoint Documentation
The summary accurately reflects the multi-ID search operation, and the description provides a well-structured status code table along with troubleshooting guidance.
248-250: Comprehensive/search/multipleEndpoint Overview
The updated summary and description clearly explain the ability to search with multiple vectors in a single request. Please double-check that the formatting, especially of the notice and table sections, is consistent with the rest of the API documentation.internal/params/option.go (3)
38-45: LGTM: Well-structured option functionThe implementation appropriately checks for empty input and only sets the name if a non-empty value is provided.
89-94: LGTM: Clear implementation for flag override optionGood implementation with descriptive comments explaining the purpose of this option.
96-109: Well-designed argument filtering mechanismThe implementation has good safety checks:
- Returns early if no filters provided
- Initializes the filters slice if it's nil
- Pre-allocates the slice with appropriate capacity
internal/params/params.go (6)
29-35: LGTM: Good type aliasing for ErrorHandlingClean approach to exposing flag package constants with a proper type alias.
53-70: LGTM: Extended parser struct with new capabilitiesThe additions to the parser struct enable proper flag set management and argument filtering capabilities needed for the new E2E testing framework.
78-82: LGTM: Improved flag set initialization in New methodThe parser now correctly initializes both the default flag set and a custom flag set with the specified name and error handling.
87-90: LGTM: Added nil safety checksGood defensive programming by checking for nil parser or flag set before proceeding.
108-118: LGTM: Robust argument filtering implementationThe implementation uses
slices.DeleteFuncefficiently and includes proper safety checks for nil filters.
122-126: Improved error checking using errors.IsUsing
errors.Is()is the correct way to check for specific error types compared to direct equality checks.internal/net/quic/listener_test.go (3)
16-207: Commented-out test code should be implemented or removedThis entire test file consists of commented-out code marked with "NOT IMPLEMENTED BELOW". While the test structure follows good practices (table-driven tests, setup/teardown functions, goroutine leak detection), committing commented-out code isn't recommended.
Consider one of these approaches:
- Implement at least basic test cases before merging
- Remove the commented code and add a TODO with an issue reference
- If there's a specific reason for keeping this structure, document it clearly
45-87: Implement test cases for the Listen functionThe test structure is well-designed, but lacks actual test cases. Consider implementing these scenarios:
- Successful listener creation with valid address and TLS config
- Error handling with invalid address
- Context cancellation behavior
- Error handling with nil TLS configuration
Example implementation for the first test case:
tests := []test{ + { + name: "successful_listener_creation", + args: args{ + ctx: context.Background(), + addr: "localhost:0", // system-assigned port + tcfg: &tls.Config{ + Certificates: []tls.Certificate{generateSelfSignedCert(t)}, + }, + }, + want: want{ + want: nil, // Don't check exact listener, just verify non-nil + err: nil, + }, + checkFunc: func(w want, got net.Listener, err error) error { + if err != nil { + return errors.Errorf("expected no error, got: %v", err) + } + if got == nil { + return errors.New("expected non-nil listener") + } + // Don't need to check exact listener value + return nil + }, + afterFunc: func(t *testing.T, args args) { + t.Helper() + // Ensure listener is closed after test + if l, ok := t.Conn.(*Listener); ok && l != nil { + l.Close() + } + }, + },
139-179: Implement test cases for Listener.Accept methodThe test structure for
TestListener_Acceptis well-designed but lacks actual test cases. Consider implementing these scenarios:
- Successful connection acceptance
- Context cancellation during Accept
- Error handling when the underlying listener fails
Example for a context cancellation test:
tests := []test{ + { + name: "context_cancellation_during_accept", + fields: fields{ + Listener: createMockQuicListener(t), + ctx: func() context.Context { + ctx, cancel := context.WithCancel(context.Background()) + // Cancel immediately to simulate cancellation during Accept + cancel() + return ctx + }(), + }, + want: want{ + want: nil, + err: context.Canceled, + }, + checkFunc: defaultCheckFunc, + },You'll need to create helper functions like
createMockQuicListenerto set up test fixtures.pkg/index/job/deletion/service/deleter_test.go (1)
416-527: Error handling test scenarios are missing for Test_index_doDeleteIndexThe
Test_index_doDeleteIndexfunction needs test cases for error conditions during the deletion process.apis/docs/v1/docs.md (1)
1636-1649: Ensure Consistent Method Descriptions in Filter Service
The updated Filter service overview and method descriptions appear comprehensive. However, please double-check that the descriptions for “SearchObject”, “MultiSearchObject”, and “StreamSearchObject” are correctly aligned with their intended functionality. This update seems related to previous feedback on method mismatch.go.mod (46)
3-3: Upgrade Go Version
The Go version is now set to 1.24.1. This minor version bump should bring in improvements and bug fixes. Please confirm that your build and CI environments support this version.
6-8: Update cloud.google.com Dependencies
The versions forcloud.google.com/go,.../bigquery, and.../computehave been bumped (e.g. to v0.120.0, v1.67.0, and v1.36.0 respectively). These changes likely include performance improvements and bug fixes. Verify that any API changes do not break existing usage in your codebase or tests.
11-18: Update Additional cloud.google.com & Bytefmt Modules
Updates have been applied to several modules:
•cloud.google.com/go/iam→ v1.4.2
•cloud.google.com/go/kms→ v1.21.1
•cloud.google.com/go/monitoring→ v1.24.1
•cloud.google.com/go/pubsub→ v1.48.0
•cloud.google.com/go/secretmanager→ v1.14.6
•cloud.google.com/go/storage→ v1.51.0
•cloud.google.com/go/trace→ v1.11.5
•code.cloudfoundry.org/bytefmt→ v0.34.0Please ensure that these dependency bumps are compatible with your configuration and that all functionality (such as metrics and storage handling) is properly tested.
25-25: Update Azure SDK azcore Module
The modulegithub.com/Azure/azure-sdk-for-go/sdk/azcorehas been updated to v1.17.1. Check if any new behaviors or fixes require corresponding code or configuration updates.
28-28: Update Azure go-amqp Library
The dependencygithub.com/Azure/go-amqpis updated to v1.4.0. Make sure that error handling and messaging patterns used in your project still behave as expected after this change.
37-37: Update toml Parser
Thegithub.com/BurntSushi/tomlpackage has been bumped to v1.5.0. It may include performance and bug fixes in configuration parsing. Please run tests on your TOML handling functionality to ensure compatibility.
39-39: Update Cloud SQL Proxy Dependency
The version ofgithub.com/GoogleCloudPlatform/cloudsql-proxyis now v1.37.6. Double‐check that connection handling continues to work properly with your database integration.
48-69: Multiple AWS SDK V2 Updates
A series of updates affect the AWS SDK for Go V2 and its submodules (e.g. credentials, config, imds, service clients for S3, KMS, SNS, SQS, etc.). These changes can impact behavior if your code depends on specific API responses or error handling. Extensive integration tests against AWS services are recommended.
110-112: Updates for OpenGL & LaTeX Related Packages
The dependencies forgithub.com/go-gl/glfw/v3.3/glfwandgithub.com/go-latex/latexhave been updated. Ensure that any features tied to graphics rendering or document generation remain stable after these bumps.
117-119: Update go-openapi Modules
The modulesgithub.com/go-openapi/jsonpointerandgithub.com/go-openapi/swaghave been updated. Verify that any auto-generated API documentation or schema validations are still correct in your Swagger or OpenAPI integrations.
124-124: Update Validator Library
github.com/go-playground/validator/v10is now at v10.26.0. Since this library is integral for data validation, please confirm that custom validation tags and rules continue to operate as expected.
126-126: Update MySQL Driver
The MySQL driver (github.com/go-sql-driver/mysql) has been updated to v1.9.1. Verify that connection strings, query behaviors, and any driver-specific configurations are still valid.
134-134: Update go-json Library
The update ofgithub.com/goccy/go-jsonto v0.10.5 is intended to improve performance. Ensure that JSON serialization and deserialization in your application still produce the expected results.
140-140: Update JWT Library
Thegithub.com/golang-jwt/jwt/v4module update to v4.5.2 may include changes in claim validation or signing methods. Re-run authentication tests to verify that token encoding/decoding remains secure and functional.
148-148: Update Snappy Compression Library
Thegithub.com/golang/snappylibrary is now at v1.0.0. Though this is typically a minor update, it is prudent to check that any compression/decompression routines integrated into your data processing still work as expected.
151-151: Update go-cmp Comparison Utility
The update togithub.com/google/go-cmpversion v0.7.0 should improve diff functionality. Verify that your unit tests relying on deep comparisons remain unaffected by any changes in comparison logic.
157-157: Update pprof Tool
The version ofgithub.com/google/pprofhas been bumped. It is advisable to ensure that your profiling outputs or performance analyses (if used in E2E tests) remain accurate.
167-167: Update grpc-gateway
github.com/grpc-ecosystem/grpc-gateway/v2is now at v2.26.3. Revalidate the REST-to-gRPC translation and any custom handlers to ensure API consistency.
263-264: Update Afero and Cobra
The librariesgithub.com/spf13/aferoandgithub.com/spf13/cobrahave been updated. Since they underpin file system abstraction and CLI behavior, please check that command parsing and file interactions remain intact.
352-364: Require Block Dependency Updates (Part 1)
In the first require block, several modules (e.g. the bufbuild protovalidate, code.cloudfoundry.org/bytefmt, pogreb, aws-sdk-go, fsnotify, go-redis, go-json, gocql, and gocraft/dbr) have been updated to new versions. Because these dependencies can affect core functionality (from protocol validation to database access), thorough integration tests are essential.
366-367: Update Gorilla Mux & Pyroscope
The updates forgithub.com/gorilla/muxandgithub.com/grafana/pyroscope-go/godeltaprofin this block use pseudo-versions. Confirm that these updates are intentional and that your routing and profiling features continue to work as expected.
368-368: Review Hashicorp go-version Pseudo-version
The update forgithub.com/hashicorp/go-versionuses a pseudo-version. Please verify that this version accurately reflects a stable release and does not introduce unintended side effects in version comparisons.
371-372: Update KPango Dependencies
Bothgithub.com/kpango/fuidandgithub.com/kpango/gache/v2have been updated to pseudo-versions. Ensure these align with the upstream releases and that caching or unique ID generation in your application is unaffected.
374-377: Update Kubernetes CSI and Related Modules
The dependencygithub.com/kubernetes-csi/external-snapshotter/client/v6and others such asgithub.com/leanovate/gopter,github.com/lucasb-eyer/go-colorful, andgithub.com/pierrec/lz4/v3have been bumped. Since these often interplay with Kubernetes and data processing, verify that E2E and integration tests (especially those involving CSI) pass without issue.
379-382: Update Quasilyte and scylladb Packages
The updates forgithub.com/quasilyte/go-ruleguard,github.com/quic-go/quic-go, andgithub.com/scylladb/gocqlxbring in pseudo-versions. These changes could affect code analysis or high-performance network routines. Testing under load is recommended.
386-395: Significant Opentelemetry and Related Updates
A cluster of updates affectgo.opentelemetry.io/otel, its exporters, and related telemetry modules, as well asgo.uber.org/automaxprocs. These are critical for observability. Please validate that metrics collection, trace exporting, and automatic CPU configuration work correctly under the new versions.
396-407: Update gocloud.dev and golang.org/x Modules
Several updates have been applied togocloud.devand variousgolang.org/x/*packages (net, oauth2, sync, sys, text, time, tools). These libraries underpin many low-level operations, so a full run of your test suite is necessary to catch any regressions.
408-414: Update Additional Dependencies: gonum and gRPC-Related Modules
The updates forgonum.org/v1/hdf5,gonum.org/v1/plot,google.golang.org/genproto,google.golang.org/grpc, andgoogle.golang.org/protobufmay impact numerical computations and gRPC communications. Careful runtime tests and benchmarks are advised.
415-422: Update Kubernetes & Controller Dependencies
The Kubernetes‑related modules—k8s.io/api,k8s.io/apimachinery,k8s.io/cli-runtime,k8s.io/client-go,k8s.io/metrics,k8s.io/utils, andsigs.k8s.io/controller-runtime—have been updated. Verify these against your cluster integration tests or Helm charts, as even minor API changes can cause issues in a Kubernetes environment.
426-434: Second Require Block – Cloud and Codeberg Updates
In the second require block, updates tocel.dev/expr,cloud.google.com/go(indirect),cloud.google.com/go/auth, and Codeberg libraries (go-fonts/liberation,go-latex/latex,go-pdf/fpdf) have been applied. Although these are marked as indirect, please verify that any tooling or documentation that depends on these libraries is not adversely affected.
438-441: Second Block – Azure and OTEL Operations Updates
Dependencies related to Azure terminal formatting (github.com/Azure/go-ansiterm) and several components from GoogleCloudPlatform’s OpenTelemetry operations have been updated. Ensure that resource detection and metric exporting behave correctly.
443-443: Update benbjohnson/clock
Thegithub.com/benbjohnson/clockpackage has been updated to v1.3.0. Double-check any time-based logic or tests that rely on this package, as subtle differences might occur.
449-449: Update CNCF xds Dependency
The update togithub.com/cncf/xds/go(indirect) should be validated against your networking configuration, particularly if you leverage xDS protocols in your Kubernetes integrations.
458-458: Update go-errors Library
Thegithub.com/go-errors/errorspackage is now at v1.4.2. This update might provide improved error wrapping and stack trace details – confirm with your error handling tests.
462-463: Update go-openapi (Second Block)
The modulesgithub.com/go-openapi/jsonreferenceandgithub.com/go-openapi/swagin the second block are updated. Their changes can affect auto-generated documentation and API schema validation.
465-466: Update go-toolsmith Utilities
Bothgithub.com/go-toolsmith/astcopyandgithub.com/go-toolsmith/astequalare now updated. Ensure that any custom logic using AST comparisons or code generation is still functioning correctly.
476-476: Consistent pprof Version
Thegithub.com/google/pprofupdate in the second block should be consistent with the earlier update. Make sure profiling data and diagnostics are not disrupted.
480-480: Update Enterprise Certificate Proxy Module
The modulegithub.com/googleapis/enterprise-certificate-proxyis updated to v0.3.6. If your project uses this for certificate management or validation, please test those flows.
482-482: Update Gorilla Websocket (Second Block)
The second-block update togithub.com/gorilla/websocketshould be validated to ensure that real-time communication or websocket-based interactions remain robust and error‑free.
490-490: Update cpuid Library
Thegithub.com/klauspost/cpuid/v2update to v2.0.9 is critical if your application relies on CPU feature detection for optimizations. Please run relevant benchmarks or tests to ensure correctness.
494-494: Update Moby Term Dependency
Thegithub.com/moby/termpackage has been updated. Check that any terminal or TTY-related operations in your E2E tests still behave as expected, especially if interactive command execution is involved.
500-500: Update Ginkgo v2 for Testing
The version ofgithub.com/onsi/ginkgo/v2has been updated to v2.23.3. This change impacts your test framework; please run your entire test suite to catch any breaking changes in behavior or reporting.
504-504: Update Prometheus Client_golang
The update togithub.com/prometheus/client_golang(v1.20.4) may affect custom metrics collection and exposition. Verify that your application’s monitoring endpoints continue to deliver accurate metrics.
523-529: Second Block – Update golang.org/x/ Modules*
Severalgolang.org/xmodules (including crypto, exp, typeparams, image, mod, term, and xerrors) have been updated. These underpin many standard library extensions and low‑level operations. A broad test run is recommended to ensure no regressions in functionality.
531-532: Update Google API & Genproto
The updates togoogle.golang.org/apiandgoogle.golang.org/genprotoshould be reviewed carefully for any breaking changes affecting interactions with Google Cloud services.
538-542: Update Kubernetes & Kustomize Dependencies
Changes tok8s.io/kube-openapi,sigs.k8s.io/json,sigs.k8s.io/kustomize/api,sigs.k8s.io/kustomize/kyaml, andsigs.k8s.io/randfillhave been applied. These updates may impact Helm charts and controller-runtime behavior. Verify these in your Kubernetes integration and deployment tests.internal/test/data/vector/noise/option_test.go (3)
16-101: Add test coverage forTestWithLevelFactor.All test logic is commented out, resulting in no actual coverage. This mirrors a previous comment urging test coverage for
WithLevelFactor.
103-186: Add test coverage forTestWithTableDivisionFactor.All test logic is commented out, meaning there is no coverage for
WithTableDivisionFactor. This duplicates a past review comment requesting complete test coverage.
188-271: Add test coverage forTestWithMinTableSize.No active test cases exist for this function, so the behavior of
WithMinTableSizeremains unverified. This repeats a prior suggestion to validate various bounds (e.g., zero or large inputs).internal/iter/iter.go (1)
22-22: Remove or correct the self-import to avoid a circular dependency.Importing
"iter"from within theiterpackage can cause circular dependencies or confusion. If this is meant to reference the same package, remove it. Otherwise, replace"iter"with the intended fully qualified path..gitfiles (1)
126-126: No code snippet provided.This line references a new or updated file (
.github/workflows/unit-test-rust.yaml), but no code details are shown here for review.internal/net/grpc/status/status_test.go (1)
245-331: Implement test cases for theTestIsfunctionThe newly added
TestIsfunction is commented out and lacks actual test cases, which means there's no test coverage for theIsfunctionality that verifies error status codes.Please uncomment this function and implement proper test cases to ensure the
Isfunction works correctly. At minimum, include cases for:
- When the error is nil
- When the error is a status error with the matching code
- When the error is a status error with a different code
- When the error is a non-status error
func TestIs(t *testing.T) { type args struct { err error code codes.Code } type want struct { want bool } type test struct { name string args args want want checkFunc func(want, bool) error beforeFunc func(*testing.T, args) afterFunc func(*testing.T, args) } defaultCheckFunc := func(w want, got bool) error { if !reflect.DeepEqual(got, w.want) { return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) } return nil } tests := []test{ { name: "nil error returns false", args: args{ err: nil, code: codes.OK, }, want: want{ want: false, }, checkFunc: defaultCheckFunc, }, { name: "matching status code returns true", args: args{ err: Error(codes.NotFound, "not found"), code: codes.NotFound, }, want: want{ want: true, }, checkFunc: defaultCheckFunc, }, { name: "non-matching status code returns false", args: args{ err: Error(codes.NotFound, "not found"), code: codes.Internal, }, want: want{ want: false, }, checkFunc: defaultCheckFunc, }, { name: "non-status error returns false", args: args{ err: errors.New("regular error"), code: codes.Internal, }, want: want{ want: false, }, checkFunc: defaultCheckFunc, }, } for _, tc := range tests { test := tc t.Run(test.name, func(tt *testing.T) { tt.Parallel() defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) if test.beforeFunc != nil { test.beforeFunc(tt, test.args) } if test.afterFunc != nil { defer test.afterFunc(tt, test.args) } checkFunc := test.checkFunc if test.checkFunc == nil { checkFunc = defaultCheckFunc } got := Is(test.args.err, test.args.code) if err := checkFunc(test.want, got); err != nil { tt.Errorf("error = %v", err) } }) } }internal/servers/server/server.go (2)
461-463: Good refactoring of error handling logicUsing
errors.IsNotimproves code readability by consolidating multiple error condition checks into a single function call. This makes the error handling more concise while maintaining the same behavior.
466-468: Consistent use of error handling patternThe refactoring consistently applies the same error handling pattern throughout the method, making the code more maintainable and easier to understand.
pkg/agent/core/faiss/service/faiss.go (2)
1125-1127: Good refactoring usingerrors.IsNotfor cleaner error handling.The new error handling implementation using
errors.IsNotimproves readability while preserving the same logical flow. This pattern is more concise than multipleerrors.Ischecks.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
1260-1261: Consistent use oferrors.IsNotfor specific error handling.The consistent use of
errors.IsNotacross the codebase enhances maintainability by establishing a common pattern for error handling.Also applies to: 1268-1269
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by testsinternal/servers/servers.go (2)
100-102: Good simplification usingerrors.IsNot.Replacing the verbose error checking with the more concise
errors.IsNotfunction improves code readability while maintaining the same logic.
123-125: Consistent error handling witherrors.IsNot.The consistent use of
errors.IsNotfor error checking across the codebase makes the error handling pattern more uniform and easier to understand.Also applies to: 132-134
pkg/agent/core/ngt/service/ngt.go (5)
929-933: Clean error handling witherrors.IsNot.The use of
errors.IsNotprovides a more concise way to check if an error is not one of the specified errors, improving code readability.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by tests
1475-1478: Good defensive programming to prevent nil pointer dereference.Adding this nil check is essential to prevent a potential panic when
statsis nil but no error was returned fromGetGraphStatistics. This properly logs a warning and returns gracefully instead of attempting to access fields of a nil pointer.#!/bin/bash # Check for existing test coverage of the nil stats case rg -A 3 "Test.*loadStatistics" --type go🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
1973-1979: Good use oferrors.IsNotfor targeted error checking.Using the
errors.IsNotfunction for checking specific errors improves code readability and maintainability by making the intent clearer and reducing verbosity.
1987-1993: Consistent error handling across the codebase.The consistent use of
errors.IsNotfor error handling improves code maintainability by establishing a common pattern across the codebase.Also applies to: 1995-2001
1824-1827: Simplification of error handling witherrors.IsNot.This consistent approach to error handling makes the code easier to read and maintain while preserving the same logic as before.
internal/errors/errors.go (3)
86-88: LGTM: New error function for unimplemented featuresThe
ErrUnimplementedfunction is a well-implemented addition that follows the existing error function pattern, providing a standardized way to indicate unimplemented features.
170-180: LGTM: Well-implemented error matching helperThe
IsAnyfunction is a useful addition that checks if an error matches any of the provided target errors. The implementation correctly handles edge cases (nil error, empty targets) and leverages the existingIsfunction.
182-192: LGTM: Complementary error matching functionThe
IsNotfunction provides the logical complement toIsAnyand is implemented consistently. It correctly returns true when an error doesn't match any targets, and handles the nil error case appropriately.internal/sync/errgroup/group_test.go (1)
946-1057: Update commented-out test to reflect new exec methodThe commented-out
Test_group_execfunction replaces the previousTest_group_TryGo, with several important changes:
- Modified
wantstruct to remove the boolean field (line 958)- Updated
checkFuncsignature to only acceptwantparameter (line 964)- Added
limit atomic.Int64field to the test'sfieldsstruct (line 953)- Changed function call from
g.TryGo(test.args.f)tog.exec(test.args.f)(line 1051)These changes align with refactoring in the underlying implementation, transitioning from a boolean-returning
TryGomethod to a non-returningexecmethod.internal/net/grpc/stream_test.go (2)
206-210: Update test structure to match new function signatureThe commented-out test now reflects the updated signature of
BidirectionalStreamClientwith important changes:
- Added
concurrency intparameter- Changed
sendDataProviderto return a typed value rather than a pointer- Updated
callBackfunction to return a boolean for flow controlThis aligns with the API changes in the implementation file.
290-290: Updated function call matches new signatureThe test's call to
BidirectionalStreamClienthas been updated to pass all required arguments according to the new function signature, ensuring proper testing when this code is uncommented.internal/errors/k8s.go (1)
20-46: Enhanced error handling for Kubernetes operationsGood addition of specialized error types for Kubernetes operations. The introduction of:
- Factory functions for formatted errors like
ErrPodIsNotRunningandErrUnsupportedKubernetesResourceType- Specific error constants for common error conditions
These additions will improve error handling, debugging, and user feedback for Kubernetes-related operations in the E2E testing framework.
internal/net/grpc/stream.go (10)
44-53: Improved type safety with specialized stream interfacesThe introduction of
TypedClientStreamandTypedServerStreaminterfaces provides strong type guarantees for gRPC streams. These generic interfaces ensure that:
- The correct message types are used for sending and receiving
- The compiler can catch type mismatches
- IDEs can provide better code completion and documentation
This is a great enhancement to type safety while maintaining compatibility with the underlying gRPC interfaces.
60-62: Breaking API change with improved type safetyThe signature of
BidirectionalStreamhas been significantly enhanced:
- Added type constraints requiring
proto.Messagefor proper protobuf compatibility- Changed parameter types from pointers to values for messages
- Changed callback parameter name from
fto the more descriptivehandle- Made the stream parameter use the new typed interface
While these are breaking changes, they greatly improve type safety and code clarity.
106-126: Enhanced stream receive and processing logicThe stream message handling has been improved:
- Direct use of typed
Recv()method instead of genericRecvMsg- Better error wrapping with contextual information
- Enhanced goroutine execution with unique ID tracking
- Improved tracing with span management
These changes make the code more robust and easier to debug.
127-144: Improved error handling in processing pipelineThe error handling logic has been refined to:
- Only log errors for significant error codes
- Parse and format gRPC status information
- Set appropriate trace attributes for observability
- Use runtime.Gosched() to improve scheduling during error conditions
This provides better error discrimination and improves operational visibility.
145-167: Synchronized response sending with detailed error trackingThe response sending logic now:
- Uses mutex to synchronize stream access
- Provides detailed error context including request ID
- Captures error details in trace spans
- Wraps errors with operation context
This ensures thread safety and improves debugging capabilities.
173-175: Breaking API change in BidirectionalStreamClient signatureThe function has been significantly enhanced with:
- Type constraints requiring protobuf messages
- The new
concurrencyparameter for controlling parallelism- Changed
sendDataProvidersignature to return a value and boolean- Modified
callBackto return a boolean for flow controlThese changes improve the API but require updates to all callers.
186-191: Improved error filtering in receiver goroutineThe error handling now uses
errors.IsNotto filter out expected errors like EOF and cancellation, only propagating unexpected errors. This prevents normal termination conditions from being treated as errors.
192-196: Enhanced receiver logic with type-safe Recv methodThe receive operation now:
- Uses the type-safe
Recv()method that returns typed messages- Better identifies termination conditions with
errors.IsAny- Properly cancels the context when stream ends
This improves type safety and prevents resource leaks.
206-230: Robust error management with dedicated finalizationThe finalization logic has been significantly improved:
- Introduced a dedicated error channel for collecting errors from goroutines
- Added synchronized stream closing with mutex protection
- Better error filtering to ignore expected termination errors
- Proper resource cleanup with context cancellation
This comprehensive approach ensures reliable cleanup and accurate error reporting.
238-254: Enhanced data sending with improved concurrency controlThe data sending logic now:
- Uses the return value and boolean from
sendDataProviderfor better flow control- Properly synchronizes stream access with mutex
- Communicates errors through a dedicated channel
- Manages goroutines with the specified concurrency limit
This provides better control over resource usage and improves error handling.
internal/net/grpc/pool/pool.go (3)
17-20: Great documentation addition on lock-free approach.It is helpful to clarify the underlying lock-free design using atomic operations, as it better conveys the rationale behind the re-implementation.
166-168: Validate initial dial logic and fallback.When the initial dial fails, the code proceeds to a port scan or alternative approach. This is good for resilience, but ensure that in high-latency or heavily restricted networks, scanning does not cause severe slowdowns. Consider guarding it with a maximum time or iteration limit beyond which the operation fails quickly.
306-326: Ensure thread-safety in the loop function.Accessing
slots := *p.getSlots()once and iterating over it is likely safe even ifp.connSlotsis replaced, because the old slice remains valid in memory. However, if you intend to see up-to-date slot changes (e.g., new growth or flush) in the same loop, reloading might be necessary. Confirm that reloading mid-loop is not required for your use case.internal/net/grpc/option_test.go (20)
532-615: Function renamed with parameter type changeThe test function has been renamed from
TestWithDialOptionstoTestWithBackoffMaxDelay, and its parameter type has changed from[]grpc.DialOptiontodur string. This change aligns the test function with the specific option being tested, making the test suite more focused and clear.
617-700: Renamed test function with consistent structureThe test function has been renamed from
TestWithBackoffMaxDelaytoTestWithBackoffBaseDelaywhile maintaining the parameter typedur string. This renaming approach creates a more accurate mapping between test functions and the GRPC options they test.
702-785: Parameter type change in renamed test functionThe test function has been renamed from
TestWithBackoffBaseDelaytoTestWithBackoffMultiplierwith the parameter type changing fromdur stringtom float64. This change reflects the appropriate data type for the multiplier parameter.
787-870: Appropriate parameter naming in renamed testThe test function has been renamed from
TestWithBackoffMultipliertoTestWithBackoffJitterwith the parameter type changing fromm float64toj float64. The parameter variable namejproperly matches the jitter concept being tested.
872-955: Parameter type change to duration stringThe test function has been renamed from
TestWithBackoffJittertoTestWithMinConnectTimeoutwith the parameter type changing fromj float64todur string. Using a string for duration provides flexibility in how the time is specified.
957-1040: Complex type parameter introducedThe test function has been renamed from
TestWithMinConnectTimeouttoTestWithErrGroupwith the parameter type changing fromdur stringtoerrgroup.Group. This moves the test to use a more complex type rather than a primitive.
1042-1125: Parameter type changed to match option functionalityThe test function has been renamed from
TestWithErrGrouptoTestWithBackoffwith the parameter type changing fromerrgroup.Grouptobackoff.Backoff. This aligns the test parameter with the actual type used in the backoff implementation.
1127-1210: Specific circuit breaker type introducedThe test function has been renamed from
TestWithCircuitBreakertoTestWithCallOptionswith the parameter type changing fromcircuitbreaker.CircuitBreakerto[]grpc.CallOption. This change reflects a shift in testing focus.
1212-1295: More specific option testing approachThe test function has been renamed from
TestWithCallOptionstoTestWithCallContentSubtypewith the parameter type changing from[]grpc.CallOptiontocontentSubtype string. This represents a more granular approach to testing individual option properties.
1297-1380: Parameter type reflects string contentThe test function has been renamed from
TestWithCallContentSubtypetoTestWithMaxRecvMsgSizewith the parameter type changing fromcontentSubtype stringtosize int. This appropriately changes the parameter type to match the size integer required.
1382-1465: Consistent size parameter maintainedThe test function has been renamed from
TestWithMaxRecvMsgSizetoTestWithMaxSendMsgSizewhile maintaining the parameter typesize int. This consistent parameter type is appropriate as both options deal with message sizes.
1467-1550: Consistent size parameter for buffer testingThe test function has been renamed from
TestWithMaxSendMsgSizetoTestWithMaxRetryRPCBufferSizewhile maintaining the parameter typesize int. This maintains consistency in using integer parameters for size-related options.
1552-1720: Parameter type change for boolean flagThe test function has been renamed from
TestWithMaxRetryRPCBufferSizetoTestWithWaitForReadywith the parameter type changing fromsize inttoflg bool. The parameter type appropriately matches the boolean nature of the wait-for-ready flag.
1722-1805: Function renamed with parameter type preservedThe function has been renamed from
TestWithWaitForReadytoTestWithDialOptionswith the parameter type changing fromflg boolto[]grpc.DialOption. This change completes the reorganization cycle of test functions to better match their implementations.
1977-1980: Integer type specificationThe parameter type for
TestWithInitialWindowSizehas been specified asint32, which is more precise than just usingint. This accurately reflects the type expected by the gRPC library for window size configurations.
2063-2065: Consistent type specification for window sizeThe parameter type for
TestWithInitialConnectionWindowSizehas been specified asint32, consistent with the window size parameter inTestWithInitialWindowSize. This maintains consistency across related configuration parameters.
2408-2412: Network parameter added to dialerThe parameter list for
TestWithDialerhas been updated to includenetwork stringbefore the dialer parameter. This more accurately reflects the actual API which requires specifying the network type.
2488-2488: Updated function signature to include network parameterThe function call has been updated to
WithDialer(test.args.network, test.args.der)to include the network parameter, making the test align properly with the actual implementation.
3261-3338: Renamed for clarity and consistencyThe test function has been renamed from
TestWithOldConnCloseDurationtoTestWithOldConnCloseDelaywhile maintaining the parameter typedur string. This renaming provides better consistency with other function names in the codebase.
2581-3174: New test functions added to improve coverageSeveral new test functions have been added, including
TestWithAuthority,TestWithDisableRetry,TestWithIdleTimeout, and others. These additions enhance the test coverage for gRPC connection options.While these tests follow the same structure as existing tests, most of them (like the others) are still commented out with TODO markers. Consider implementing these tests as part of this PR or creating follow-up tickets to ensure they don't remain as TODOs indefinitely.
| // Copyright (C) 2019-2025 vdaas.org vald team <[email protected]> | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // You may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // https://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| package quic | ||
|
|
There was a problem hiding this comment.
Newly added file needs complete implementation
This file appears to be a placeholder for QUIC connection tests with no actual implemented test code (everything is commented out). While the structure is well-defined, all code is preceded by a "NOT IMPLEMENTED BELOW" comment.
You should uncomment and implement the test functions, along with adding necessary imports:
package quic
import (
"context"
"crypto/tls"
"net"
"reflect"
"sync"
"testing"
"go.uber.org/goleak"
"github.com/quic-go/quic-go" // or appropriate quic package
"github.com/vdaas/vald/internal/errors" // based on custom Errorf usage
)| err := f.CreateIndex(ctx) | ||
| if err != nil && | ||
| !errors.Is(err, errors.ErrUncommittedIndexNotFound) && | ||
| !errors.Is(err, context.Canceled) && | ||
| !errors.Is(err, context.DeadlineExceeded) { | ||
| if errors.IsNot(err, errors.ErrUncommittedIndexNotFound, context.Canceled, context.DeadlineExceeded) { | ||
| return err |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Consider adding tests for these error handling code paths.
Static analysis indicates that these error handling code paths are not covered by tests. Consider adding tests to ensure the refactored error handling logic works correctly.
Also applies to: 1246-1248, 1260-1261, 1268-1269
🏁 Script executed:
#!/bin/bash
# Check current test coverage for the faiss service
echo "Checking test coverage for faiss service error handling..."
rg -A 3 "Test.*CreateAndSaveIndex|Test.*Close" pkg/agent/core/faiss/ --type goLength of output: 1430
Error Handling Tests Coverage Incomplete
The static analysis points out that the error handling paths in pkg/agent/core/faiss/service/faiss.go (lines 1125–1127, 1246–1248, 1260–1261, 1268–1269) aren’t explicitly exercised in the current tests. While tests exist for related routines (e.g., in faiss_test.go, grpc/index_test.go, and rest/handler_test.go covering CreateAndSaveIndex and Close), none of them simulate error scenarios such as errors.ErrUncommittedIndexNotFound, context.Canceled, or context.DeadlineExceeded conditions.
Please consider adding dedicated tests that trigger these error cases to ensure that the refactored error handling logic is thoroughly validated.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
kpango
force-pushed
the
feature/test-e2e/add-e2e-test-v2-strategic-testing
branch
3 times, most recently
from
6afe62e to
cda32d6
Compare
Signed-off-by: kpango <[email protected]>
kpango
force-pushed
the
feature/test-e2e/add-e2e-test-v2-strategic-testing
branch
from
cda32d6 to
5247196
Compare
|
Caution Review failedAn error occurred during the review process. Please try again later. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
1 similar comment
|
Caution Review failedAn error occurred during the review process. Please try again later. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 13
🔭 Outside diff range comments (2)
pkg/index/job/deletion/service/options_test.go (1)
1-357: 🛠️ Refactor suggestionUncomment and implement at least one test case
To improve coverage in the new E2E testing framework, consider uncommenting and implementing at least one test case for each function. This would provide immediate value while the rest of the test cases are being developed.
For example, you could implement a basic test for
WithDiscoverer:func TestWithDiscoverer(t *testing.T) { type args struct { client discoverer.Client } type want struct { want Option } type test struct { name string args args want want checkFunc func(want, Option) error beforeFunc func(*testing.T, args) afterFunc func(*testing.T, args) } defaultCheckFunc := func(w want, got Option) error { if !reflect.DeepEqual(got, w.want) { return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) } return nil } // Mock discoverer client mockClient := &mockDiscoverer{} tests := []test{ { name: "Test with nil client", args: args{ client: nil, }, want: want{ want: func(c *Config) { c.Discoverer = nil }, }, checkFunc: defaultCheckFunc, }, { name: "Test with valid client", args: args{ client: mockClient, }, want: want{ want: func(c *Config) { c.Discoverer = mockClient }, }, checkFunc: defaultCheckFunc, }, } for _, tc := range tests { test := tc t.Run(test.name, func(tt *testing.T) { tt.Parallel() defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) if test.beforeFunc != nil { test.beforeFunc(tt, test.args) } if test.afterFunc != nil { defer test.afterFunc(tt, test.args) } checkFunc := test.checkFunc if test.checkFunc == nil { checkFunc = defaultCheckFunc } got := WithDiscoverer(test.args.client) if err := checkFunc(test.want, got); err != nil { tt.Errorf("error = %v", err) } }) } } // mockDiscoverer implements the discoverer.Client interface type mockDiscoverer struct{}internal/test/data/vector/noise/option_test.go (1)
188-272: 🛠️ Refactor suggestionVerify min table size handling
TestWithMinTableSizerequires actual test cases, including edge conditions (like zero or negative if possible) and large values. Confirm that theOptionproperly enforces the desired minimal noise table size.
♻️ Duplicate comments (14)
internal/timeutil/time_test.go (2)
184-263:⚠️ Potential issueImplement TestDurationString_Duration function
The test function is structured well but remains commented out. This test is needed to ensure the
Duration()method works correctly for theDurationStringtype.Based on the structure of the existing
TestParsefunction, you should implement test cases for:
- Valid duration strings (e.g., "10s", "5m", "2h")
- Invalid format handling
- Empty string handling
-// -// func TestDurationString_Duration(t *testing.T) { +func TestDurationString_Duration(t *testing.T) { -// type want struct { + type want struct { -// want time.Duration + want time.Duration -// err error + err error -// } + }And continue uncommenting and implementing with actual test cases to replace the TODOs.
265-349:⚠️ Potential issueImplement TestDurationString_DurationWithDefault function
This test function is well-structured but remains commented out. The
DurationWithDefaultmethod needs proper test coverage to ensure it functions correctly.Based on the existing
TestParseWithDefaultfunction, you should implement test cases that verify:
- Valid duration string processing
- Default value usage when the string is empty
- Default value usage when the string is invalid
-// -// func TestDurationString_DurationWithDefault(t *testing.T) { +func TestDurationString_DurationWithDefault(t *testing.T) { -// type args struct { + type args struct { -// def time.Duration + def time.Duration -// } + }And continue uncommenting and implementing the function with actual test cases to replace the TODOs.
internal/info/info_test.go (1)
996-1094: 🛠️ Refactor suggestionTestStackTrace_ShortString implementation addresses previous review feedback
This new test function addresses the previous review comment about the untested
ShortString()method. However, the actual test cases are still marked as TODOs and not implemented yet.The
ShortString()method is now being tested in line 1088, which addresses the previous review comment from commit eccd001 that highlighted this missing test coverage.While the test function structure is in place, you should implement concrete test cases to verify the method's behavior.
Consider adding at least these two test cases:
tests := []test{ - // TODO test cases - /* - { - name: "test_case_1", - fields: fields { - URL:"", - FuncName:"", - File:"", - Line:0, - }, - want: want{}, - checkFunc: defaultCheckFunc, - beforeFunc: func(t *testing.T,) { - t.Helper() - }, - afterFunc: func(t *testing.T,) { - t.Helper() - }, - }, - */ + { + name: "normal_case", + fields: fields{ + URL: "https://example.com", + FuncName: "TestFunction", + File: "example.go", + Line: 42, + }, + want: want{ + want: "https://example.com TestFunction", + }, + checkFunc: defaultCheckFunc, + }, + { + name: "empty_values", + fields: fields{ + URL: "", + FuncName: "", + File: "", + Line: 0, + }, + want: want{ + want: " ", + }, + checkFunc: defaultCheckFunc, + },internal/servers/server/option_test.go (1)
3727-4066:⚠️ Potential issueImplement the commented-out test functions for GRPC options
The file still contains several commented-out test function stubs for GRPC-related server options that should be implemented to improve test coverage:
TestWithGRPCMaxConcurrentStreamsTestWithGRPCNumStreamWorkersTestWithGRPCSharedWriteBufferTestWithGRPCWaitForHandlersThese tests are particularly important as they're part of the new E2E strategic testing framework mentioned in your PR objectives.
I recommend implementing these tests following the pattern of other tests in the file. For example:
func TestWithGRPCMaxConcurrentStreams(t *testing.T) { type test struct { name string size uint32 checkFunc func(opt Option) error } tests := []test{ { name: "set success when size is more than 0", size: 1024, checkFunc: func(opt Option) error { got := new(server) opt(got) if len(got.grpc.opts) != 1 { return errors.New("invalid param was set") } return nil }, }, { name: "not set when size is 0", checkFunc: func(opt Option) error { gopts := []grpc.ServerOption{ grpc.ConnectionTimeout(10 * time.Second), } got := new(server) got.grpc.opts = gopts opt(got) if !reflect.DeepEqual(got.grpc.opts, gopts) { return errors.New("invalid param was set") } return nil }, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { opt := WithGRPCMaxConcurrentStreams(tt.size) if err := tt.checkFunc(opt); err != nil { t.Error(err) } }) } }internal/params/option_test.go (1)
451-705: 🛠️ Refactor suggestionImplement or remove these commented test templates
The file contains three commented-out test functions (TestWithName, TestWithOverrideDefault, and TestWithArgumentFilters) that are not contributing to code coverage or validation of the codebase. These test templates should either be implemented with actual test cases or removed entirely to avoid code clutter.
Here's an example implementation for the first function to get you started:
-// -// func TestWithName(t *testing.T) { +func TestWithName(t *testing.T) { -// type args struct { + type args struct { -// name string + name string -// } + } -// type want struct { + type want struct { -// want Option + want Option -// } + } -// type test struct { + type test struct { -// name string + name string -// args args + args args -// want want + want want -// checkFunc func(want, Option) error + checkFunc func(want, Option) error -// beforeFunc func(*testing.T, args) + beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) + afterFunc func(*testing.T, args) -// } + } -// defaultCheckFunc := func(w want, got Option) error { + defaultCheckFunc := func(w want, got Option) error { -// if !reflect.DeepEqual(got, w.want) { + if !reflect.DeepEqual(got, w.want) { -// return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) + return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) -// } + } -// return nil + return nil -// } + } -// tests := []test{ + tests := []test{ + { + name: "set name successfully", + args: args { + name: "test-name", + }, + want: want{ + want: func(p *parser) { + p.name = "test-name" + }, + }, + }, + { + name: "set empty name", + args: args { + name: "", + }, + want: want{ + want: func(p *parser) { + p.name = "" + }, + }, + }, -// // TODO test cases -// /* -// { -// name: "test_case_1", -// args: args { -// name:"", -// }, -// want: want{}, -// checkFunc: defaultCheckFunc, -// beforeFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// afterFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// }, -// */ -// -// // TODO test cases -// /* -// func() test { -// return test { -// name: "test_case_2", -// args: args { -// name:"", -// }, -// want: want{}, -// checkFunc: defaultCheckFunc, -// beforeFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// afterFunc: func(t *testing.T, args args) { -// t.Helper() -// }, -// } -// }(), -// */ -// } + } -// + -// for _, tc := range tests { + for _, tc := range tests { -// test := tc + test := tc -// t.Run(test.name, func(tt *testing.T) { + t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() + tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { + if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) + test.beforeFunc(tt, test.args) -// } + } -// if test.afterFunc != nil { + if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) + defer test.afterFunc(tt, test.args) -// } + } -// checkFunc := test.checkFunc + checkFunc := test.checkFunc -// if test.checkFunc == nil { + if test.checkFunc == nil { -// checkFunc = defaultCheckFunc + checkFunc = defaultCheckFunc -// } + } -// + -// got := WithName(test.args.name) + got := WithName(test.args.name) -// if err := checkFunc(test.want, got); err != nil { + if err := checkFunc(test.want, got); err != nil { -// tt.Errorf("error = %v", err) + tt.Errorf("error = %v", err) -// } + } -// }) + }) -// } + } -// } +}Similarly, you should implement or remove the other test functions as well.
internal/test/mock/server_stream.go (1)
40-53:⚠️ Potential issueFix error variable shadowing and ensure test coverage
There's a variable shadowing issue in line 46 - the
:=declaration creates a new localerrvariable that shadows the named return parameter, which could lead to unexpected behavior.Fix the declaration by using
=instead of:=:- err := m.ServerStream.RecvMsg(res) + err = m.ServerStream.RecvMsg(res)As previously noted, these changes should be validated with tests. Consider adding test cases that verify:
- Behavior when
mis nil- Behavior when
RecvFuncis nil (fallback toServerStream.RecvMsg)- Error handling when
RecvMsgreturns an error🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by testsinternal/net/quic/conn_test.go (6)
1-15:⚠️ Potential issueImplementation needed for QUIC connection tests
The file contains a proper license header and package declaration but has no active code. This test file is a crucial part of the E2E testing framework mentioned in the PR objectives, but it's entirely commented out and prefixed with "NOT IMPLEMENTED BELOW".
Uncomment and implement this test file with the following required imports:
package quic import ( "context" "crypto/tls" "net" "reflect" "sync" "testing" "go.uber.org/goleak" "github.com/quic-go/quic-go" // or appropriate quic package "github.com/vdaas/vald/internal/errors" // for errors.Is and errors.Errorf )
44-83: 🛠️ Refactor suggestionImplement actual test cases for NewConn function
The test structure is well-defined but contains only placeholder TODO comments. Given that this PR focuses on improving test infrastructure, implementing actual tests is essential.
Implement concrete test cases to validate both successful and error scenarios for the
NewConnfunction. For example:tests := []test{ { name: "successful_connection", args: args{ ctx: context.Background(), conn: &mockQuicConnection{}, }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: &mockQuicStream{}, }, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "nil_connection", args: args{ ctx: context.Background(), conn: nil, }, want: want{ want: nil, err: errors.New("connection is required"), }, checkFunc: defaultCheckFunc, }, }
110-200: 🛠️ Refactor suggestionImplement test cases for Conn_Close function
Tests for closing QUIC connections are crucial to prevent resource leaks. The test structure is defined but lacks actual test cases.
Implement test cases to validate connection closing behavior, including:
- Successful closing of both Connection and Stream
- Error handling when Stream closure fails
- Error handling when Connection closure fails
- Testing the behavior when either component is nil
tests := []test{ { name: "successful_close", fields: fields{ Connection: &mockQuicConnection{ closeFunc: func() error { return nil }, }, Stream: &mockQuicStream{ closeFunc: func() error { return nil }, }, }, want: want{ err: nil, }, checkFunc: defaultCheckFunc, }, { name: "stream_close_error", fields: fields{ Connection: &mockQuicConnection{ closeFunc: func() error { return nil }, }, Stream: &mockQuicStream{ closeFunc: func() error { return errors.New("stream close error") }, }, }, want: want{ err: errors.New("stream close error"), }, checkFunc: defaultCheckFunc, }, }
297-403: 🛠️ Refactor suggestionImplement test cases for qconn_dialQuicContext function
This function tests advanced QUIC dial logic including connection caching, but contains no actual test cases.
Implement test cases to validate:
- Connection caching behavior (retrieving existing connections)
- Creating new connections when not in cache
- Error handling for invalid addresses or TLS configurations
- Context cancellation behavior
tests := []test{ { name: "new_connection_success", args: args{ ctx: context.Background(), addr: "example.com:443", tcfg: &tls.Config{}, }, fields: fields{ connectionCache: sync.Map[string, quic.Connection]{}, }, want: want{ want: &Conn{/* expected fields */}, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "cached_connection_retrieval", args: args{ ctx: context.Background(), addr: "example.com:443", tcfg: &tls.Config{}, }, fields: fields{ // Pre-populate the connection cache // This requires initialization in beforeFunc }, want: want{ want: &Conn{/* expected fields */}, err: nil, }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T, args args) { // Setup code to populate cache }, }, }
405-491: 🛠️ Refactor suggestionImplement test cases for qconn_Close function
The
qconn_Closefunction needs to properly close all cached connections, but the test doesn't contain any actual test cases.Implement test cases that verify:
- All connections in cache are properly closed
- Error handling when connections fail to close
- Behavior with an empty connection cache
tests := []test{ { name: "close_all_connections_success", fields: fields{ // Pre-populate the connection cache with mock connections // This requires initialization in beforeFunc }, want: want{ err: nil, }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T) { // Setup code to populate cache with successful close funcs }, }, { name: "connection_close_error", fields: fields{ // Pre-populate with connection that will error on close // This requires initialization in beforeFunc }, want: want{ err: errors.New("close error"), }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T) { // Setup code to populate cache with error-producing close func }, }, }
471-476: 🛠️ Refactor suggestionCreate mock implementations for QUIC interfaces
For effective testing, you'll need mock implementations of the QUIC interfaces.
Create mock implementations for
quic.Connectionandquic.Streaminterfaces:type mockQuicConnection struct { closeFunc func() error openStreamFunc func(context.Context) (quic.Stream, error) // other methods required by the quic.Connection interface } func (m *mockQuicConnection) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } func (m *mockQuicConnection) OpenStream() (quic.Stream, error) { if m.openStreamFunc != nil { return m.openStreamFunc(context.Background()) } return &mockQuicStream{}, nil } // Implement other required methods type mockQuicStream struct { closeFunc func() error readFunc func([]byte) (int, error) writeFunc func([]byte) (int, error) // other methods required by the quic.Stream interface } func (m *mockQuicStream) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } // Implement other required methodsinternal/net/grpc/status/status_test.go (1)
245-331: 🛠️ Refactor suggestionUncomment and implement tests for the
IsfunctionThis test function for the
Isfunction is structured properly but is entirely commented out, leaving a gap in test coverage. TheIsfunction (checking if an error corresponds to a specific gRPC status code) is an important utility that should be thoroughly tested.Uncomment the test function and implement appropriate test cases. Here's a suggested implementation:
-// func TestIs(t *testing.T) { +func TestIs(t *testing.T) { type args struct { err error code codes.Code } type want struct { want bool } type test struct { name string args args want want checkFunc func(want, bool) error beforeFunc func(*testing.T, args) afterFunc func(*testing.T, args) } defaultCheckFunc := func(w want, got bool) error { if !reflect.DeepEqual(got, w.want) { return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) } return nil } tests := []test{ - // TODO test cases - /* - { - name: "test_case_1", - args: args { - err:nil, - code:nil, - }, - want: want{}, - checkFunc: defaultCheckFunc, - beforeFunc: func(t *testing.T, args args) { - t.Helper() - }, - afterFunc: func(t *testing.T, args args) { - t.Helper() - }, - }, - */ + { + name: "nil error returns false", + args: args{ + err: nil, + code: codes.OK, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "non-status error returns false", + args: args{ + err: errors.New("regular error"), + code: codes.Internal, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "status error with matching code returns true", + args: args{ + err: Error(codes.NotFound, "not found"), + code: codes.NotFound, + }, + want: want{ + want: true, + }, + checkFunc: defaultCheckFunc, + }, + { + name: "status error with different code returns false", + args: args{ + err: Error(codes.NotFound, "not found"), + code: codes.Internal, + }, + want: want{ + want: false, + }, + checkFunc: defaultCheckFunc, + }, } for _, tc := range tests { test := tc t.Run(test.name, func(tt *testing.T) { tt.Parallel() defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) if test.beforeFunc != nil { test.beforeFunc(tt, test.args) } if test.afterFunc != nil { defer test.afterFunc(tt, test.args) } checkFunc := test.checkFunc if test.checkFunc == nil { checkFunc = defaultCheckFunc } got := Is(test.args.err, test.args.code) if err := checkFunc(test.want, got); err != nil { tt.Errorf("error = %v", err) } }) } -// } +}internal/iter/iter.go (1)
52-59: 🛠️ Refactor suggestionReturn a no-op cycle for empty slices rather than
nil.Returning
nilhere forces callers to manually check fornilbefore calling other methods, risking potential panics. Consider returning a valid “no-op” cycle instead to streamline usage.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
🧹 Nitpick comments (31)
k8s/agent/ngt/configmap.yaml (1)
257-257: Enable Statistics Flag UpdateThe configuration now enables statistics collection by setting
enable_statisticstotrue. Please verify that this change is aligned with your overall observability strategy and that the corresponding Helm charts or other environment-specific configurations are updated accordingly.apis/swagger/v1/vald/filter.swagger.json (2)
177-178: Expanded Details for/update/object/multipleEndpoint
The updated summary and the new description now provide a caution regarding gRPC’s message size limitations. Please verify that the embedded HTML snippet (<div class="notice">…</div>) renders properly in your Swagger UI.
242-244: Expanded Summary for/upsert/object/multipleEndpoint
The revised summary and description clearly articulate that multiple objects can be upserted in one request, and include a warning about message size limitations. Similar to the update/multiple endpoint, ensure that the HTML snippet renders as expected in your Swagger viewer.internal/errors/k8s.go (2)
43-45: Fix inconsistent error message capitalization.The error messages for
ErrUndefinedNamespaceandErrUndefinedServicestart with uppercase "Undefined", while all other error messages in this file start with lowercase. This inconsistency should be fixed to maintain a uniform style.- ErrUndefinedNamespace = New("Undefined namespace") + ErrUndefinedNamespace = New("undefined namespace") - ErrUndefinedService = New("Undefined service") + ErrUndefinedService = New("undefined service")
20-46: Consider adding documentation comments for error variables.These new error variables would benefit from documentation comments explaining when each error occurs, providing context for developers who might encounter these errors. This is especially important for exported variables that are part of the public API.
For example:
var ( + // ErrInvalidReconcilerConfig is returned when the reconciler configuration is invalid ErrInvalidReconcilerConfig = New("invalid reconciler config") + // ErrPodIsNotRunning is returned when attempting to interact with a pod that is not in running state ErrPodIsNotRunning = func(namespace, name string) error { return Errorf("pod %s/%s is not running", namespace, name) }apis/swagger/v1/vald/upsert.swagger.json (2)
17-18: Enhanced Documentation for the/upsertEndpoint
The updated "summary" and "description" fields now provide a clear and structured overview that includes a status code table and troubleshooting guidelines. This increased detail will help API consumers understand the endpoint's behavior and potential error scenarios.
Please verify that the markdown formatting (including the table separators and line breaks) renders correctly in all your Swagger UI or documentation tooling setups.
50-51: Improved Clarity for the/upsert/multipleEndpoint
The revised "summary" now succinctly describes the MultiUpsert operation, and the enriched "description" adds valuable context—including a warning about gRPC message size limitations and a troubleshooting table.
Ensure that any embedded HTML elements (such as the<div>tag) and markdown formatting render as intended in your chosen documentation viewer.internal/params/params.go (2)
128-134: Fixed path handling and usage display logicUsing absolute path for config file path and improving the condition check for when to display usage. The condition on line 131 can be simplified for better readability.
- if exist, _, err := file.ExistsWithDetail(d.configFilePath); !d.showVersion && - (!exist || d.configFilePath == "") { + if !d.showVersion && (d.configFilePath == "" || !exist) {
139-149: Added flag management methodsGood addition of
RestoreandOverridemethods that provide control over the global flag state. This is useful for testing scenarios where isolation is needed.Consider adding documentation comments to explain the purpose and effects of these methods, especially since they affect global state.
apis/swagger/v1/vald/search.swagger.json (7)
17-18: Enhanced/linearsearchEndpoint DocumentationThe updated summary and description now provide a comprehensive overview including explicit status codes and troubleshooting details. Please verify that the markdown formatting (e.g., table syntax and line breaks) is rendered correctly by your API documentation tools.
50-51: Improved/linearsearch/idDocumentationThe expanded summary and description now clearly state how searching by vector ID works along with detailed error-handling instructions. Please check that the mixed use of markdown and escaped HTML (e.g.,
\u003cbr\u003e) renders consistently across documentation viewers.
83-84: Enhanced/linearsearch/id/multipleEndpoint DetailsThe new summary succinctly explains multi-ID searching and the description effectively highlights the gRPC message size limitation with a dedicated notice section and enriched troubleshooting table. Confirm that the embedded HTML snippet (i.e.
<div class="notice">) is supported by your documentation renderer.
116-117: Clarified/linearsearch/multipleEndpoint InformationThe modified summary and description clearly outline the functionality for multi-vector search requests. Ensure that the markdown elements and any embedded HTML (if applicable) are consistently formatted as in other endpoints.
182-183: Refined/search/idEndpoint DetailsThe updated summary and description deliver clear instructions for search-by-ID requests, including prerequisites for vector indexing. Verify that the use of
<br>(escaped as\u003cbr\u003e) and markdown formatting appear as intended.
215-216: Better Articulated/search/id/multipleDocumentationThe changes effectively consolidate the information for multi-ID search requests, including the gRPC limitation notice and a detailed error troubleshooting table. Ensure consistency with similar multi-search endpoints.
248-249: Revised/search/multipleEndpoint DocumentationThe updated summary and descriptive content now clearly specify the multi-vector search capabilities along with associated troubleshooting details. Please confirm that all markdown and embedded HTML elements are rendered consistently across your documentation tools.
.github/workflows/codeql-analysis.yml (1)
46-57: Enhanced 'analyze' Job Configuration and Dynamic Environment Settings
The updated job definition foranalyzenow dynamically selects the runner (macos-latestfor Swift, otherwiseubuntu-latest) and introduces two new environment variables:CODEQL_ENABLE_EXPERIMENTAL_FEATURESandCODEQL_ACTION_TEST_MODE. This configuration enhances flexibility and testing capability. Please ensure that these experimental flags are well documented for future maintainers so they understand the implications of enabling these features.pkg/index/job/deletion/service/options_test.go (2)
1-15: Files containing only commented-out code should be properly marked as a TODO or WIP.This file contains test skeletons that are entirely commented out. While creating templates for future implementation is a good practice, it would be better to:
- Add a clear TODO header comment explaining why the tests are commented out
- Add a JIRA ticket or issue reference for tracking implementation
- Consider adding at least one basic test case that can be executed
package service +// TODO: Implement test cases for the service options. +// This file contains test skeletons that need to be implemented as part of the +// E2E strategic testing framework. +// Issue: <add-issue-reference-here> // NOT IMPLEMENTED BELOW
1-357: Missing imports for the test fileThe commented-out test functions would require imports that are currently missing from the file.
Add the required imports at the top of the file:
package service +import ( + "reflect" + "testing" + + "github.com/vdaas/vald/internal/errors" + "github.com/vdaas/vald/internal/test/goleak" + "github.com/vdaas/vald/internal/net/grpc/client/discoverer" +)pkg/index/job/deletion/service/deleter_test.go (1)
16-17: Remove misleading comment.The "NOT IMPLEMENTED BELOW" comment should be removed once you implement the tests. This comment suggests the file is a template or skeleton, which is not appropriate for a PR introducing new functionality.
apis/docs/v1/docs.md (2)
1373-1377: Mirror Service Table Formatting and Consistency:
The updated Mirror service section now provides an "Overview" for the Register RPC along with a table that details the method name, request/response types, description, and status code information. However, the table appears to have a very dense set of columns and extra inline status code details that might not render well in all Markdown viewers. Please verify the rendering of the table and consider simplifying or splitting the status code details into an accompanying explanatory text if needed.
1636-1639: Filter Service Overview Clarity:
The updated overview for the Filter service indicates that the service “provides insert, update, upsert and search interface forVald Filter Gateway” and that it “forward[s] user request to user-defined ingress/egress filter components allowing user to run custom logic.” For enhanced clarity and grammatical consistency, consider revising it as: “Filter Server is responsible for providing interfaces for insert, update, upsert, and search operations for theVald Filter Gateway. It forwards user requests to user-defined ingress/egress filter components, thereby enabling the execution of custom logic.”internal/test/data/vector/noise/noise.go (1)
180-193: Consider the performance impact of cloning
Cloning both the noise table (line 181) and the input vector (line 185) can be expensive for large inputs. If immutability is not strictly required, allowing in-place modifications or reusing shared structures with caution might improve performance.internal/net/quic/conn_test.go (1)
16-16: Remove "NOT IMPLEMENTED BELOW" comment when implementing testsThis comment indicates that the file is not yet ready for production use, which is inconsistent with the PR objective of adding a comprehensive E2E testing framework.
When implementing the tests, remove this comment to indicate that the file is fully implemented.
pkg/agent/core/ngt/service/ngt.go (1)
929-929: Improved error handling witherrors.IsNotThe code now uses the
errors.IsNotfunction to check if an error is not of the specified type, which simplifies the previously verbose conditional check.Consider applying this pattern consistently throughout the codebase for improved readability and maintainability.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by testsinternal/sync/errgroup/group_test.go (1)
1172-1287: Commented out redundant test implementationThe entire
Test_group_TryGofunction has been commented out, which is appropriate as it's being replaced by theTest_group_execfunction. However, ifTryGois still part of the public API, consider keeping tests for both functions.Consider removing commented-out code if it's no longer needed, or add a comment explaining why it's being kept for reference.
internal/net/grpc/stream_test.go (1)
202-296: Consider implementing the commented testThis test function remains commented out, which suggests it's not actively being used. If the corresponding
BidirectionalStreamClientfunction is being used elsewhere, consider implementing this test to ensure its correctness.-// NOT IMPLEMENTED BELOW -// -// func TestBidirectionalStreamClient(t *testing.T) { +func TestBidirectionalStreamClient(t *testing.T) {Add actual test cases that cover different scenarios such as:
- Successful streaming with different concurrency levels
- Error handling
- Context cancellation
internal/net/grpc/pool/pool.go (1)
694-695: Consider implementing IP address sortingThere's a commented line about sorting IP addresses, but no implementation. Sorting IPs based on criteria such as latency or historical connection success could improve connection reliability and performance.
Consider implementing a sorting mechanism for IP addresses based on factors such as:
- Historical connection success rate
- Measured latency (RTT)
- Geographic proximity (if known)
This would help prioritize more reliable or performant connections.
internal/file/file.go (1)
374-391: Consider simplifying AbsolutePath implementationThe new
AbsolutePathfunction follows multiple paths to compute an absolute path, which adds complexity. It first joins with the current working directory, then falls back tofilepath.Absif that doesn't work.func AbsolutePath(path string) string { if path == "" { return "" } - if !filepath.IsAbs(path) { - root, err := os.Getwd() - if err == nil { - path = joinFilePaths(root, path) - } - if !filepath.IsAbs(path) { - absPath, err := filepath.Abs(path) - if err == nil { - path = absPath - } - } - } + if !filepath.IsAbs(path) { + absPath, err := filepath.Abs(path) + if err == nil { + path = absPath + } + } return filepath.Clean(path) }Using
filepath.Absdirectly is simpler and more predictable, as it already handles joining with the current working directory for relative paths.apis/swagger/v1/vald/insert.swagger.json (2)
50-50: Confirm Markdown Support in Summary for/insert/multipleThe
/insert/multipleendpoint summary uses Markdown formatting (i.e. 1) for emphasis. Please confirm that your API documentation tool supports such markdown styling. If not, consider using plain text for clarity.
51-51: HTML Markup and Consistency Check in/insert/multipleDescriptionThe description begins with an HTML
<div class="notice">block to caution users about gRPC message size limitations. Please verify that this HTML markup renders as intended in your API documentation tool. Additionally, ensure that the detailed error status codes and their troubleshooting advice match the actual endpoint behavior.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
apis/grpc/v1/agent/core/agent.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/agent/sidecar/sidecar.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/discoverer/discoverer.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/egress/egress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/ingress/ingress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/meta/meta.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/payload/payload.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/rpc/errdetails/error_details.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goexample/client/go.sumis excluded by!**/*.sumgo.sumis excluded by!**/*.sumrust/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (153)
.gitfiles(15 hunks).github/ISSUE_TEMPLATE/bug_report.md(1 hunks).github/ISSUE_TEMPLATE/security_issue_report.md(1 hunks).github/PULL_REQUEST_TEMPLATE.md(1 hunks).github/workflows/_docker-image.yaml(1 hunks).github/workflows/_release-pr.yaml(1 hunks).github/workflows/build-binaries.yaml(1 hunks).github/workflows/build-protobuf.yaml(1 hunks).github/workflows/chatops.yaml(5 hunks).github/workflows/codeql-analysis.yml(2 hunks).github/workflows/coverage.yaml(1 hunks).github/workflows/e2e-chaos.yaml(4 hunks).github/workflows/e2e-code-bench-agent.yaml(2 hunks).github/workflows/e2e-max-dim.yaml(1 hunks).github/workflows/e2e-profiling.yaml(1 hunks).github/workflows/e2e.yaml(8 hunks).github/workflows/format.yaml(3 hunks).github/workflows/fossa.yaml(1 hunks).github/workflows/helm-lint.yaml(3 hunks).github/workflows/helm.yaml(2 hunks).github/workflows/reviewdog-k8s.yaml(3 hunks).github/workflows/reviewdog.yaml(2 hunks).github/workflows/test-hack.yaml(1 hunks).github/workflows/unit-test-rust.yaml(1 hunks).github/workflows/unit-test.yaml(3 hunks).github/workflows/update-deps.yaml(1 hunks)Makefile(8 hunks)Makefile.d/e2e.mk(1 hunks)Makefile.d/functions.mk(1 hunks)Makefile.d/minikube.mk(1 hunks)Makefile.d/tools.mk(2 hunks)apis/docs/v1/docs.md(10 hunks)apis/swagger/v1/mirror/mirror.swagger.json(1 hunks)apis/swagger/v1/vald/filter.swagger.json(8 hunks)apis/swagger/v1/vald/flush.swagger.json(1 hunks)apis/swagger/v1/vald/index.swagger.json(5 hunks)apis/swagger/v1/vald/insert.swagger.json(2 hunks)apis/swagger/v1/vald/object.swagger.json(4 hunks)apis/swagger/v1/vald/remove.swagger.json(3 hunks)apis/swagger/v1/vald/search.swagger.json(8 hunks)apis/swagger/v1/vald/update.swagger.json(3 hunks)apis/swagger/v1/vald/upsert.swagger.json(2 hunks)charts/vald/values/dev.yaml(1 hunks)cmd/index/job/deletion/main_test.go(1 hunks)dockers/agent/core/agent/Dockerfile(1 hunks)dockers/agent/core/faiss/Dockerfile(1 hunks)dockers/agent/core/ngt/Dockerfile(1 hunks)dockers/agent/sidecar/Dockerfile(1 hunks)dockers/binfmt/Dockerfile(1 hunks)dockers/buildbase/Dockerfile(1 hunks)dockers/buildkit/Dockerfile(1 hunks)dockers/buildkit/syft/scanner/Dockerfile(1 hunks)dockers/ci/base/Dockerfile(1 hunks)dockers/dev/Dockerfile(1 hunks)dockers/discoverer/k8s/Dockerfile(1 hunks)dockers/example/client/Dockerfile(1 hunks)dockers/gateway/filter/Dockerfile(1 hunks)dockers/gateway/lb/Dockerfile(1 hunks)dockers/gateway/mirror/Dockerfile(1 hunks)dockers/index/job/correction/Dockerfile(1 hunks)dockers/index/job/creation/Dockerfile(1 hunks)dockers/index/job/deletion/Dockerfile(1 hunks)dockers/index/job/readreplica/rotate/Dockerfile(1 hunks)dockers/index/job/save/Dockerfile(1 hunks)dockers/index/operator/Dockerfile(1 hunks)dockers/manager/index/Dockerfile(1 hunks)dockers/operator/helm/Dockerfile(1 hunks)dockers/tools/benchmark/job/Dockerfile(1 hunks)dockers/tools/benchmark/operator/Dockerfile(1 hunks)dockers/tools/cli/loadtest/Dockerfile(1 hunks)example/client/go.mod(2 hunks)example/client/go.mod.default(1 hunks)go.mod(14 hunks)hack/go.mod.default(4 hunks)internal/backoff/backoff.go(2 hunks)internal/backoff/backoff_test.go(1 hunks)internal/backoff/option.go(0 hunks)internal/backoff/option_test.go(0 hunks)internal/client/v1/client/discoverer/discover.go(1 hunks)internal/client/v1/client/discoverer/discover_test.go(5 hunks)internal/config/benchmark_test.go(4 hunks)internal/config/config.go(2 hunks)internal/config/config_test.go(1 hunks)internal/config/grpc.go(1 hunks)internal/config/index_deleter_test.go(1 hunks)internal/core/algorithm/usearch/usearch_test.go(16 hunks)internal/db/nosql/cassandra/option_test.go(8 hunks)internal/errors/errors.go(2 hunks)internal/errors/errors_test.go(1 hunks)internal/errors/k8s.go(1 hunks)internal/errors/params.go(1 hunks)internal/file/file.go(3 hunks)internal/file/file_test.go(1 hunks)internal/info/info_test.go(17 hunks)internal/iter/iter.go(1 hunks)internal/net/dialer_test.go(1 hunks)internal/net/grpc/client.go(13 hunks)internal/net/grpc/errdetails/errdetails.go(6 hunks)internal/net/grpc/metadata_test.go(1 hunks)internal/net/grpc/option.go(1 hunks)internal/net/grpc/option_test.go(46 hunks)internal/net/grpc/pool/option.go(1 hunks)internal/net/grpc/pool/option_test.go(6 hunks)internal/net/grpc/pool/pool.go(6 hunks)internal/net/grpc/pool/pool_bench_test.go(2 hunks)internal/net/grpc/server_test.go(2 hunks)internal/net/grpc/status/status.go(1 hunks)internal/net/grpc/status/status_test.go(1 hunks)internal/net/grpc/stream.go(4 hunks)internal/net/grpc/stream_test.go(4 hunks)internal/net/http/client/client.go(2 hunks)internal/net/http/client/client_test.go(3 hunks)internal/net/http/client/option_test.go(4 hunks)internal/net/net.go(1 hunks)internal/net/net_test.go(3 hunks)internal/net/quic/conn_test.go(1 hunks)internal/net/quic/listener_test.go(1 hunks)internal/params/option.go(2 hunks)internal/params/option_test.go(1 hunks)internal/params/params.go(4 hunks)internal/params/params_test.go(12 hunks)internal/rand/rand.go(1 hunks)internal/runner/runner.go(4 hunks)internal/servers/server/option_test.go(1 hunks)internal/servers/server/server.go(1 hunks)internal/servers/servers.go(3 hunks)internal/sync/errgroup/group.go(7 hunks)internal/sync/errgroup/group_test.go(16 hunks)internal/test/data/vector/gen.go(0 hunks)internal/test/data/vector/noise/noise.go(1 hunks)internal/test/data/vector/noise/noise_test.go(1 hunks)internal/test/data/vector/noise/option.go(1 hunks)internal/test/data/vector/noise/option_test.go(1 hunks)internal/test/mock/server_stream.go(1 hunks)internal/timeutil/time.go(3 hunks)internal/timeutil/time_test.go(1 hunks)k8s/agent/ngt/configmap.yaml(1 hunks)k8s/discoverer/deployment.yaml(1 hunks)k8s/gateway/gateway/lb/deployment.yaml(1 hunks)k8s/gateway/gateway/mirror/deployment.yaml(1 hunks)k8s/index/operator/deployment.yaml(1 hunks)k8s/manager/index/deployment.yaml(1 hunks)pkg/agent/core/faiss/service/faiss.go(3 hunks)pkg/agent/core/ngt/service/ngt.go(5 hunks)pkg/agent/core/ngt/service/ngt_test.go(8 hunks)pkg/agent/core/ngt/service/option_test.go(8 hunks)pkg/discoverer/k8s/service/discover_test.go(4 hunks)pkg/gateway/lb/handler/grpc/aggregation_test.go(2 hunks)pkg/gateway/lb/handler/grpc/handler_test.go(2 hunks)pkg/gateway/mirror/service/mirror.go(2 hunks)pkg/index/job/deletion/config/config_test.go(1 hunks)pkg/index/job/deletion/service/deleter_test.go(1 hunks)pkg/index/job/deletion/service/options_test.go(1 hunks)
💤 Files with no reviewable changes (3)
- internal/backoff/option.go
- internal/test/data/vector/gen.go
- internal/backoff/option_test.go
🚧 Files skipped from review as they are similar to previous changes (115)
- dockers/buildbase/Dockerfile
- dockers/buildkit/syft/scanner/Dockerfile
- .github/workflows/unit-test.yaml
- dockers/manager/index/Dockerfile
- .github/workflows/helm.yaml
- Makefile.d/e2e.mk
- dockers/index/job/save/Dockerfile
- .github/workflows/build-binaries.yaml
- dockers/index/job/deletion/Dockerfile
- dockers/agent/core/agent/Dockerfile
- .github/workflows/_release-pr.yaml
- dockers/index/job/creation/Dockerfile
- .github/workflows/test-hack.yaml
- dockers/gateway/mirror/Dockerfile
- .github/workflows/e2e-code-bench-agent.yaml
- dockers/agent/core/faiss/Dockerfile
- .github/workflows/coverage.yaml
- .github/workflows/build-protobuf.yaml
- .github/workflows/reviewdog.yaml
- internal/errors/params.go
- dockers/discoverer/k8s/Dockerfile
- dockers/example/client/Dockerfile
- .github/workflows/e2e-chaos.yaml
- .github/workflows/e2e-profiling.yaml
- dockers/tools/benchmark/job/Dockerfile
- dockers/agent/sidecar/Dockerfile
- .github/workflows/unit-test-rust.yaml
- dockers/gateway/filter/Dockerfile
- example/client/go.mod.default
- internal/backoff/backoff_test.go
- .github/workflows/update-deps.yaml
- charts/vald/values/dev.yaml
- dockers/ci/base/Dockerfile
- dockers/tools/benchmark/operator/Dockerfile
- dockers/index/operator/Dockerfile
- .github/workflows/_docker-image.yaml
- .github/workflows/e2e-max-dim.yaml
- dockers/tools/cli/loadtest/Dockerfile
- dockers/operator/helm/Dockerfile
- dockers/gateway/lb/Dockerfile
- internal/config/benchmark_test.go
- dockers/buildkit/Dockerfile
- .github/workflows/chatops.yaml
- internal/net/grpc/option.go
- dockers/index/job/correction/Dockerfile
- Makefile.d/minikube.mk
- .github/workflows/fossa.yaml
- k8s/discoverer/deployment.yaml
- .github/workflows/e2e.yaml
- .github/ISSUE_TEMPLATE/bug_report.md
- Makefile.d/tools.mk
- internal/net/grpc/status/status.go
- .github/workflows/format.yaml
- dockers/dev/Dockerfile
- .github/ISSUE_TEMPLATE/security_issue_report.md
- internal/backoff/backoff.go
- dockers/agent/core/ngt/Dockerfile
- internal/rand/rand.go
- .github/workflows/reviewdog-k8s.yaml
- .github/workflows/helm-lint.yaml
- dockers/index/job/readreplica/rotate/Dockerfile
- internal/errors/errors_test.go
- internal/net/grpc/pool/pool_bench_test.go
- internal/config/grpc.go
- pkg/index/job/deletion/config/config_test.go
- pkg/agent/core/ngt/service/ngt_test.go
- internal/timeutil/time.go
- apis/swagger/v1/mirror/mirror.swagger.json
- internal/servers/server/server.go
- pkg/gateway/mirror/service/mirror.go
- apis/swagger/v1/vald/flush.swagger.json
- internal/net/http/client/client.go
- internal/runner/runner.go
- internal/db/nosql/cassandra/option_test.go
- internal/file/file_test.go
- Makefile.d/functions.mk
- internal/net/http/client/option_test.go
- k8s/index/operator/deployment.yaml
- pkg/agent/core/ngt/service/option_test.go
- internal/net/dialer_test.go
- internal/params/option.go
- k8s/manager/index/deployment.yaml
- internal/net/grpc/server_test.go
- cmd/index/job/deletion/main_test.go
- internal/config/index_deleter_test.go
- .github/PULL_REQUEST_TEMPLATE.md
- internal/client/v1/client/discoverer/discover.go
- internal/net/http/client/client_test.go
- internal/params/params_test.go
- k8s/gateway/gateway/mirror/deployment.yaml
- internal/net/quic/listener_test.go
- apis/swagger/v1/vald/object.swagger.json
- internal/net/grpc/metadata_test.go
- internal/net/grpc/pool/option_test.go
- k8s/gateway/gateway/lb/deployment.yaml
- internal/servers/servers.go
- internal/net/net.go
- internal/client/v1/client/discoverer/discover_test.go
- dockers/binfmt/Dockerfile
- internal/net/grpc/pool/option.go
- internal/net/net_test.go
- pkg/discoverer/k8s/service/discover_test.go
- internal/core/algorithm/usearch/usearch_test.go
- pkg/gateway/lb/handler/grpc/aggregation_test.go
- internal/test/data/vector/noise/option.go
- apis/swagger/v1/vald/index.swagger.json
- internal/config/config.go
- pkg/gateway/lb/handler/grpc/handler_test.go
- apis/swagger/v1/vald/update.swagger.json
- Makefile
- internal/sync/errgroup/group.go
- hack/go.mod.default
- internal/net/grpc/client.go
- example/client/go.mod
- go.mod
🧰 Additional context used
🧠 Learnings (1)
.gitfiles (1)
Learnt from: datelier
PR: vdaas/vald#2807
File: rust/bin/agent/src/handler/object.rs:36-38
Timestamp: 2025-04-01T12:29:37.305Z
Learning: In Rust code for Vald handlers, avoid using unwrap() on Option types. Instead, handle None values explicitly by returning appropriate error statuses to maintain robustness.
🧬 Code Definitions (4)
internal/errors/errors.go (1)
internal/net/grpc/status/status.go (2)
Errorf(129-131)Is(48-50)
internal/net/grpc/stream.go (1)
internal/net/grpc/status/status.go (5)
Is(48-50)Errorf(129-131)ParseError(133-183)Code(41-41)Error(125-127)
internal/net/grpc/pool/pool.go (4)
internal/net/grpc/client.go (3)
ClientConn(50-50)DialOption(49-49)New(129-154)internal/errors/errors.go (3)
Is(163-168)IsNot(182-192)New(92-97)internal/net/grpc/status/status.go (2)
Is(48-50)New(44-46)internal/net/grpc/codes/codes.go (2)
Canceled(26-26)DeadlineExceeded(29-29)
pkg/agent/core/ngt/service/ngt.go (2)
internal/errors/errors.go (2)
IsNot(182-192)Join(254-319)internal/errors/agent.go (1)
ErrUncommittedIndexNotFound(98-98)
🪛 GitHub Check: codecov/patch
internal/iter/iter.go
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests
[warning] 81-87: internal/iter/iter.go#L81-L87
Added lines #L81 - L87 were not covered by tests
[warning] 89-91: internal/iter/iter.go#L89-L91
Added lines #L89 - L91 were not covered by tests
[warning] 96-97: internal/iter/iter.go#L96-L97
Added lines #L96 - L97 were not covered by tests
[warning] 101-107: internal/iter/iter.go#L101-L107
Added lines #L101 - L107 were not covered by tests
[warning] 109-111: internal/iter/iter.go#L109-L111
Added lines #L109 - L111 were not covered by tests
[warning] 117-123: internal/iter/iter.go#L117-L123
Added lines #L117 - L123 were not covered by tests
[warning] 125-127: internal/iter/iter.go#L125-L127
Added lines #L125 - L127 were not covered by tests
[warning] 132-137: internal/iter/iter.go#L132-L137
Added lines #L132 - L137 were not covered by tests
[warning] 139-141: internal/iter/iter.go#L139-L141
Added lines #L139 - L141 were not covered by tests
[warning] 145-146: internal/iter/iter.go#L145-L146
Added lines #L145 - L146 were not covered by tests
[warning] 150-151: internal/iter/iter.go#L150-L151
Added lines #L150 - L151 were not covered by tests
internal/test/mock/server_stream.go
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by tests
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by tests
pkg/agent/core/faiss/service/faiss.go
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by tests
[warning] 1250-1250: pkg/agent/core/faiss/service/faiss.go#L1250
Added line #L1250 was not covered by tests
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by tests
[warning] 1268-1268: pkg/agent/core/faiss/service/faiss.go#L1268
Added line #L1268 was not covered by tests
pkg/agent/core/ngt/service/ngt.go
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by tests
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
⏰ Context from checks skipped due to timeout of 90000ms (40)
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: CodeQL Analyze (rust)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
🔇 Additional comments (105)
apis/swagger/v1/vald/remove.swagger.json (3)
17-18: Enhanced API Documentation for/removeEndpoint
The updated summary and description now include structured status codes and detailed troubleshooting steps, which greatly improves clarity for users. Please verify that the markdown elements (e.g., newlines, table formatting) render correctly in your Swagger UI.
50-51: Detailed MultiVector Removal Update for/remove/multiple
The revised summary clearly states the multi-vector removal capabilities and the description now adds important warnings about gRPC message size limits along with structured status codes and troubleshooting steps. It is recommended to verify that both the HTML components and markdown styling display effectively in all API documentation tools.
83-84: Comprehensive Timestamp-based Removal Update for/remove/timestamp
The improved summary and description provide precise information on timestamp-based removal and explain the use of multiple timestamps with anANDcondition. As the description embeds HTML elements for styling, please ensure that these render properly in all client tools.apis/swagger/v1/vald/filter.swagger.json (6)
17-17: Enhanced Summary for/insert/objectEndpoint
The updated summary now provides a clear and informative description of the InsertObject RPC, complete with a detailed status code table. This is a significant improvement in documentation clarity.
49-49: Enhanced Summary for/insert/object/multipleEndpoint
The revised summary succinctly explains the MultiInsertObject RPC with an emphasis on handling multiple objects in one request. The inclusion of the status code table further aids understanding.
81-81: Refined Summary for/search/objectEndpoint
The updated summary clearly details the SearchObject RPC functionality and provides a comprehensive status code table. This update improves the endpoint’s clarity.
113-113: Detailed Summary for/search/object/multipleEndpoint
The enhanced summary now explains the StreamSearchObject RPC, including its use of bidirectional streaming. The additional context and status code table are valuable for developers.
145-145: Updated Summary for/update/objectEndpoint
The revised summary now clearly identifies the endpoint as handling the update of a single vector and includes a consistent status code table.
210-210: Enhanced Summary for/upsert/objectEndpoint
The updated summary now clearly differentiates the upsert functionality for a single object, with a comprehensive status code table that enhances clarity.internal/errors/k8s.go (1)
20-46: Grouping error variables improves code organization.The refactoring to group all related Kubernetes error variables in a single
varblock is a good change. It improves code organization and maintainability. These new error types will enhance error handling capabilities for Kubernetes operations, matching well with the PR objective to improve the E2E testing framework with Kubernetes integration.internal/info/info_test.go (1)
339-342: New valdReplacer field added to fields structsThe addition of the
valdReplacer *strings.Replacerfield to thefieldsstruct indicates that theinfostruct in the main code now includes string replacement functionality. This is a good step towards improving the testing infrastructure.internal/params/params.go (7)
23-23: New import slices package for element deletionGood addition, using the standard library
slicespackage provides a cleaner way to implement filtering operations.
29-35: Good type aliasing for ErrorHandlingWell-structured approach to create a type alias and constants for error handling strategies. This provides better encapsulation and abstraction from the underlying flag package implementation.
54-59: Improved parser structure with additional configuration optionsThe enhanced struct with fields for override control, naming, filtering, and custom flag handling significantly improves flexibility. The ErrorHandler field properly exposes error handling strategy configuration.
Also applies to: 69-69
78-82: Improved parser initializationGood implementation to properly initialize defaults and custom flag sets. The conditional override provides flexibility in how the parser interacts with the standard flag handling.
88-90: Added null safety checksGood defensive programming practice to check for nil parser or flag set before proceeding with parsing operations.
108-119: Added argument filtering mechanismWell-implemented filtering system that allows for custom argument filtering based on provided predicates. This adds flexibility for handling complex command-line scenarios.
120-126: Improved error handling using errors.IsGood practice to use
errors.Isinstead of direct error comparison. This follows Go 1.13+ error handling best practices.apis/swagger/v1/vald/search.swagger.json (1)
149-150: Updated/searchEndpoint DocumentationThe revised summary now gives a structured overview of the search operation, including relevant status codes and troubleshooting guidance. The structured format improves clarity—please verify consistent rendering with other endpoint descriptions.
.github/workflows/codeql-analysis.yml (5)
58-67: Matrix Strategy Configuration Validation
The matrix strategy clearly defines build modes per language—with Go usingautobuildand the others set tonone—which is concise and flexible. Double-check that these settings fully align with the requirements for each language and that any future language additions or changes in build requirements are accommodated.
70-72: Container Credentials Integration
The container block now provides credentials (usernameandpassword) for pulling the image from the container registry. Ensure that the secretsPACKAGE_USERandPACKAGE_TOKENare correctly set up in the repository’s secrets management and that their usage complies with security best practices.
84-88: Upgraded CodeQL Initialization Step
The initialization step has been updated to usegithub/codeql-action/init@v3along with dynamically passinglanguagesandbuild-modefrom the matrix. This update should help maintain compatibility with new features in CodeQL v3, but please verify that the configuration file (./.github/codeql/codeql-config.yaml) remains valid and effective under the new version.
90-91: Conditional Autobuild Execution for Go Projects
The Autobuild step now conditionally runs only whenmatrix.languageequals'go'and usesgithub/codeql-action/autobuild@v3. This is appropriate given that Go projects typically require autobuilding with CodeQL. Confirm that this condition fully covers the expected usage scenarios for Go within the repository.
93-95: Updated CodeQL Analysis Step
The analysis step is correctly updated to usegithub/codeql-action/analyze@v3and now includes a dynamiccategoryparameter (/language:${{matrix.language}}). It’s important to verify that this category format meets the expectations of the v3 action. Overall, these updates contribute to improved clarity and maintainability in the CodeQL scanning workflow.pkg/index/job/deletion/service/options_test.go (4)
16-101: Framework for TestWithDiscoverer implementationThis is a new test file with a commented-out structure for testing the
WithDiscovererfunction. It follows the project's established test pattern but needs concrete test cases.Since this is part of a new E2E testing framework, implementing actual test cases is important for ensuring reliability.
103-186: No real test cases inTestWithIndexingConcurrency.
Implement test coverage for:
- Positive concurrency value.
- Zero concurrency.
- Negative or invalid values.
188-271: Framework for TestWithTargetAddrs implementationAnother template for testing the
WithTargetAddrsfunction without actual test cases implemented.Consider adding test cases for:
- Multiple valid addresses
- Empty address list
- Addresses with different formats
This will ensure that target address handling is robust in the E2E testing framework.
273-356: Framework for TestWithTargetIndexID implementationA final template for testing the
WithTargetIndexIDfunction without concrete test cases.Consider implementing test cases for:
- A valid index ID
- An empty index ID
- Special character handling in index IDs (if applicable)
This would complete the testing coverage for the service options in this new package.
pkg/index/job/deletion/service/deleter_test.go (5)
1-528:⚠️ Potential issueUncomment and implement the test functions.
This entire file contains commented-out test functions with TODOs. While the structure is good (parallel execution, goroutine leak detection, table-driven tests), the tests need to be uncommented and implemented before merging.
The file needs:
- Required imports at the top (context, errors, reflect, etc.)
- A MockClient implementation for testing client interactions
- Concrete test cases for each function
- Error handling test scenarios
Previous review comments have indicated these issues, and some were supposedly addressed in commits, but they're not reflected in this version of the file.
416-527: Add error handling tests for Test_index_doDeleteIndex.The
Test_index_doDeleteIndexfunction should include tests for error conditions during the deletion process.Implement test cases that simulate:
- Network errors
- Context cancellation/timeout
- Partial success scenarios
- Retry logic (if implemented)
These edge cases are important to ensure the robustness of the deletion service in production environments.
192-304: Implement Test_index_StartClient with mock client.The
Test_index_StartClientfunction should use mocked clients to test the client initialization logic.Use a mock discoverer.Client implementation to test different scenarios:
- Successful client initialization
- Context cancellation handling
- Client errors
A MockClient implementation should be added at the top of the file to facilitate testing.
107-190: Implement Test_delDuplicateAddrs with actual deduplication scenarios.The
Test_delDuplicateAddrsfunction needs concrete test cases with a variety of input address patterns to verify the deduplication logic correctly.Implement test cases covering common deduplication scenarios:
- Empty slice
- Slice with no duplicates
- Slice with duplicates
- Slice with all duplicates
18-105: Implement TestNew with option validation tests.The
TestNewfunction should test the constructor with various option combinations to verify proper initialization.Test cases should include:
- Default initialization
- Custom concurrency settings
- Custom target index ID
- Combinations of multiple options
apis/docs/v1/docs.md (1)
1-1982: Overall Documentation Enhancements:
The updates throughout the documentation file add comprehensive overviews for each service, complete method descriptions, and detailed troubleshooting guidance and status code tables. These improvements greatly enhance the clarity and usability of the API documentation. Please review the formatting across different sections (especially the extensive tables) to ensure uniformity and correct rendering in your target documentation viewer.🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
688-688: Link fragments should be valid
null(MD051, link-fragments)
782-782: Link fragments should be valid
null(MD051, link-fragments)
826-826: Link fragments should be valid
null(MD051, link-fragments)
837-837: Link fragments should be valid
null(MD051, link-fragments)
848-848: Link fragments should be valid
null(MD051, link-fragments)
859-859: Link fragments should be valid
null(MD051, link-fragments)
997-997: Link fragments should be valid
null(MD051, link-fragments)
1094-1094: Link fragments should be valid
null(MD051, link-fragments)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1514-1514: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Reference links and images should use a label that is defined
Missing link or image reference definition: "google.rpc.code.permissiondenied"(MD052, reference-links-images)
1624-1624: Link fragments should be valid
null(MD051, link-fragments)
internal/test/data/vector/noise/noise.go (2)
78-105: Potential division-by-zero with zero-length vectors
If the dataset is non-empty but all vectors are zero-length,countremains 0 and triggers a division-by-zero at line 105 (totalStd/float64(count)).
112-123: Double-check power-of-two calculation for edge cases
Whenrequiredequals 0 or is below the configured minimum (if it can be zero), subtracting 1 (line 122) can lead to an underflow or pass 0 tobits.Len64. Please ensurerequiredremains ≥ 1.internal/net/quic/conn_test.go (1)
90-91: Add goleak support for detecting goroutine leaksThe test uses
goleak.VerifyNone()to check for goroutine leaks, but the import isn't included.#!/bin/bash # Check if goleak is already imported in the project rg --type go "import.*goleak" --statsinternal/config/config_test.go (1)
999-1003:❓ Verification inconclusive
Test case update reflects a behavior change in file handling.
The renamed test case and modified expectation indicate that the
GetActualValuefunction now returns an empty result when a file doesn't exist, rather than returning the file path itself. This is a significant behavior change that could affect dependent code.Could you verify if this behavior change was intentional and whether any dependent code that might expect the previous behavior (returning the file path) has been updated accordingly?
🏁 Script executed:
#!/bin/bash # Search for any other code that might be relying on GetActualValue with file:// prefixes rg -A 5 "GetActualValue.*file://" --glob "!internal/config/config_test.go"Length of output: 76
Action Required: Verify Downstream Dependencies for Behavior Change
The updated test case in
internal/config/config_test.go(lines 999–1003) now expects thatGetActualValuereturns an empty result when a file does not exist—changing from the previous behavior of returning the file path. Although our initial search for references to this behavior (i.e. usage ofGetActualValuewithfile://prefixes) outside the test file produced no output, the absence of evidence in this scan leaves room for uncertainty.Please manually verify that no other parts of the codebase (or dependent code) rely on the original behavior. If any such dependencies exist, ensure they’re updated to accommodate the new behavior.
internal/net/grpc/errdetails/errdetails.go (6)
80-82: Good addition of structured Details type for improved error handling.This new type provides a clean way to encapsulate multiple Detail objects in a single structure, enhancing error representation and serialization capabilities.
26-26: Appropriate imports to support new functionality.The addition of
convpackage (line 26) andprotojson(line 35) properly supports the enhanced error marshaling capabilities introduced in this update.Also applies to: 35-35
93-94: Enhanced type handling with fallback for unknown types.The improved type detection with the fallback to
protojson.Marshalfor unknown types makes the error handling more robust and flexible.Also applies to: 165-167
168-172: Improved error handling with detailed logging.Good enhancement to log warnings with useful context when marshaling fails or produces nil results, making debugging easier.
316-322: Enhanced serialization with proper null checks.The update to
Serializefunction now properly handles multiple messages using the newDetailstype and includes a null check before returning results.
416-420: Added flexible handling for unknown message types.The new default case in
AnyToErrorDetailimproves the function's versatility by attempting to unmarshal any unknown types withUnmarshalNew().internal/net/grpc/option_test.go (6)
532-614: Test function renamed to match implementation changes.The function
TestWithBackoffMaxDelayhas been renamed from the previousTestWithDialOptions, with the parameter type changed fromopts []grpc.DialOptiontodur string. This appears to be part of a larger refactoring to better match test functions with their corresponding implementations.
617-700: Test function renamed with matching parameter update.
TestWithBackoffBaseDelay(previouslyTestWithBackoffMaxDelay) now properly reflects the function it's testing, maintaining consistency with the implementation changes.
702-785: Comprehensive test function renaming.This extensive renaming aligns the test suite with the implementation changes, ensuring that each test function properly exercises its corresponding feature. Parameter types have been appropriately updated to match the new function signatures.
Also applies to: 787-870, 872-955, 957-1040, 1042-1125, 1127-1210, 1212-1295, 1297-1380, 1382-1465
1467-3259: Maintained consistent test structure throughout refactoring.The refactoring preserves the test structure while updating function names and parameter types, maintaining the test coverage for all option functions.
3261-3344: Function renamed for better clarity.
TestWithOldConnCloseDelay(previouslyTestWithOldConnCloseDuration) uses a more precise term "delay" instead of "duration," improving clarity while maintaining the same functionality.
532-3344:❓ Verification inconclusive
Verify implementation compatibility with renamed tests.
The extensive renaming of test functions suggests significant changes to the corresponding implementation file. Ensure that all renamed test functions correctly align with their implementation counterparts.
🏁 Script executed:
#!/bin/bash # Check if the option function implementations exist and match the test names echo "Checking for implementation of option functions that match test functions..." # Find the implementation file implementation_file=$(find internal/net/grpc -name "option.go") echo "Implementation file: $implementation_file" # Check for key functions that were renamed in tests grep -n "WithBackoffMaxDelay" "$implementation_file" grep -n "WithBackoffBaseDelay" "$implementation_file" grep -n "WithBackoffMultiplier" "$implementation_file" grep -n "WithBackoffJitter" "$implementation_file" grep -n "WithOldConnCloseDelay" "$implementation_file" echo "Done checking implementation functions."Length of output: 1489
Action Required: Verify that tests reference the correct implementation file.
Our verification indicates that the option functions (e.g. WithBackoffMaxDelay, WithBackoffBaseDelay, WithBackoffMultiplier, WithBackoffJitter, and WithOldConnCloseDelay) are implemented in the file located at
internal/net/grpc/pool/option.gorather than a single expected file. The automated check script returned errors due to multiple file paths being aggregated, so it couldn’t directly confirm each function within one file.Please ensure that the renamed test functions in
internal/net/grpc/option_test.gocorrectly map to the implementations found ininternal/net/grpc/pool/option.go. Double-check that:
- The tests invoking functions like
WithBackoffMaxDelayand others are updated to refer to the implementations ininternal/net/grpc/pool/option.go.- No lingering references or inconsistencies remain from the previous naming/location.
pkg/agent/core/ngt/service/ngt.go (5)
1475-1478: Added nil check prevents potential null pointer dereferenceThis new check for nil statistics prevents a potential panic that could occur if attempting to dereference a nil pointer in the subsequent code.
The warning log also helps with debugging by explicitly indicating when statistics are unavailable.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
1824-1825: Simplified error handling witherrors.IsNotSimilar to the change above, this replaces a verbose conditional with the more concise
errors.IsNotfunction to check for multiple error types.
1973-1974: Consistent use oferrors.IsNotfor error checkingThe updated error handling pattern continues to be applied consistently throughout the file.
1987-1988: Simplified error handling in Close methodThe use of
errors.IsNotsimplifies the check against multiple error types, making the code more readable.
1995-1996: Continued consistent application of simplified error handlingThe updated error handling pattern is applied to the SaveIndex error check as well.
internal/errors/errors.go (3)
86-88: Added useful error utility for unimplemented featuresThe new
ErrUnimplementedfunction provides a standardized way to create errors for unimplemented features, which improves consistency in error reporting.
170-180: AddedIsAnyfunction for more flexible error comparisonThis new utility function allows checking if an error matches any of the provided target errors, which is useful for handling multiple expected error types in a single check.
This improves error handling capabilities and can simplify complex error checking logic.
182-192: AddedIsNotfunction to simplify negative error checkingThis is a key addition that inverts the error comparison logic, making it easier to check if an error is not one of the specified types. This function is being used extensively in the updated error handling patterns throughout the codebase.
The implementation properly handles edge cases such as nil errors and empty target lists.
pkg/agent/core/faiss/service/faiss.go (3)
1126-1126: Simplified error handling witherrors.IsNotThe code now uses
errors.IsNotto check multiple error types in a more readable manner.This pattern is being applied consistently across the codebase.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
1260-1261: Consistent application of new error handling patternThe use of
errors.IsNotis applied consistently for both CreateIndex and SaveIndex error checks.Also applies to: 1268-1269
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by tests
1246-1248:✅ Verification successful
Changed error handling behavior to suppress certain errors
The added condition sets
errto nil if it's a context cancellation or deadline exceeded error, effectively ignoring these types of errors.This is a subtle change in error handling behavior that might affect how callers handle errors. Ensure that ignoring these context-related errors is the intended behavior.
🏁 Script executed:
#!/bin/bash # Check how these errors are handled in other parts of the code echo "Checking for similar error handling patterns:" rg -A 2 "errors.IsNot.*context.Canceled.*context.DeadlineExceeded" --type goLength of output: 5403
Confirmed: Context Error Suppression Is Intentional
The new error handling change in
pkg/agent/core/faiss/service/faiss.go—which setserr = nilwhen the error is due to a context cancellation or deadline exceeded—is consistent with similar patterns found throughout the codebase (e.g., inpkg/manager/index/service/indexer.go,internal/servers/servers.go, and others). This consistency indicates that suppressing these context-related errors is indeed the intended behavior.Please ensure that all consumers of this function are aware of this behavior and that it aligns with the overall error handling strategy.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by testsinternal/sync/errgroup/group_test.go (7)
945-947: Renamed test function reflects implementation changesThe test function has been renamed from
Test_group_TryGotoTest_group_exec, suggesting a refactoring of the underlying implementation where theTryGomethod has been renamed or replaced with anexecmethod.
958-959: Simplified test structure by removing unused fieldsThe
wantstruct has been simplified by removing thewant boolfield, suggesting that theexecfunction no longer returns a boolean value unlike the previousTryGofunction.
964-965: Updated checkFunc signature to match simplified structThe function signature for
checkFunchas been updated to only take awantparameter, removing the boolean parameter to match the simplifiedwantstruct.
968-970: Simplified defaultCheckFunc implementationThe
defaultCheckFuncimplementation now directly returnsnilwithout needing to compare a boolean value, which aligns with the changes to thewantstruct and function under test.
953-954: Added atomic limit field to support concurrency controlA new
limit atomic.Int64field has been added to thefieldsstruct, indicating that the refactored implementation now uses atomic operations for concurrency limiting.
1051-1052: Function call updated to match implementationThe call to
g.TryGo(test.args.f)has been replaced withg.exec(test.args.f), reflecting the underlying implementation change fromTryGotoexec.
1045-1046: Consistent struct initialization with new limit fieldThe initialization of the
groupstruct in the test function now includes setting thelimitfield from the test case, ensuring all fields are properly initialized.internal/net/grpc/stream_test.go (2)
206-209: Enhanced type-safety in stream client test argsThe
argsstruct has been updated to use more specific types with*Sand*Rgeneric-style parameters instead ofany, and includes aconcurrencyparameter for parallel processing. This improves type safety and allows for concurrent stream handling.
290-290: Updated function call matches new parameter structureThe call to
BidirectionalStreamClienthas been updated to pass the new parameters in the correct order, reflecting the changes in the function signature.internal/net/grpc/pool/pool.go (11)
17-21: Improved documentation reflecting architectural changesThe package documentation has been updated to clearly explain the lock-free nature of the implementation and highlight its key features. This is excellent for maintainers and users of the package.
49-71: Enhanced interface documentation and context supportThe
Conninterface has been updated to include context parameters for key methods, allowing better control over cancellation and timeouts. The detailed documentation for each method clarifies their purpose and behavior.
79-121: Robust connection closing with state checkingThe
Closemethod implementation forpoolConnensures graceful shutdown of connections by periodically checking connection state and handling different connectivity states appropriately. This helps prevent resource leaks and abrupt connection terminations.
127-128: Lock-free connection slot storage using atomic pointersThe
connSlotsfield uses nested atomic pointers to store connection slots without requiring locks, allowing for efficient concurrent access. This is a significant performance improvement for high-concurrency scenarios.
661-696: Thorough DNS resolution with connection verificationThe
lookupIPAddrmethod not only resolves DNS names to IP addresses but also verifies their reachability with quick TCP dials, ensuring that only accessible IPs are used for connections. This improves connection reliability.
725-738: Added metrics for connection health monitoringThe new
Metricsfunction provides valuable insights into connection health across different target addresses, which is useful for monitoring and debugging. This is a welcome addition for observability.
740-770: Comprehensive connection health checkingThe
isHealthymethod has been enhanced to handle various connection states more explicitly, including triggering reconnection for idle connections and providing detailed error messages for unhealthy connections. This improves reliability and diagnostics.
550-553: Consider simplifying error handling in dial methodWhen an error occurs during backoff, the code attempts to close the connection twice: once with
_ = conn.Close()and then withconn.Close()in theerrors.Join. While this may not cause issues if the second close is a no-op, it adds unnecessary complexity.- if err != nil && conn != nil { - _ = conn.Close() - return nil, errors.Join(err, conn.Close()) - } + if err != nil && conn != nil { + closeErr := conn.Close() + return nil, errors.Join(err, closeErr) + }
436-457: Efficient single-target connection handlingThe
singleTargetConnectmethod includes early termination logic if multiple connection attempts fail, which can avoid wasting resources on unavailable targets. This is especially important for handling connection failures gracefully.
163-165: Added global metrics storage using sync.MapThe use of a thread-safe
sync.Mapto store metrics is appropriate for this concurrent context. This allows for efficient data collection without adding lock contention.
402-407: Intelligent handling of DNS resolution resultsThe code efficiently handles different DNS resolution outcomes, using single-target connection for single IP results and multi-target connection for multiple IPs. This optimizes connection establishment based on the actual DNS environment.
internal/net/grpc/stream.go (7)
44-53: Well-designed type interfaces for improved type safetyThe new
TypedClientStreamandTypedServerStreaminterfaces provide strong type constraints using Go generics. This improves type safety by ensuring that the stream interfaces properly match the message types being sent and received.This change enhances compile-time type checking and reduces the potential for runtime errors caused by type mismatches.
106-107: Improved stream reception with proper type handlingThe stream receive operation has been changed from using
RecvMsgwith a pointer to usingRecv()that returns a value and error. This follows Go idioms better and leverages the new type interfaces.
125-144: Enhanced error handling with more granular code checksThe error handling logic now checks for specific error codes before deciding whether to log errors. This prevents logging expected errors like cancellations, deadline exceeded, and not found errors.
This change improves the signal-to-noise ratio in logs by only reporting unexpected errors.
178-180: Concurrency limiting for better resource managementThe addition of concurrency limiting in both server and client stream functions allows for better control of resource usage, especially in high-throughput scenarios.
This change helps prevent resource exhaustion by controlling the number of concurrent operations.
187-190: Improved error handling with IsNot checksThe new error handling approach using
errors.IsNotis more readable and maintainable than the previous approach with multiple&&conditions.This approach makes it clear which error types are being excluded from the error handling logic.
206-230: Well-structured finalization with comprehensive error handlingThe new
finalizefunction encapsulates cleanup logic with proper error aggregation, ensuring that all errors are properly joined and returned. The use of a dedicated error channel (ech) improves synchronization between goroutines.This refactoring improves code maintainability and ensures consistent cleanup behavior.
238-241: Improved data provider interfaceThe
sendDataProviderfunction now returns a value and a boolean flag instead of a pointer, which provides a clearer interface for signaling completion.This change follows Go idioms better and makes the intention of the code clearer - the boolean explicitly signals when to stop sending data.
internal/file/file.go (1)
259-259:✅ Verification successful
Check if errors.IsNot handles nil errors correctly
The change from what was likely
err != nil && !errors.Is(err, io.EOF)toerrors.IsNot(err, io.EOF)simplifies the code, but we should verify thaterrors.IsNothandles nil errors correctly to ensure consistent behavior.
🏁 Script executed:
#!/bin/bash # Search for the implementation of errors.IsNot to verify nil error handling rg -A 5 "func IsNot" --glob "internal/errors/*.go"Length of output: 380
Confirmed nil error handling in errors.IsNot
The implementation in
internal/errors/errors.goshows that whenerris nil,errors.IsNotreturns false (viareturn err != nil), which is consistent with the original logic (err != nil && !errors.Is(err, io.EOF)). No additional changes are necessary.apis/swagger/v1/vald/insert.swagger.json (1)
17-17: Typographical Error in/insertSummaryThe summary for the
/insertendpoint still contains "Inset RPC" rather than "Insert RPC". This issue was flagged in previous reviews.- "summary": "Overview\nInset RPC is the method to add a new single vector.\n---\nStatus Code\n| 0 | OK |... + "summary": "Overview\nInsert RPC is the method to add a new single vector.\n---\nStatus Code\n| 0 | OK |...internal/iter/iter.go (3)
22-22: Potential circular import issue.This file’s package is
iter, yet it imports"iter". If this import points to the same package, it may cause a self-import loop. Consider removing or renaming the import if it is unintentional, or fully qualifying it if it's a distinct module.
72-77: Enhance unit test coverage of theAt()method.Currently, these lines are not covered by tests, making edge cases (e.g.
iclose toc.num) unverified. Ensure boundary conditions, offset logic, andmodFuncapplication are tested.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests
52-152:❓ Verification inconclusive
Automate coverage verification for
NewCycleand iteration methods.Since multiple coverage gaps were reported, consider running or modifying the following script to confirm thorough testing of all new methods:
🏁 Script executed:
#!/bin/bash # Description: Check references and test coverage of Cycle methods. # 1. Locate all references to NewCycle & cycle methods in test files: rg -A 10 'NewCycle\|\.At\|\.Seq2\|\.Seq\|\.Values\|\.Indexes\|\.ForEach' --type go --glob '*_test.go' # 2. Verify coverage reports (if any): # Typically run 'go test -coverprofile cover.out ./internal/iter/...' # then 'go tool cover -func cover.out'Length of output: 103
Action Required: Manual Coverage Verification for Cycle Methods
The automated script did not return any output, so please manually verify that adequate tests exist for the new cycle methods (such as
NewCycle,At,Seq2,Seq,Values,Indexes,ForEach, etc.) ininternal/iter/iter.go. In particular, check that:
- There are test cases calling these methods, either directly via functions like
TestNewCycleor indirectly through integration tests.- Running the following commands produces a comprehensive coverage report:
go test -coverprofile=cover.out ./internal/iter/...go tool cover -func=cover.outIf tests are missing or coverage is insufficient, please add the necessary tests to confirm that the cycle methods perform as expected.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests
[warning] 81-87: internal/iter/iter.go#L81-L87
Added lines #L81 - L87 were not covered by tests
[warning] 89-91: internal/iter/iter.go#L89-L91
Added lines #L89 - L91 were not covered by tests
[warning] 96-97: internal/iter/iter.go#L96-L97
Added lines #L96 - L97 were not covered by tests
[warning] 101-107: internal/iter/iter.go#L101-L107
Added lines #L101 - L107 were not covered by tests
[warning] 109-111: internal/iter/iter.go#L109-L111
Added lines #L109 - L111 were not covered by tests
[warning] 117-123: internal/iter/iter.go#L117-L123
Added lines #L117 - L123 were not covered by tests
[warning] 125-127: internal/iter/iter.go#L125-L127
Added lines #L125 - L127 were not covered by tests
[warning] 132-137: internal/iter/iter.go#L132-L137
Added lines #L132 - L137 were not covered by tests
[warning] 139-141: internal/iter/iter.go#L139-L141
Added lines #L139 - L141 were not covered by tests
[warning] 145-146: internal/iter/iter.go#L145-L146
Added lines #L145 - L146 were not covered by tests
[warning] 150-151: internal/iter/iter.go#L150-L151
Added lines #L150 - L151 were not covered by tests.gitfiles (12)
126-126: LGTM: New Rust unit testing workflow added.This new workflow file
.github/workflows/unit-test-rust.yamlaligns with the PR objectives of improving the testing infrastructure by adding support for testing Rust components.
161-164: Enhanced API documentation structure looks good.The addition of new documentation generation files (
buf.gen.doc.yaml,buf.gen.payload.yaml,buf.gen.tmpl.yaml) and API documentation files for various operations improves the project's documentation coverage, which supports better maintainability.Also applies to: 166-177
488-488: Added QBG sample configuration file.This new file adds configuration support for the QBG algorithm, which complements the new QBG Rust library being added in this PR.
512-512: Improved test coverage for index operations.The addition of test files for index deletion operations enhances test coverage of critical infrastructure components, which aligns with the PR objective of improving testing maintainability.
Also applies to: 863-863, 825-832
1051-1051: New error handling for parameters.The addition of
internal/errors/params.goimproves error handling capabilities, which is consistent with the PR's goal of refactoring the internal package.
1088-1088: New iterator abstraction package.This file implements iterator abstractions as mentioned in the PR objectives, which should help improve code maintainability and reduce duplication.
1274-1274: Please add "quic" to the project dictionary.The QUIC protocol test files are flagged by cspell as containing unknown words. This was previously identified in the codebase.
Run the following to update
.cspell.jsonto include "quic":- // Add "quic" to the words array in .cspell.json + // Example: "words": ["vald", "quic", ...]Also applies to: 1276-1276
1966-1966: Critical: Replaceunwrap()with explicit Option handling in Rust code.Based on previous learnings, Rust handlers in Vald should avoid using
unwrap()on Option types. Please ensure proper error handling by explicitly handling None values.
1986-1990: New QBG algorithm support added.The addition of the QBG algorithm library with C++ integration expands Vald's vector search capabilities. This aligns with the overall project goal of enhancing the vector search infrastructure.
1407-1410: Added noise generation for vector testing.These files provide noise generation capabilities for vector testing, which enhances the testing framework's ability to generate realistic test data.
2035-2055: Comprehensive new E2E testing framework added.The new V2 E2E testing framework provides a more maintainable and structured approach to testing, with:
- Dedicated asset configurations for different CRUD operations
- Kubernetes integration with client, resources, and portforwarding
- Structured test cases covering various operation types
This aligns perfectly with the PR's main objective of enhancing testing infrastructure maintainability.
2067-2067: Added K3D version specification.Adding a version file for K3D ensures consistent testing environments, which supports the E2E testing improvements.
| func (m *StreamInsertServerMock) Send(l *payload.Object_StreamLocation) error { | ||
| return m.SendFunc(l) | ||
| if m != nil { | ||
| if m.SendFunc != nil { | ||
| return m.SendFunc(l) | ||
| } | ||
| return m.ServerStream.SendMsg(l) | ||
| } | ||
| return nil | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Good defensive coding improvements
The added nil checks enhance robustness by gracefully handling both nil receiver and nil function pointer cases. This pattern is consistent with defensive programming practices.
As noted in a previous review, these changes should be validated with tests. Consider adding test cases that verify:
- Behavior when
mis nil - Behavior when
SendFuncis nil (fallback toServerStream.SendMsg)
🏁 Script executed:
#!/bin/bash
# Description: Find existing test coverage for StreamInsertServerMock.Send
# Look for tests that might exercise the Send method
rg -A 5 "StreamInsertServerMock.*Send" --type goLength of output: 449
Action Required: Add Tests for Nil Receiver and Fallback Behavior
The defensive checks added in the Send method are a strong improvement. However, based on the grep results, it appears that there are no dedicated tests verifying:
- Behavior when the receiver (
m) is nil. - Behavior when
SendFuncis nil, ensuring the fallback toServerStream.SendMsg.
Please add test cases for these conditions to confirm the intended behavior.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by tests
| | SearchObject | [.payload.v1.Search.ObjectRequest](#payload-v1-Search-ObjectRequest) | [.payload.v1.Search.Response](#payload-v1-Search-Response) | Overview SearchObject RPC is the method to search object(s) similar to request object. --- Status Code | 0 | OK | | 1 | CANCELLED | | 3 | INVALID_ARGUMENT | | 4 | DEADLINE_EXCEEDED | | 6 | ALREADY_EXISTS | | 13 | INTERNAL | | | ||
| | MultiSearchObject | [.payload.v1.Search.MultiObjectRequest](#payload-v1-Search-MultiObjectRequest) | [.payload.v1.Search.Responses](#payload-v1-Search-Responses) | Overview StreamSearchObject RPC is the method to search vectors with multi queries(objects) using the [bidirectional streaming RPC](https://grpc.io/docs/what-is-grpc/core-concepts/#bidirectional-streaming-rpc).<br> By using the bidirectional streaming RPC, the search request can be communicated in any order between client and server. Each Search request and response are independent. --- Status Code | 0 | OK | | 1 | CANCELLED | | 3 | INVALID_ARGUMENT | | 4 | DEADLINE_EXCEEDED | | 6 | ALREADY_EXISTS | | 13 | INTERNAL | | | ||
| | StreamSearchObject | [.payload.v1.Search.ObjectRequest](#payload-v1-Search-ObjectRequest) stream | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | Overview MultiSearchObject RPC is the method to search objects with multiple objects in **1** request. | |
There was a problem hiding this comment.
Inconsistency Between MultiSearchObject and StreamSearchObject Descriptions:
There seems to be a mix-up in the descriptions within the table rows for the search-related RPCs in the Filter service. The “MultiSearchObject” row is described as “Overview StreamSearchObject RPC is the method to search vectors with multi queries…” while the “StreamSearchObject” row states “Overview MultiSearchObject RPC is the method to search objects with multiple objects in 1 request.” This appears to be an accidental swap of descriptions. Please verify the intended functionality and update the descriptions so that each method’s overview correctly corresponds to its row.
| // func Test_noiseGenerator_Mod(t *testing.T) { | ||
| // type fields struct { | ||
| // noiseLevelFactor float32 | ||
| // noiseTableDivisionFactor uint64 | ||
| // minNoiseTableSize uint64 | ||
| // noiseTable []float32 | ||
| // } | ||
| // type want struct { | ||
| // want Func | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // fields fields | ||
| // want want | ||
| // checkFunc func(want, Func) error | ||
| // beforeFunc func(*testing.T) | ||
| // afterFunc func(*testing.T) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got Func) error { | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // fields: fields { | ||
| // noiseLevelFactor:0, | ||
| // noiseTableDivisionFactor:0, | ||
| // minNoiseTableSize:0, | ||
| // noiseTable:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // fields: fields { | ||
| // noiseLevelFactor:0, | ||
| // noiseTableDivisionFactor:0, | ||
| // minNoiseTableSize:0, | ||
| // noiseTable:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // ng := &noiseGenerator{ | ||
| // noiseLevelFactor: test.fields.noiseLevelFactor, | ||
| // noiseTableDivisionFactor: test.fields.noiseTableDivisionFactor, | ||
| // minNoiseTableSize: test.fields.minNoiseTableSize, | ||
| // noiseTable: test.fields.noiseTable, | ||
| // } | ||
| // | ||
| // got := ng.Mod() | ||
| // if err := checkFunc(test.want, got); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Implement test coverage for noise modification
Test_noiseGenerator_Mod is also missing real test cases. Verifying both normal and edge cases (e.g., zero-length vectors) will help confirm deterministic noise generation and ensure stable concurrency.
| // NOT IMPLEMENTED BELOW | ||
| // | ||
| // func TestNew(t *testing.T) { | ||
| // type args struct { | ||
| // data [][]float32 | ||
| // num uint64 | ||
| // opts []Option | ||
| // } | ||
| // type want struct { | ||
| // want Modifier | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // args args | ||
| // want want | ||
| // checkFunc func(want, Modifier) error | ||
| // beforeFunc func(*testing.T, args) | ||
| // afterFunc func(*testing.T, args) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got Modifier) error { | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // args: args { | ||
| // data:nil, | ||
| // num:0, | ||
| // opts:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // args: args { | ||
| // data:nil, | ||
| // num:0, | ||
| // opts:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt, test.args) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt, test.args) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // | ||
| // got := New(test.args.data, test.args.num, test.args.opts...) | ||
| // if err := checkFunc(test.want, got); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Implement test coverage for data & constructor behavior
These placeholders for TestNew lack actual test cases. Testing initialization with valid data, empty data, and different option combinations is critical for ensuring correct behavior of the noise generator.
| // func TestWithTableDivisionFactor(t *testing.T) { | ||
| // type args struct { | ||
| // f uint64 | ||
| // } | ||
| // type want struct { | ||
| // want Option | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // args args | ||
| // want want | ||
| // checkFunc func(want, Option) error | ||
| // beforeFunc func(*testing.T, args) | ||
| // afterFunc func(*testing.T, args) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got Option) error { | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // args: args { | ||
| // f:0, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // args: args { | ||
| // f:0, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt, test.args) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt, test.args) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // | ||
| // got := WithTableDivisionFactor(test.args.f) | ||
| // if err := checkFunc(test.want, got); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add diverse test coverage for table division
TestWithTableDivisionFactor lacks test cases. Try boundary values (e.g., zero, one) and typical production values to confirm the option’s effect on noise table size.
| func BidirectionalStream[Q, R proto.Message, S TypedServerStream[Q, R]]( | ||
| ctx context.Context, stream S, concurrency int, handle func(context.Context, Q) (R, error), | ||
| ) (err error) { |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Breaking API change requires caller updates
The function signature for BidirectionalStream has been significantly modified with several breaking changes:
- Added type constraints requiring protobuf messages
- Changed from generic
ServerStreamto the more specificTypedServerStream - Changed parameter types from pointers to values
- Changed parameter name from
ftohandle
This improves type safety but will require updates to all callers. Run the following script to identify affected call sites:
🏁 Script executed:
#!/bin/bash
# Search for direct calls to BidirectionalStream
rg "BidirectionalStream\(" --glob '*.go' | grep -v "func BidirectionalStream"Length of output: 4712
Breaking API Change: Update All Call Sites for BidirectionalStream
The updated function signature now improves type safety by:
- Introducing type constraints (requiring protobuf messages)
- Switching from a generic
ServerStreamto a more specificTypedServerStream - Changing parameter types from pointers to values
- Renaming the callback parameter from
ftohandle
Action Required:
- Update every call site of
BidirectionalStreamto match the new signature. - The following locations were identified as affected:
pkg/gateway/lb/handler/grpc/handler.gopkg/gateway/filter/handler/grpc/handler.go- Multiple files in
pkg/agent/core/ngt/handler/grpc/(e.g.,update.go,upsert.go,search.go,remove.go,object.go,insert.go,linear_search.go) pkg/gateway/mirror/handler/grpc/handler.gointernal/net/grpc/stream_test.go(test cases)
Please update the call sites accordingly and ensure that the constraints and type changes are properly handled. After updating, run the tests to verify that behavior remains correct.
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (2)
pkg/index/job/deletion/service/options_test.go (1)
1-357: 💡 Verification agent🧩 Analysis chain
Umbrella testing framework needs completion
This file is part of the new E2E testing framework but currently contains only commented-out test structures without actual implementations.
Would you like assistance in implementing actual test cases for these functions? I can help create concrete examples that would provide robust test coverage for these service options.
🏁 Script executed:
#!/bin/bash # Find the corresponding implementation file to better understand the functions being tested echo "Looking for the corresponding implementation file..." fd -p "pkg/index/job/deletion/service/options\\.go$" # Check how these functions are being used in other implemented tests echo "Looking for examples of similar test patterns in the codebase..." rg "TestWith.*" --type go | grep -v "_test.go:" | head -10Length of output: 408
Complete Test Implementation for Service Options
The file
pkg/index/job/deletion/service/options_test.gocurrently only contains commented-out test scaffolds without any active test cases. Since we have an implementation file (pkg/index/job/deletion/service/options.go), it's important to move beyond placeholders and provide concrete tests for functions likeWithDiscoverer,WithIndexingConcurrency,WithTargetAddrs, andWithTargetIndexID.
- The test file remains a scaffold without actual test logic.
- No similar live test implementations were detected elsewhere in the codebase to serve as direct examples.
- Replacing the commented code with active, robust test cases will help validate the functionality of the service options.
Would you like assistance in drafting these test cases?
internal/sync/errgroup/group_test.go (1)
946-1058: 🛠️ Refactor suggestionReintroduce or remove commented-out test
TheTest_group_execfunction is commented out, which may reduce test coverage. Consider reactivating or removing this code if it is no longer needed.
♻️ Duplicate comments (20)
internal/servers/server/option_test.go (1)
3727-4066:⚠️ Potential issueImplement the commented-out gRPC option tests
These test functions continue to be commented out and are not implementing any actual test logic. As part of the E2E strategic testing framework improvements, these tests should be properly implemented rather than left as placeholders.
Following the same pattern as other tests in this file, each function needs:
- A test struct with name, arguments, and a checkFunc
- Test cases with success and failure scenarios
- Implementation that verifies the option behavior when applied to a server
-// func TestWithGRPCMaxConcurrentStreams(t *testing.T) { +func TestWithGRPCMaxConcurrentStreams(t *testing.T) { + type test struct { + name string + size uint32 + checkFunc func(opt Option) error + } + + tests := []test{ + { + name: "set success when size is more than 0", + size: 1024, + checkFunc: func(opt Option) error { + got := new(server) + opt(got) + + if len(got.grpc.opts) != 1 { + return errors.New("invalid param was set") + } + return nil + }, + }, + { + name: "not set when size is 0", + checkFunc: func(opt Option) error { + gopts := []grpc.ServerOption{ + grpc.ConnectionTimeout(10 * time.Second), + } + got := new(server) + got.grpc.opts = gopts + opt(got) + + if !reflect.DeepEqual(got.grpc.opts, gopts) { + return errors.New("invalid param was set") + } + return nil + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + opt := WithGRPCMaxConcurrentStreams(tt.size) + if err := tt.checkFunc(opt); err != nil { + t.Error(err) + } + }) + } +}The same pattern should be implemented for the other commented test functions:
TestWithGRPCNumStreamWorkersTestWithGRPCSharedWriteBufferTestWithGRPCWaitForHandlersinternal/net/grpc/pool/pool.go (1)
550-553:⚠️ Potential issueFix potential double-close in error handling.
If the backoff operation fails and the connection is not nil, the code attempts to close the connection twice: once with
_ = conn.Close()and then withconn.Close()in theerrors.Join. This could lead to unexpected behavior.- if err != nil && conn != nil { - _ = conn.Close() - return nil, errors.Join(err, conn.Close()) - } + if err != nil && conn != nil { + closeErr := conn.Close() + return nil, errors.Join(err, closeErr) + }internal/net/quic/conn_test.go (6)
1-16:⚠️ Potential issueEntire file is commented out and marked as "NOT IMPLEMENTED"
The entire file is commented out and labeled as "NOT IMPLEMENTED BELOW" on line 16. This test file contains well-structured test skeletons, but no actual implementations. To make this file functional, you need to uncomment the code and implement actual test cases.
To enable this file:
- Uncomment all the code
- Add the required imports:
import ( "context" "crypto/tls" "net" "reflect" "sync" "testing" "go.uber.org/goleak" "github.com/quic-go/quic-go" // or appropriate quic package "github.com/vdaas/vald/internal/errors" // based on custom Errorf usage )
44-83: 🛠️ Refactor suggestionImplement actual test cases instead of placeholder comments
The test file contains thorough test structure and boilerplate, but no actual test cases. Both commented-out templates (lines 46-62 and 65-82) are providing good structure but need real implementations.
Implement at least one basic test case for
TestNewConnto validate the structure:tests := []test{ { name: "successful_connection", args: args{ ctx: context.Background(), conn: &mockQuicConnection{}, }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: &mockQuicStream{}, }, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "null_connection", args: args{ ctx: context.Background(), conn: nil, }, want: want{ want: nil, err: errors.New("connection is nil"), }, checkFunc: defaultCheckFunc, }, }
471-476: 🛠️ Refactor suggestionImplement mock QUIC connections for testing
For effective testing, you'll need mock implementations of
quic.Connectionandquic.Streaminterfaces. These mocks should be created to facilitate testing different scenarios, including error cases.Create mock implementations like this:
type mockQuicConnection struct { closeFunc func() error openStreamFunc func(context.Context) (quic.Stream, error) // other methods required by the quic.Connection interface } func (m *mockQuicConnection) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } func (m *mockQuicConnection) OpenStream(ctx context.Context) (quic.Stream, error) { if m.openStreamFunc != nil { return m.openStreamFunc(ctx) } return &mockQuicStream{}, nil } // Implement other required methods type mockQuicStream struct { readFunc func([]byte) (int, error) writeFunc func([]byte) (int, error) closeFunc func() error // other methods required by the quic.Stream interface } func (m *mockQuicStream) Read(b []byte) (int, error) { if m.readFunc != nil { return m.readFunc(b) } return 0, io.EOF } func (m *mockQuicStream) Write(b []byte) (int, error) { if m.writeFunc != nil { return m.writeFunc(b) } return len(b), nil } func (m *mockQuicStream) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } // Implement other required methods
110-200: 🛠️ Refactor suggestionImplement test cases for
TestConn_CloseThe
TestConn_Closefunction is crucial for ensuring resources are properly released. Implement meaningful test cases to validate that connections are closed correctly and errors are handled properly.Add at least these basic test cases:
tests := []test{ { name: "successful_close", fields: fields{ Connection: &mockQuicConnection{ closeFunc: func() error { return nil }, }, Stream: &mockQuicStream{ closeFunc: func() error { return nil }, }, }, want: want{ err: nil, }, checkFunc: defaultCheckFunc, }, { name: "stream_close_error", fields: fields{ Connection: &mockQuicConnection{ closeFunc: func() error { return nil }, }, Stream: &mockQuicStream{ closeFunc: func() error { return errors.New("failed to close stream") }, }, }, want: want{ err: errors.New("failed to close stream"), }, checkFunc: defaultCheckFunc, }, { name: "connection_close_error", fields: fields{ Connection: &mockQuicConnection{ closeFunc: func() error { return errors.New("failed to close connection") }, }, Stream: &mockQuicStream{ closeFunc: func() error { return nil }, }, }, want: want{ err: errors.New("failed to close connection"), }, checkFunc: defaultCheckFunc, }, }
297-403: 🛠️ Refactor suggestionImplement test cases for
Test_qconn_dialQuicContextThe
Test_qconn_dialQuicContextfunction tests advanced QUIC dial logic with connection caching. Add real test cases to validate connection caching, reconnect logic, and error handling.Add test cases like:
tests := []test{ { name: "successful_new_connection", args: args{ ctx: context.Background(), addr: "example.com:443", tcfg: &tls.Config{}, }, fields: fields{ connectionCache: sync.Map[string, quic.Connection]{}, }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: &mockQuicStream{}, }, err: nil, }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T, args args) { // Setup mock dialer or expectations }, }, { name: "reuse_cached_connection", args: args{ ctx: context.Background(), addr: "example.com:443", tcfg: &tls.Config{}, }, fields: fields{ connectionCache: func() sync.Map[string, quic.Connection] { m := sync.Map[string, quic.Connection]{} m.Store("example.com:443", &mockQuicConnection{}) return m }(), }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: &mockQuicStream{}, }, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "dial_error", args: args{ ctx: context.Background(), addr: "invalid:443", tcfg: &tls.Config{}, }, fields: fields{ connectionCache: sync.Map[string, quic.Connection]{}, }, want: want{ want: nil, err: errors.New("dial error"), }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T, args args) { // Setup expectations for dial failure }, }, }
405-491: 🛠️ Refactor suggestionImplement test cases for
Test_qconn_CloseThe
Test_qconn_Closefunction tests closing all cached QUIC connections. Add test cases to verify proper closure of connections and error handling.Add test cases like:
tests := []test{ { name: "successfully_close_all_connections", fields: fields{ connectionCache: func() sync.Map[string, quic.Connection] { m := sync.Map[string, quic.Connection]{} m.Store("conn1", &mockQuicConnection{ closeFunc: func() error { return nil }, }) m.Store("conn2", &mockQuicConnection{ closeFunc: func() error { return nil }, }) return m }(), }, want: want{ err: nil, }, checkFunc: defaultCheckFunc, }, { name: "error_closing_one_connection", fields: fields{ connectionCache: func() sync.Map[string, quic.Connection] { m := sync.Map[string, quic.Connection]{} m.Store("conn1", &mockQuicConnection{ closeFunc: func() error { return nil }, }) m.Store("conn2", &mockQuicConnection{ closeFunc: func() error { return errors.New("failed to close connection") }, }) return m }(), }, want: want{ err: errors.New("failed to close connection"), }, checkFunc: defaultCheckFunc, }, { name: "empty_connection_cache", fields: fields{ connectionCache: sync.Map[string, quic.Connection]{}, }, want: want{ err: nil, }, checkFunc: defaultCheckFunc, }, }internal/params/option.go (1)
27-28:⚠️ Potential issueEnhance os.Args[0] handling in package-level initialization.
Using
filepath.Base(os.Args[0])directly in the package-level initialization is risky whenos.Argsis empty or manipulated, especially in tests or specific environments.Consider implementing a safer approach:
-var defaultOptions = []Option{ - WithName(filepath.Base(os.Args[0])), +// getDefaultName returns the base name of the executable or a fallback value +func getDefaultName() string { + if len(os.Args) > 0 { + return filepath.Base(os.Args[0]) + } + return "default-name" // or another appropriate fallback +} + +var defaultOptions = []Option{ + WithName(getDefaultName()),pkg/index/job/deletion/service/deleter_test.go (6)
1-15:⚠️ Potential issueUncomment the file and add required imports
The entire test file is commented out, making these tests non-functional. The file needs to be uncommented and properly imported to be executable.
package service +import ( + "context" + "reflect" + "testing" + + "github.com/vdaas/vald/internal/errors" + "github.com/vdaas/vald/internal/net/grpc" + "github.com/vdaas/vald/apis/grpc/v1/payload" + "github.com/vdaas/vald/apis/grpc/v1/vald" + "github.com/vdaas/vald/pkg/discoverer" + "go.uber.org/goleak" +)
16-105: 🛠️ Refactor suggestionImplement TestNew test cases
The test cases for TestNew are missing. This function needs concrete test cases to validate the creation of the Deleter with various configuration options.
-// NOT IMPLEMENTED BELOW -// -// func TestNew(t *testing.T) { +func TestNew(t *testing.T) { -// type args struct { -// opts []Option -// } + type args struct { + opts []Option + } -// type want struct { -// want Deleter -// err error -// } + type want struct { + want Deleter + err error + } -// type test struct { -// name string -// args args -// want want -// checkFunc func(want, Deleter, error) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } + type test struct { + name string + args args + want want + checkFunc func(want, Deleter, error) error + beforeFunc func(*testing.T, args) + afterFunc func(*testing.T, args) + } -// defaultCheckFunc := func(w want, got Deleter, err error) error { -// if !errors.Is(err, w.err) { -// return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) -// } -// if !reflect.DeepEqual(got, w.want) { -// return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) -// } -// return nil -// } + defaultCheckFunc := func(w want, got Deleter, err error) error { + if !errors.Is(err, w.err) { + return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) + } + if !reflect.DeepEqual(got, w.want) { + return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) + } + return nil + } -// tests := []test{ -// // TODO test cases + tests := []test{ + { + name: "create_with_default_options", + args: args{ + opts: []Option{}, + }, + want: want{ + want: &index{ + concurrency: 1, // Default value + }, + err: nil, + }, + }, + { + name: "create_with_custom_concurrency", + args: args{ + opts: []Option{ + WithConcurrency(5), + }, + }, + want: want{ + want: &index{ + concurrency: 5, + }, + err: nil, + }, + }, + { + name: "create_with_target_index_id", + args: args{ + opts: []Option{ + WithTargetIndexID("test-index"), + }, + }, + want: want{ + want: &index{ + targetIndexID: "test-index", + concurrency: 1, // Default value + }, + err: nil, + }, + }, + }Then continue with the rest of the test function, but uncommented:
-// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// -// got, err := New(test.args.opts...) -// if err := checkFunc(test.want, got, err); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// } + for _, tc := range tests { + test := tc + t.Run(test.name, func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + if test.beforeFunc != nil { + test.beforeFunc(tt, test.args) + } + if test.afterFunc != nil { + defer test.afterFunc(tt, test.args) + } + checkFunc := test.checkFunc + if test.checkFunc == nil { + checkFunc = defaultCheckFunc + } + + got, err := New(test.args.opts...) + if err := checkFunc(test.want, got, err); err != nil { + tt.Errorf("error = %v", err) + } + }) + } +}
106-190: 🛠️ Refactor suggestionImplement Test_delDuplicateAddrs test cases
The test cases for the address deduplication function are missing. Implement concrete test cases that cover empty slices, non-duplicate entries, and duplicate entries.
-// func Test_delDuplicateAddrs(t *testing.T) { +func Test_delDuplicateAddrs(t *testing.T) { -// type args struct { -// targetAddrs []string -// } + type args struct { + targetAddrs []string + } -// type want struct { -// want []string -// } + type want struct { + want []string + } -// type test struct { -// name string -// args args -// want want -// checkFunc func(want, []string) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } + type test struct { + name string + args args + want want + checkFunc func(want, []string) error + beforeFunc func(*testing.T, args) + afterFunc func(*testing.T, args) + } -// defaultCheckFunc := func(w want, got []string) error { -// if !reflect.DeepEqual(got, w.want) { -// return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) -// } -// return nil -// } + defaultCheckFunc := func(w want, got []string) error { + if !reflect.DeepEqual(got, w.want) { + return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) + } + return nil + } -// tests := []test{ -// // TODO test cases + tests := []test{ + { + name: "empty_slice", + args: args{ + targetAddrs: []string{}, + }, + want: want{ + want: []string{}, + }, + }, + { + name: "no_duplicates", + args: args{ + targetAddrs: []string{"addr1", "addr2", "addr3"}, + }, + want: want{ + want: []string{"addr1", "addr2", "addr3"}, + }, + }, + { + name: "with_duplicates", + args: args{ + targetAddrs: []string{"addr1", "addr2", "addr1", "addr3", "addr2"}, + }, + want: want{ + want: []string{"addr1", "addr2", "addr3"}, + }, + },Then continue with the rest of the test function, but uncommented:
-// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// -// got := delDuplicateAddrs(test.args.targetAddrs) -// if err := checkFunc(test.want, got); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// } + for _, tc := range tests { + test := tc + t.Run(test.name, func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + if test.beforeFunc != nil { + test.beforeFunc(tt, test.args) + } + if test.afterFunc != nil { + defer test.afterFunc(tt, test.args) + } + checkFunc := test.checkFunc + if test.checkFunc == nil { + checkFunc = defaultCheckFunc + } + + got := delDuplicateAddrs(test.args.targetAddrs) + if err := checkFunc(test.want, got); err != nil { + tt.Errorf("error = %v", err) + } + }) + } +}
191-304: 🛠️ Refactor suggestionImplement Test_index_StartClient with mock client
The test function needs to use a mock client implementation to properly test different scenarios of client initialization.
First, add a mock client implementation at the top of the file:
+// MockClient implements the discoverer.Client interface for testing +type mockClient struct { + mockStartClient func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) + mockClose func() error +} + +func (m *mockClient) StartClient(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + if m.mockStartClient != nil { + return m.mockStartClient(ctx, addr, targetIndexID, opts...) + } + return nil, errors.New("mock not implemented") +} + +func (m *mockClient) Close() error { + if m.mockClose != nil { + return m.mockClose() + } + return nil +} + +// MockRemoveClient implements the vald.RemoveClient interface for testing +type mockRemoveClient struct { + mockRemove func(ctx context.Context, req *payload.Remove_Request, opts ...grpc.CallOption) (*payload.Object_Location, error) + mockStreamRemove func(ctx context.Context, opts ...grpc.CallOption) (vald.Remove_StreamClient, error) +} + +func (m *mockRemoveClient) Remove(ctx context.Context, req *payload.Remove_Request, opts ...grpc.CallOption) (*payload.Object_Location, error) { + if m.mockRemove != nil { + return m.mockRemove(ctx, req, opts...) + } + return nil, errors.New("mock not implemented") +} + +func (m *mockRemoveClient) StreamRemove(ctx context.Context, opts ...grpc.CallOption) (vald.Remove_StreamClient, error) { + if m.mockStreamRemove != nil { + return m.mockStreamRemove(ctx, opts...) + } + return nil, errors.New("mock not implemented") +}Then implement the test cases:
-// func Test_index_StartClient(t *testing.T) { +func Test_index_StartClient(t *testing.T) { -// type args struct { -// ctx context.Context -// } + type args struct { + ctx context.Context + } -// type fields struct { -// client discoverer.Client -// targetAddrs []string -// targetIndexID string -// concurrency int -// } + type fields struct { + client discoverer.Client + targetAddrs []string + targetIndexID string + concurrency int + } -// type want struct { -// want <-chan error -// err error -// } + type want struct { + want <-chan error + err error + } -// type test struct { -// name string -// args args -// fields fields -// want want -// checkFunc func(want, <-chan error, error) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } + type test struct { + name string + args args + fields fields + want want + checkFunc func(want, <-chan error, error) error + beforeFunc func(*testing.T, args) + afterFunc func(*testing.T, args) + } -// defaultCheckFunc := func(w want, got <-chan error, err error) error { -// if !errors.Is(err, w.err) { -// return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) -// } -// if !reflect.DeepEqual(got, w.want) { -// return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) -// } -// return nil -// } + defaultCheckFunc := func(w want, got <-chan error, err error) error { + if !errors.Is(err, w.err) { + return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) + } + // Skip comparison of channels as they can't be directly compared + return nil + } -// tests := []test{ -// // TODO test cases + tests := []test{ + { + name: "successful_client_initialization", + args: args{ + ctx: context.Background(), + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return &mockRemoveClient{}, nil + }, + }, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: nil, + }, + }, + { + name: "client_initialization_error", + args: args{ + ctx: context.Background(), + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return nil, errors.New("failed to connect") + }, + }, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: nil, // Error will be sent through the channel, not returned directly + }, + }, + { + name: "context_cancellation", + args: args{ + ctx: func() context.Context { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + return ctx + }(), + }, + fields: fields{ + client: &mockClient{}, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: context.Canceled, + }, + },Then continue with the rest of the test function, but uncommented:
-// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// idx := &index{ -// client: test.fields.client, -// targetAddrs: test.fields.targetAddrs, -// targetIndexID: test.fields.targetIndexID, -// concurrency: test.fields.concurrency, -// } -// -// got, err := idx.StartClient(test.args.ctx) -// if err := checkFunc(test.want, got, err); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// } + for _, tc := range tests { + test := tc + t.Run(test.name, func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + if test.beforeFunc != nil { + test.beforeFunc(tt, test.args) + } + if test.afterFunc != nil { + defer test.afterFunc(tt, test.args) + } + checkFunc := test.checkFunc + if test.checkFunc == nil { + checkFunc = defaultCheckFunc + } + idx := &index{ + client: test.fields.client, + targetAddrs: test.fields.targetAddrs, + targetIndexID: test.fields.targetIndexID, + concurrency: test.fields.concurrency, + } + + got, err := idx.StartClient(test.args.ctx) + if err := checkFunc(test.want, got, err); err != nil { + tt.Errorf("error = %v", err) + } + }) + } +}
305-414: 🛠️ Refactor suggestionImplement test cases for Test_index_Start
The test function needs concrete test cases to verify the behavior of the Start method, including successful starts and error scenarios.
-// func Test_index_Start(t *testing.T) { +func Test_index_Start(t *testing.T) { -// type args struct { -// ctx context.Context -// } + type args struct { + ctx context.Context + } -// type fields struct { -// client discoverer.Client -// targetAddrs []string -// targetIndexID string -// concurrency int -// } + type fields struct { + client discoverer.Client + targetAddrs []string + targetIndexID string + concurrency int + } -// type want struct { -// err error -// } + type want struct { + err error + } -// type test struct { -// name string -// args args -// fields fields -// want want -// checkFunc func(want, error) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } + type test struct { + name string + args args + fields fields + want want + checkFunc func(want, error) error + beforeFunc func(*testing.T, args) + afterFunc func(*testing.T, args) + } -// defaultCheckFunc := func(w want, err error) error { -// if !errors.Is(err, w.err) { -// return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) -// } -// return nil -// } + defaultCheckFunc := func(w want, err error) error { + if !errors.Is(err, w.err) { + return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) + } + return nil + } -// tests := []test{ -// // TODO test cases + tests := []test{ + { + name: "successful_start", + args: args{ + ctx: context.Background(), + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return &mockRemoveClient{}, nil + }, + }, + targetAddrs: []string{"addr1", "addr2"}, + targetIndexID: "test-index", + concurrency: 2, + }, + want: want{ + err: nil, + }, + }, + { + name: "empty_target_addresses", + args: args{ + ctx: context.Background(), + }, + fields: fields{ + client: &mockClient{}, + targetAddrs: []string{}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: errors.New("no target addresses specified"), + }, + }, + { + name: "context_canceled", + args: args{ + ctx: func() context.Context { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + return ctx + }(), + }, + fields: fields{ + client: &mockClient{}, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: context.Canceled, + }, + },Then continue with the rest of the test function, but uncommented:
-// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// idx := &index{ -// client: test.fields.client, -// targetAddrs: test.fields.targetAddrs, -// targetIndexID: test.fields.targetIndexID, -// concurrency: test.fields.concurrency, -// } -// -// err := idx.Start(test.args.ctx) -// if err := checkFunc(test.want, err); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// } + for _, tc := range tests { + test := tc + t.Run(test.name, func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + if test.beforeFunc != nil { + test.beforeFunc(tt, test.args) + } + if test.afterFunc != nil { + defer test.afterFunc(tt, test.args) + } + checkFunc := test.checkFunc + if test.checkFunc == nil { + checkFunc = defaultCheckFunc + } + idx := &index{ + client: test.fields.client, + targetAddrs: test.fields.targetAddrs, + targetIndexID: test.fields.targetIndexID, + concurrency: test.fields.concurrency, + } + + err := idx.Start(test.args.ctx) + if err := checkFunc(test.want, err); err != nil { + tt.Errorf("error = %v", err) + } + }) + } +}
415-527: 🛠️ Refactor suggestionAdd error handling tests for Test_index_doDeleteIndex
The test function should include test cases for happy path and various error conditions to ensure the robustness of the deletion process.
-// func Test_index_doDeleteIndex(t *testing.T) { +func Test_index_doDeleteIndex(t *testing.T) { -// type args struct { -// ctx context.Context -// fn func(_ context.Context, _ vald.RemoveClient, _ ...grpc.CallOption) (*payload.Object_Location, error) -// } + type args struct { + ctx context.Context + fn func(_ context.Context, _ vald.RemoveClient, _ ...grpc.CallOption) (*payload.Object_Location, error) + } -// type fields struct { -// client discoverer.Client -// targetAddrs []string -// targetIndexID string -// concurrency int -// } + type fields struct { + client discoverer.Client + targetAddrs []string + targetIndexID string + concurrency int + } -// type want struct { -// err error -// } + type want struct { + err error + } -// type test struct { -// name string -// args args -// fields fields -// want want -// checkFunc func(want, error) error -// beforeFunc func(*testing.T, args) -// afterFunc func(*testing.T, args) -// } + type test struct { + name string + args args + fields fields + want want + checkFunc func(want, error) error + beforeFunc func(*testing.T, args) + afterFunc func(*testing.T, args) + } -// defaultCheckFunc := func(w want, err error) error { -// if !errors.Is(err, w.err) { -// return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) -// } -// return nil -// } + defaultCheckFunc := func(w want, err error) error { + if !errors.Is(err, w.err) { + return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) + } + return nil + } -// tests := []test{ -// // TODO test cases + tests := []test{ + { + name: "successful_delete", + args: args{ + ctx: context.Background(), + fn: func(_ context.Context, _ vald.RemoveClient, _ ...grpc.CallOption) (*payload.Object_Location, error) { + return &payload.Object_Location{}, nil + }, + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return &mockRemoveClient{}, nil + }, + }, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: nil, + }, + }, + { + name: "network_error", + args: args{ + ctx: context.Background(), + fn: func(_ context.Context, _ vald.RemoveClient, _ ...grpc.CallOption) (*payload.Object_Location, error) { + return nil, errors.New("network error") + }, + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return &mockRemoveClient{}, nil + }, + }, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: errors.New("network error"), + }, + }, + { + name: "context_cancellation", + args: args{ + ctx: func() context.Context { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + return ctx + }(), + fn: func(_ context.Context, _ vald.RemoveClient, _ ...grpc.CallOption) (*payload.Object_Location, error) { + return &payload.Object_Location{}, nil + }, + }, + fields: fields{ + client: &mockClient{ + mockStartClient: func(ctx context.Context, addr string, targetIndexID string, opts ...grpc.CallOption) (vald.RemoveClient, error) { + return &mockRemoveClient{}, nil + }, + }, + targetAddrs: []string{"addr1"}, + targetIndexID: "test-index", + concurrency: 1, + }, + want: want{ + err: context.Canceled, + }, + },Then continue with the rest of the test function, but uncommented:
-// for _, tc := range tests { -// test := tc -// t.Run(test.name, func(tt *testing.T) { -// tt.Parallel() -// defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) -// if test.beforeFunc != nil { -// test.beforeFunc(tt, test.args) -// } -// if test.afterFunc != nil { -// defer test.afterFunc(tt, test.args) -// } -// checkFunc := test.checkFunc -// if test.checkFunc == nil { -// checkFunc = defaultCheckFunc -// } -// idx := &index{ -// client: test.fields.client, -// targetAddrs: test.fields.targetAddrs, -// targetIndexID: test.fields.targetIndexID, -// concurrency: test.fields.concurrency, -// } -// -// err := idx.doDeleteIndex(test.args.ctx, test.args.fn) -// if err := checkFunc(test.want, err); err != nil { -// tt.Errorf("error = %v", err) -// } -// }) -// } -// } + for _, tc := range tests { + test := tc + t.Run(test.name, func(tt *testing.T) { + tt.Parallel() + defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) + if test.beforeFunc != nil { + test.beforeFunc(tt, test.args) + } + if test.afterFunc != nil { + defer test.afterFunc(tt, test.args) + } + checkFunc := test.checkFunc + if test.checkFunc == nil { + checkFunc = defaultCheckFunc + } + idx := &index{ + client: test.fields.client, + targetAddrs: test.fields.targetAddrs, + targetIndexID: test.fields.targetIndexID, + concurrency: test.fields.concurrency, + } + + err := idx.doDeleteIndex(test.args.ctx, test.args.fn) + if err := checkFunc(test.want, err); err != nil { + tt.Errorf("error = %v", err) + } + }) + } +}internal/timeutil/time_test.go (2)
265-349:⚠️ Potential issueUncomment and implement the test function.
The
TestDurationString_DurationWithDefaultfunction is commented out but has a good test structure. This test is necessary to verify that theDurationWithDefaultmethod works correctly, especially for handling default values.Apply this diff to uncomment and implement the test function with concrete test cases:
-// -// func TestDurationString_DurationWithDefault(t *testing.T) { +func TestDurationString_DurationWithDefault(t *testing.T) {Then add test cases that cover:
- Valid duration string processing
- Fallback to default value for empty strings
- Fallback to default value for invalid strings
184-263:⚠️ Potential issueUncomment and implement the test function.
The
TestDurationString_Durationfunction is well-structured but still commented out. This test function is essential for ensuring that theDuration()method works correctly for various inputs.Apply this diff to uncomment and implement the test function with concrete test cases:
-// -// func TestDurationString_Duration(t *testing.T) { +func TestDurationString_Duration(t *testing.T) {Then add test cases that cover:
- Valid duration strings (e.g., "1s", "1m", "1h")
- Empty string handling
- Invalid format handling
internal/test/data/vector/noise/option_test.go (3)
103-186:⚠️ Potential issueUncomment and implement the test function for WithTableDivisionFactor.
The
TestWithTableDivisionFactorfunction is commented out but has a good structure. This test is essential for ensuring that the table division factor is correctly applied in the noise generation process.Apply this diff to uncomment and implement the test function with concrete test cases:
-// -// func TestWithTableDivisionFactor(t *testing.T) { +func TestWithTableDivisionFactor(t *testing.T) {Then add test cases that cover:
- Normal values
- Edge cases (e.g., minimum allowed value, very large values)
- Zero or invalid values if applicable
188-271:⚠️ Potential issueUncomment and implement the test function for WithMinTableSize.
The
TestWithMinTableSizefunction is commented out. This test is necessary to verify that the minimum table size option is correctly applied when generating noise tables.Apply this diff to uncomment and implement the test function with concrete test cases:
-// -// func TestWithMinTableSize(t *testing.T) { +func TestWithMinTableSize(t *testing.T) {Then add test cases that cover:
- Normal values
- Edge cases (e.g., 0, very large values)
- Values that might trigger boundary conditions in the implementation
18-101:⚠️ Potential issueUncomment and implement the test function for WithLevelFactor.
The
TestWithLevelFactorfunction is well-structured but still commented out. This test is crucial for verifying that theWithLevelFactoroption behaves correctly with different input values.Apply this diff to uncomment and implement the test function with concrete test cases:
-// -// func TestWithLevelFactor(t *testing.T) { +func TestWithLevelFactor(t *testing.T) {Then add test cases that cover:
- Normal values (e.g., 1.0, 2.0)
- Edge cases (e.g., 0.0, very large values)
- Negative values if applicable
🧹 Nitpick comments (22)
apis/swagger/v1/vald/remove.swagger.json (3)
17-18: Enhanced Documentation for/removeEndpoint:
The updated summary now provides a clear overview of the Remove RPC with a detailed status code table and troubleshooting guidance. This enhancement should help API consumers better understand the endpoint’s behavior. Please verify that the documented status codes match the actual responses.
50-51: Enhanced Documentation for/remove/multipleEndpoint:
The revised summary clearly reflects the multi-remove functionality, and the new description adds a valuable notice regarding gRPC message size limitations along with structured troubleshooting details. Ensure that the HTML markup (e.g., the<div class="notice">) renders correctly in Swagger UI.
83-84: Improved Documentation for/remove/timestampEndpoint:
The updated summary and description effectively clarify that this endpoint handles vector removals based on timestamps, including details about how multiple timestamps create anANDcondition. It provides comprehensive troubleshooting information that will improve usability. Please confirm that the embedded HTML and formatting render as expected in the API documentation viewers..github/workflows/codeql-analysis.yml (1)
46-51: Job Creation: New "analyze" Job Setup
The newly introducedanalyzejob correctly establishes the CodeQL analysis with configurable environment variables and leverages a matrix strategy for multi-language support. However, the conditional expression forruns-oncould be more explicit. For improved clarity and maintainability, consider using the ternary operator.- runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }} + runs-on: ${{ matrix.language == 'swift' ? 'macos-latest' : 'ubuntu-latest' }}apis/swagger/v1/vald/insert.swagger.json (1)
50-50: Refine MultiInsert Summary WordingConsider rephrasing "1 request" to "a single request" for improved clarity and readability.
internal/file/file.go (1)
374-391: New absolute path utility functionThe newly added
AbsolutePathfunction provides a useful utility for standardizing path handling. However, there are opportunities to improve error handling.Consider logging or handling errors from
os.Getwd()andfilepath.Abs(), as they're currently silently ignored.func AbsolutePath(path string) string { if path == "" { return "" } if !filepath.IsAbs(path) { root, err := os.Getwd() - if err == nil { + if err != nil { + log.Debug(errors.Wrap(err, "failed to get working directory")) + } else { path = joinFilePaths(root, path) } if !filepath.IsAbs(path) { absPath, err := filepath.Abs(path) - if err == nil { + if err != nil { + log.Debug(errors.Wrap(err, "failed to get absolute path")) + } else { path = absPath } } } return filepath.Clean(path) }apis/swagger/v1/vald/search.swagger.json (5)
50-51: Detailed Upgrade for/linearsearch/idEndpoint
The revisedsummaryanddescriptionnow include explicit instructions for searching by vector ID and inform users about potentialNOT_FOUNDerrors. Ensure that the usage of HTML line breaks (\u003cbr\u003e) is consistent with other endpoints and renders as intended in the documentation viewer.
116-117: Improved Summary for/linearsearch/multipleEndpoint
The updatedsummaryanddescriptionnow clearly outline that multiple vectors can be searched in one request, complete with a status code table and troubleshooting steps. Consider verifying if the repeated troubleshooting information across endpoints could be centralized to reduce maintenance overhead.
149-150: Clear and Consistent Update for/searchEndpoint
The modifications deliver a well-structured overview and a detailed error handling guide. The consistency in table formatting across endpoints is appreciated. It would be beneficial to double-check that the lengthy error description meets any character limits set by your API documentation tools.
182-183: Refined Documentation for/search/idEndpoint
The enhancedsummaryanddescriptionprovide clear guidance on searching by vector ID. As with the other endpoints, ensure that HTML elements like<br>are uniformly rendered and consider aligning the formatting style with similar endpoints.
248-249: Enhanced Multi-Vector Search Overview for/search/multipleEndpoint
The updatedsummaryanddescriptionclearly define the multi-vector search functionality. The troubleshooting table and status code details match other endpoints, which is good for consistency; however, consider whether the repeated content might be abstracted to a shared reference for easier maintenance.apis/docs/v1/docs.md (5)
1643-1646: Filter Service RPC Methods Documentation:
The table rows for theSearchObject,MultiSearchObject, andStreamSearchObjectRPCs now include overviews and detailed status code information. Ensure that the differences between these methods are clearly communicated—particularly the intended use cases for single versus streaming (bidirectional) operations—and that the status code listings are consistently formatted throughout the document.
1636-1639: Filter Service Overview Update:
The added overview explains that the Filter service supports insert, update, upsert, and search operations and notes that it forwards user requests to custom ingress/egress filter components. For improved clarity, consider rephrasing “allowing user to run custom logic” to “enabling custom filtering logic.”
1928-1929: UpdateTimestamp Method Documentation:
The documentation for theUpdateTimestampRPC now includes an overview; however, placeholders for “Status Code TODO” and “Troubleshooting TODO” remain. It is important to complete these sections before merging so users have complete guidance.
1945-1946: Upsert RPC Documentation:
The table row for theUpsertRPC now describes its dual functionality—to update an inserted vector to a new single vector or add a new one if not present. Please review the status code information and overall formatting to ensure it integrates seamlessly with the surrounding documentation.
1-1982: Overall Documentation Improvements:
This PR introduces extensive updates to the API documentation that enhance clarity by adding “Overview” sections and detailed method descriptions with status code tables. Please ensure that all “TODO” placeholders are finalized before merging and that terminology remains consistent across all service sections. Consider a final pass to verify that formatting (especially within tables) is uniform throughout the document.🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
688-688: Link fragments should be valid
null(MD051, link-fragments)
782-782: Link fragments should be valid
null(MD051, link-fragments)
826-826: Link fragments should be valid
null(MD051, link-fragments)
837-837: Link fragments should be valid
null(MD051, link-fragments)
848-848: Link fragments should be valid
null(MD051, link-fragments)
859-859: Link fragments should be valid
null(MD051, link-fragments)
997-997: Link fragments should be valid
null(MD051, link-fragments)
1094-1094: Link fragments should be valid
null(MD051, link-fragments)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1514-1514: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Reference links and images should use a label that is defined
Missing link or image reference definition: "google.rpc.code.permissiondenied"(MD052, reference-links-images)
1624-1624: Link fragments should be valid
null(MD051, link-fragments)
internal/net/grpc/pool/pool.go (1)
663-696: Consider configurable timeout for IP address pinging.The current implementation uses a hard-coded 10ms timeout for pinging IP addresses during DNS resolution. This value might be too short for some network conditions.
- pingCtx, cancel := context.WithTimeout(ctx, 10*time.Millisecond) + // Use a configurable ping timeout with a reasonable default + pingTimeout := p.pingTimeout + if pingTimeout == 0 { + pingTimeout = 100 * time.Millisecond + } + pingCtx, cancel := context.WithTimeout(ctx, pingTimeout)Also consider adding a field to the
poolstruct to make this timeout configurable:type pool struct { // existing fields pingTimeout time.Duration // Timeout for pinging IP addresses during DNS resolution }internal/params/option.go (1)
89-94: Enhance documentation for WithOverrideDefaultThe current documentation doesn't fully explain what overriding the default flag.CommandLine means and its potential impacts on application behavior.
-// WithOverrideDefault returns Option that overrides default flag.CommandLine. +// WithOverrideDefault returns Option that controls whether to override the default flag.CommandLine. +// When set to true, the parser's flag set will replace the global flag.CommandLine, which affects +// how the standard flag package behaves. This can be useful when integrating with other libraries +// that use the standard flag package, but should be used with caution in concurrent environments. func WithOverrideDefault(flag bool) Option {go.mod (2)
6-18: Replace Block Dependency Version Updates
Multiple dependency versions in thereplace (block have been bumped to newer releases (e.g., updates for Cloud dependencies, Azure SDK components, and AWS SDK v2 sub-packages). This consolidated update is intended to keep the module in sync with current releases.
Please verify the following:
- All version changes have been tested for compatibility with the consuming code.
- No breaking changes have been introduced by these upstream changes.
It is recommended to rungo mod tidyand the full test suite to ensure stability after these updates.Also applies to: 24-28, 37-39, 48-69, 110-112, 117-124, 126-126, 134-134, 140-140, 148-148, 151-151, 157-157, 263-264
425-544: Indirect Dependency Updates in the Second Require Block
A large number of indirect dependencies have been bumped in the secondrequire (block (lines 425–544). These updates affect libraries used transitively and can influence your build and runtime behavior.
Recommendations:
- Perform a full dependency audit and run end-to-end tests to ensure there are no regressions.
- Review the change notes for key dependencies (for example, updates for
go.opentelemetry.iopackages,k8s.iocomponents, and others) to catch any potential disruptions in functionality.internal/iter/iter.go (1)
70-78: Consider out-of-range and overflow checks inAt(i uint64).
Althoughiis wrapped by(c.start + i) % c.size, extremely large values ofirisk integer overflow when computingc.start + i. This may not be critical for typical usage, but you can add checks or doc comments clarifying expected bounds fori.internal/errors/k8s.go (1)
20-46: Standardize error message casing
Some error message strings start with uppercase letters (e.g., "Undefined namespace") while others are lowercase (e.g., "port forward address not found"). For consistency and clarity, consider standardizing the casing across all newly introduced error messages.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
apis/grpc/v1/agent/core/agent.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/agent/sidecar/sidecar.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/discoverer/discoverer.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/egress/egress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/ingress/ingress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/meta/meta.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/payload/payload.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/rpc/errdetails/error_details.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goexample/client/go.sumis excluded by!**/*.sumgo.sumis excluded by!**/*.sumrust/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (153)
.gitfiles(15 hunks).github/ISSUE_TEMPLATE/bug_report.md(1 hunks).github/ISSUE_TEMPLATE/security_issue_report.md(1 hunks).github/PULL_REQUEST_TEMPLATE.md(1 hunks).github/workflows/_docker-image.yaml(1 hunks).github/workflows/_release-pr.yaml(1 hunks).github/workflows/build-binaries.yaml(1 hunks).github/workflows/build-protobuf.yaml(1 hunks).github/workflows/chatops.yaml(5 hunks).github/workflows/codeql-analysis.yml(2 hunks).github/workflows/coverage.yaml(1 hunks).github/workflows/e2e-chaos.yaml(4 hunks).github/workflows/e2e-code-bench-agent.yaml(2 hunks).github/workflows/e2e-max-dim.yaml(1 hunks).github/workflows/e2e-profiling.yaml(1 hunks).github/workflows/e2e.yaml(8 hunks).github/workflows/format.yaml(3 hunks).github/workflows/fossa.yaml(1 hunks).github/workflows/helm-lint.yaml(3 hunks).github/workflows/helm.yaml(2 hunks).github/workflows/reviewdog-k8s.yaml(3 hunks).github/workflows/reviewdog.yaml(2 hunks).github/workflows/test-hack.yaml(1 hunks).github/workflows/unit-test-rust.yaml(1 hunks).github/workflows/unit-test.yaml(3 hunks).github/workflows/update-deps.yaml(1 hunks)Makefile(8 hunks)Makefile.d/e2e.mk(1 hunks)Makefile.d/functions.mk(1 hunks)Makefile.d/minikube.mk(1 hunks)Makefile.d/tools.mk(2 hunks)apis/docs/v1/docs.md(10 hunks)apis/swagger/v1/mirror/mirror.swagger.json(1 hunks)apis/swagger/v1/vald/filter.swagger.json(8 hunks)apis/swagger/v1/vald/flush.swagger.json(1 hunks)apis/swagger/v1/vald/index.swagger.json(5 hunks)apis/swagger/v1/vald/insert.swagger.json(2 hunks)apis/swagger/v1/vald/object.swagger.json(4 hunks)apis/swagger/v1/vald/remove.swagger.json(3 hunks)apis/swagger/v1/vald/search.swagger.json(8 hunks)apis/swagger/v1/vald/update.swagger.json(3 hunks)apis/swagger/v1/vald/upsert.swagger.json(2 hunks)charts/vald/values/dev.yaml(1 hunks)cmd/index/job/deletion/main_test.go(1 hunks)dockers/agent/core/agent/Dockerfile(1 hunks)dockers/agent/core/faiss/Dockerfile(1 hunks)dockers/agent/core/ngt/Dockerfile(1 hunks)dockers/agent/sidecar/Dockerfile(1 hunks)dockers/binfmt/Dockerfile(1 hunks)dockers/buildbase/Dockerfile(1 hunks)dockers/buildkit/Dockerfile(1 hunks)dockers/buildkit/syft/scanner/Dockerfile(1 hunks)dockers/ci/base/Dockerfile(1 hunks)dockers/dev/Dockerfile(1 hunks)dockers/discoverer/k8s/Dockerfile(1 hunks)dockers/example/client/Dockerfile(1 hunks)dockers/gateway/filter/Dockerfile(1 hunks)dockers/gateway/lb/Dockerfile(1 hunks)dockers/gateway/mirror/Dockerfile(1 hunks)dockers/index/job/correction/Dockerfile(1 hunks)dockers/index/job/creation/Dockerfile(1 hunks)dockers/index/job/deletion/Dockerfile(1 hunks)dockers/index/job/readreplica/rotate/Dockerfile(1 hunks)dockers/index/job/save/Dockerfile(1 hunks)dockers/index/operator/Dockerfile(1 hunks)dockers/manager/index/Dockerfile(1 hunks)dockers/operator/helm/Dockerfile(1 hunks)dockers/tools/benchmark/job/Dockerfile(1 hunks)dockers/tools/benchmark/operator/Dockerfile(1 hunks)dockers/tools/cli/loadtest/Dockerfile(1 hunks)example/client/go.mod(2 hunks)example/client/go.mod.default(1 hunks)go.mod(14 hunks)hack/go.mod.default(4 hunks)internal/backoff/backoff.go(2 hunks)internal/backoff/backoff_test.go(1 hunks)internal/backoff/option.go(0 hunks)internal/backoff/option_test.go(0 hunks)internal/client/v1/client/discoverer/discover.go(1 hunks)internal/client/v1/client/discoverer/discover_test.go(5 hunks)internal/config/benchmark_test.go(4 hunks)internal/config/config.go(2 hunks)internal/config/config_test.go(1 hunks)internal/config/grpc.go(1 hunks)internal/config/index_deleter_test.go(1 hunks)internal/core/algorithm/usearch/usearch_test.go(16 hunks)internal/db/nosql/cassandra/option_test.go(8 hunks)internal/errors/errors.go(2 hunks)internal/errors/errors_test.go(1 hunks)internal/errors/k8s.go(1 hunks)internal/errors/params.go(1 hunks)internal/file/file.go(3 hunks)internal/file/file_test.go(1 hunks)internal/info/info_test.go(17 hunks)internal/iter/iter.go(1 hunks)internal/net/dialer_test.go(1 hunks)internal/net/grpc/client.go(13 hunks)internal/net/grpc/errdetails/errdetails.go(6 hunks)internal/net/grpc/metadata_test.go(1 hunks)internal/net/grpc/option.go(1 hunks)internal/net/grpc/option_test.go(46 hunks)internal/net/grpc/pool/option.go(1 hunks)internal/net/grpc/pool/option_test.go(6 hunks)internal/net/grpc/pool/pool.go(6 hunks)internal/net/grpc/pool/pool_bench_test.go(2 hunks)internal/net/grpc/server_test.go(2 hunks)internal/net/grpc/status/status.go(1 hunks)internal/net/grpc/status/status_test.go(1 hunks)internal/net/grpc/stream.go(4 hunks)internal/net/grpc/stream_test.go(4 hunks)internal/net/http/client/client.go(2 hunks)internal/net/http/client/client_test.go(3 hunks)internal/net/http/client/option_test.go(4 hunks)internal/net/net.go(1 hunks)internal/net/net_test.go(3 hunks)internal/net/quic/conn_test.go(1 hunks)internal/net/quic/listener_test.go(1 hunks)internal/params/option.go(2 hunks)internal/params/option_test.go(1 hunks)internal/params/params.go(4 hunks)internal/params/params_test.go(12 hunks)internal/rand/rand.go(1 hunks)internal/runner/runner.go(4 hunks)internal/servers/server/option_test.go(1 hunks)internal/servers/server/server.go(1 hunks)internal/servers/servers.go(3 hunks)internal/sync/errgroup/group.go(7 hunks)internal/sync/errgroup/group_test.go(16 hunks)internal/test/data/vector/gen.go(0 hunks)internal/test/data/vector/noise/noise.go(1 hunks)internal/test/data/vector/noise/noise_test.go(1 hunks)internal/test/data/vector/noise/option.go(1 hunks)internal/test/data/vector/noise/option_test.go(1 hunks)internal/test/mock/server_stream.go(1 hunks)internal/timeutil/time.go(3 hunks)internal/timeutil/time_test.go(1 hunks)k8s/agent/ngt/configmap.yaml(1 hunks)k8s/discoverer/deployment.yaml(1 hunks)k8s/gateway/gateway/lb/deployment.yaml(1 hunks)k8s/gateway/gateway/mirror/deployment.yaml(1 hunks)k8s/index/operator/deployment.yaml(1 hunks)k8s/manager/index/deployment.yaml(1 hunks)pkg/agent/core/faiss/service/faiss.go(3 hunks)pkg/agent/core/ngt/service/ngt.go(5 hunks)pkg/agent/core/ngt/service/ngt_test.go(8 hunks)pkg/agent/core/ngt/service/option_test.go(8 hunks)pkg/discoverer/k8s/service/discover_test.go(4 hunks)pkg/gateway/lb/handler/grpc/aggregation_test.go(2 hunks)pkg/gateway/lb/handler/grpc/handler_test.go(2 hunks)pkg/gateway/mirror/service/mirror.go(2 hunks)pkg/index/job/deletion/config/config_test.go(1 hunks)pkg/index/job/deletion/service/deleter_test.go(1 hunks)pkg/index/job/deletion/service/options_test.go(1 hunks)
💤 Files with no reviewable changes (3)
- internal/test/data/vector/gen.go
- internal/backoff/option.go
- internal/backoff/option_test.go
🚧 Files skipped from review as they are similar to previous changes (114)
- dockers/binfmt/Dockerfile
- dockers/buildkit/syft/scanner/Dockerfile
- internal/errors/params.go
- dockers/agent/core/agent/Dockerfile
- dockers/buildkit/Dockerfile
- dockers/agent/sidecar/Dockerfile
- .github/workflows/helm.yaml
- dockers/index/operator/Dockerfile
- .github/workflows/fossa.yaml
- .github/workflows/e2e-profiling.yaml
- .github/workflows/unit-test-rust.yaml
- dockers/gateway/filter/Dockerfile
- dockers/example/client/Dockerfile
- internal/net/grpc/status/status.go
- dockers/index/job/save/Dockerfile
- dockers/tools/benchmark/operator/Dockerfile
- .github/workflows/_docker-image.yaml
- .github/workflows/build-binaries.yaml
- example/client/go.mod.default
- .github/workflows/_release-pr.yaml
- internal/backoff/backoff.go
- dockers/tools/cli/loadtest/Dockerfile
- .github/workflows/helm-lint.yaml
- charts/vald/values/dev.yaml
- .github/workflows/e2e-code-bench-agent.yaml
- dockers/index/job/readreplica/rotate/Dockerfile
- .github/workflows/build-protobuf.yaml
- internal/config/benchmark_test.go
- dockers/gateway/mirror/Dockerfile
- k8s/discoverer/deployment.yaml
- .github/workflows/unit-test.yaml
- .github/workflows/reviewdog.yaml
- internal/net/grpc/option.go
- dockers/manager/index/Dockerfile
- .github/workflows/e2e-chaos.yaml
- dockers/tools/benchmark/job/Dockerfile
- dockers/index/job/correction/Dockerfile
- .github/workflows/update-deps.yaml
- internal/backoff/backoff_test.go
- internal/rand/rand.go
- dockers/dev/Dockerfile
- dockers/buildbase/Dockerfile
- internal/net/net.go
- .github/workflows/chatops.yaml
- dockers/discoverer/k8s/Dockerfile
- dockers/agent/core/faiss/Dockerfile
- dockers/index/job/deletion/Dockerfile
- dockers/index/job/creation/Dockerfile
- apis/swagger/v1/mirror/mirror.swagger.json
- .github/workflows/test-hack.yaml
- .github/workflows/reviewdog-k8s.yaml
- dockers/gateway/lb/Dockerfile
- .github/workflows/e2e-max-dim.yaml
- k8s/manager/index/deployment.yaml
- pkg/index/job/deletion/config/config_test.go
- dockers/operator/helm/Dockerfile
- .github/workflows/coverage.yaml
- Makefile.d/e2e.mk
- internal/net/grpc/pool/pool_bench_test.go
- internal/errors/errors_test.go
- k8s/index/operator/deployment.yaml
- internal/core/algorithm/usearch/usearch_test.go
- internal/net/http/client/client.go
- pkg/gateway/mirror/service/mirror.go
- internal/net/dialer_test.go
- .github/ISSUE_TEMPLATE/bug_report.md
- dockers/agent/core/ngt/Dockerfile
- internal/net/http/client/client_test.go
- .github/workflows/format.yaml
- cmd/index/job/deletion/main_test.go
- Makefile.d/tools.mk
- pkg/agent/core/ngt/service/option_test.go
- internal/runner/runner.go
- .github/ISSUE_TEMPLATE/security_issue_report.md
- internal/net/grpc/pool/option_test.go
- apis/swagger/v1/vald/object.swagger.json
- internal/net/grpc/status/status_test.go
- internal/timeutil/time.go
- internal/config/index_deleter_test.go
- internal/config/grpc.go
- internal/client/v1/client/discoverer/discover.go
- Makefile.d/functions.mk
- pkg/agent/core/ngt/service/ngt_test.go
- internal/net/net_test.go
- dockers/ci/base/Dockerfile
- internal/db/nosql/cassandra/option_test.go
- internal/servers/server/server.go
- internal/test/data/vector/noise/option.go
- apis/swagger/v1/vald/index.swagger.json
- internal/client/v1/client/discoverer/discover_test.go
- internal/servers/servers.go
- internal/net/http/client/option_test.go
- pkg/agent/core/faiss/service/faiss.go
- .github/PULL_REQUEST_TEMPLATE.md
- internal/file/file_test.go
- Makefile.d/minikube.mk
- internal/info/info_test.go
- apis/swagger/v1/vald/flush.swagger.json
- k8s/gateway/gateway/lb/deployment.yaml
- pkg/discoverer/k8s/service/discover_test.go
- k8s/gateway/gateway/mirror/deployment.yaml
- internal/params/params_test.go
- internal/test/mock/server_stream.go
- internal/net/grpc/metadata_test.go
- pkg/agent/core/ngt/service/ngt.go
- pkg/gateway/lb/handler/grpc/handler_test.go
- Makefile
- internal/sync/errgroup/group.go
- internal/net/grpc/pool/option.go
- internal/config/config.go
- hack/go.mod.default
- apis/swagger/v1/vald/update.swagger.json
- example/client/go.mod
- internal/net/grpc/client.go
🧰 Additional context used
🧠 Learnings (1)
.gitfiles (1)
Learnt from: datelier
PR: vdaas/vald#2807
File: rust/bin/agent/src/handler/object.rs:36-38
Timestamp: 2025-04-01T12:29:37.305Z
Learning: In Rust code for Vald handlers, avoid using unwrap() on Option types. Instead, handle None values explicitly by returning appropriate error statuses to maintain robustness.
🧬 Code Definitions (2)
internal/net/grpc/stream.go (1)
internal/net/grpc/status/status.go (5)
Is(48-50)Errorf(129-131)ParseError(133-183)Code(41-41)Error(125-127)
internal/net/grpc/pool/pool.go (4)
internal/net/grpc/client.go (2)
ClientConn(50-50)DialOption(49-49)internal/net/grpc/status/status.go (1)
Is(48-50)internal/net/grpc/codes/codes.go (2)
Canceled(26-26)DeadlineExceeded(29-29)internal/net/grpc/option.go (1)
Option(41-41)
🔇 Additional comments (120)
internal/params/option_test.go (1)
451-705: Implement or remove commented test functionsThese commented test functions (TestWithName, TestWithOverrideDefault, and TestWithArgumentFilters) remain as templates without actual test cases. Consider either implementing them with real test cases or removing them to avoid code clutter. Having proper tests for these functions would improve code coverage and ensure their correct behavior.
apis/swagger/v1/vald/upsert.swagger.json (2)
17-18: Enhanced Documentation for/upsertEndpoint
The updated"summary"and"description"now provide comprehensive details including a clearly formatted table of status codes and troubleshooting guidance. This enhanced clarity helps API consumers better understand error conditions and resolutions.
50-51: Improved Clarity for/upsert/multipleEndpoint
The modifications enrich the"summary"by explicitly stating the multi-vector upsert capability in one request and include a detailed"description"with warning about gRPC message size limitations and structured troubleshooting tips. Just ensure that the style and detail level remain consistent with the/upsertdocumentation..github/workflows/e2e.yaml (8)
45-47: Credential Injection for Container Image AccessThe introduction of the
credentialsblock in the container configuration for the "e2e-stream-crud" job is correctly implemented. Using${{ secrets.PACKAGE_USER }}and${{ secrets.PACKAGE_TOKEN }}ensures that the container pulls a private image with proper authentication.
89-91: Consistent Credential Addition in Operator JobThe credentials block for the "e2e-stream-crud-for-operator" job is applied consistently. It follows the same secure pattern used in the previous job, ensuring uniformity across workflows.
156-158: Credential Block Consistency for Index ManagementThe "e2e-stream-crud-under-index-management-jobs" job now includes a credentials section that aligns with the other job configurations. Please verify that the secret values (
PACKAGE_USERandPACKAGE_TOKEN) have the necessary permissions for this job’s operations.
204-206: Credential Section in Skip Exist Check JobThe addition of the
credentialsblock in the "e2e-stream-crud-skip-exist-check" job is properly executed. Ensure that these credentials are valid and that the associated permissions match the job’s requirements.
248-250: Credential Integration for Multi-APIs CRUDThe "e2e-multiapis-crud" job now also incorporates a credentials block, maintaining the secure, consistent approach for private container image access.
288-290: Credentials in Jobs SectionThe credentials have been correctly added to the "e2e-jobs" container configuration. It's important to confirm that the referenced secrets remain current and provide sufficient access for the job’s tasks.
325-327: Credential Application for Read Replica TestIn the "e2e-stream-crud-with-readreplica" job, the credentials block is properly utilized. Double-check that the permissions for these credentials are in line with the requirements of a read replica deployment.
379-381: Credential Block for Mirror DeploymentThe "e2e-stream-crud-with-mirror" job now includes a credentials block similar to other jobs. This uniform approach simplifies maintenance and ensures secure access for pulling the container image.
apis/swagger/v1/vald/filter.swagger.json (8)
17-17: Improved Summary for/insert/objectEndpoint.
The updated summary now provides a clear overview of the InsertObject RPC with a detailed status code table. Please verify that the newline characters (\n) and overall formatting are rendered as expected in your API documentation tool.
49-49: Updated Summary for/insert/object/multipleEndpoint.
The summary now clearly explains the MultiInsertObject RPC functionality and includes a detailed status code table. Please ensure the use of markdown syntax (e.g., 1) is supported by your documentation rendering environment.
81-81: Enhanced Summary for/search/objectEndpoint.
The revised summary succinctly describes the SearchObject RPC and provides a thorough status code list, which improves clarity for consumers of the API.
113-113: Detailed Summary for/search/object/multipleEndpoint.
The update adds comprehensive details about the bidirectional streaming RPC aspect along with the status code table. Please double-check that the HTML<br>tag and markdown formatting render correctly in your Swagger UI.
145-145: Clear Summary for/update/objectEndpoint.
The new summary concisely explains that the endpoint handles updating a single vector with detailed status codes. The changes enhance documentation clarity.
177-179: Enhanced Summary & Description for/update/object/multipleEndpoint.
The combination of a clear summary and an HTML-based description effectively alerts users to the gRPC message size limitation while outlining the operation. Please confirm that the HTML markup (e.g.,<div class="notice">) displays properly in your API docs.
210-210: Revised Summary for/upsert/objectEndpoint.
The updated summary accurately reflects the dual operation (update or insert) performed by this endpoint and includes a detailed status code table.
242-244: Updated Summary & Description for/upsert/object/multipleEndpoint.
The summary now clearly specifies that the endpoint handles both updating existing objects and adding new ones. The HTML-enhanced description warns about potential gRPC message size limitations. Verify that these HTML and markdown elements are consistently rendered alongside similar endpoints for a uniform documentation experience.k8s/agent/ngt/configmap.yaml (1)
257-257:❓ Verification inconclusive
Enable Statistics Collection
The change sets
enable_statisticstotrue, thereby enabling statistics collection. This update aligns with the intent to gather performance or operational data, but please double-check that enabling statistics is intended in your production environment, as it might introduce additional resource overhead. Additionally, ensure that related configurations (e.g., Helm charts mentioned in the PR objectives) are updated accordingly if they reference this flag.
Action Required: Verify Production Impact of Enabled Statistics
The configuration ink8s/agent/ngt/configmap.yamlnow setsenable_statistics: true. Please confirm that this flag is intentionally enabled for the production environment, given that it may introduce additional resource overhead. Additionally, ensure that any related configurations—such as the Helm charts referenced in the PR objectives—are updated accordingly to reflect this change.internal/net/grpc/server_test.go (2)
29-29: LGTM: Good practice to include gRPC proto encoding package.Including the gRPC proto encoding package as a blank import ensures protocol buffer encoding is registered with the gRPC framework, which is necessary for proper serialization/deserialization during tests.
836-1176: Action required: Implement E2E tests for gRPC server options.The commented test skeletons provide a good structure but remain unimplemented. This was previously flagged in a review and appears to still be pending implementation.
These tests are important for validating critical gRPC server configuration options:
MaxConcurrentStreams: Controls stream concurrency limitsNumStreamWorkers: Affects worker thread allocationSharedWriteBuffer: Impacts memory efficiencyWaitForHandlers: Affects server shutdown behavior#!/bin/bash # Check if these functions exist in the implementation and have relevant E2E tests # Check for the server option implementations echo "Checking server option implementations:" grep -n "func Max" internal/net/grpc/server.go grep -n "func NumStreamWorkers" internal/net/grpc/server.go grep -n "func SharedWriteBuffer" internal/net/grpc/server.go grep -n "func WaitForHandlers" internal/net/grpc/server.go # Check for E2E tests that might cover these functions echo -e "\nChecking for E2E tests that might cover these options:" find tests/v2/e2e -type f -name "*.go" | xargs grep -l "MaxConcurrentStreams\|NumStreamWorkers\|SharedWriteBuffer\|WaitForHandlers" || echo "No E2E tests found for these server options"internal/net/quic/listener_test.go (4)
16-207: Commented-out test code should be implemented or removedThe file contains commented-out test functions (
TestListenandTestListener_Accept) that are explicitly marked as "NOT IMPLEMENTED BELOW". While the test structure is well-designed with proper table-driven test patterns, helper functions, and goroutine leak checks, adding commented-out code to the repository is generally considered a bad practice.Since this PR focuses on enhancing the testing infrastructure, it would be more appropriate to either:
- Implement the tests properly before merging
- Remove the commented-out code and add a TODO comment or create an issue to track the implementation
- If there's a specific reason for keeping this skeleton, document it clearly
16-207: Consider incremental implementation approachIf fully implementing both test functions is not feasible for this PR, consider an incremental approach:
- Uncomment just one of the test functions (e.g.,
TestListen)- Implement 1-2 basic test cases for that function
- Add a TODO comment for
TestListener_Acceptwith an issue referenceThis would provide some test coverage while avoiding commented code in the repository.
45-87: Implement test cases for the Listen functionThe test structure is well-designed, but there are no actual test cases, only placeholder TODOs. Consider implementing at least basic test cases to verify:
- Successful listener creation with valid arguments
- Error handling with invalid addresses
- Context cancellation behavior
- TLS configuration handling
113-207: Implement test cases for Listener.Accept methodThe test structure for
TestListener_Acceptis well-defined, but currently commented out with no actual test cases implemented. Consider implementing tests that verify the Accept method handles connections correctly and propagates context cancellation.I recommend implementing at least these test scenarios:
- Successful connection acceptance
- Context cancellation during Accept
- Error handling when the underlying listener fails
internal/net/grpc/option_test.go (21)
532-615: Function name and parameter type mismatch with function call on line 609.The function name
TestWithBackoffMaxDelaysuggests testing theWithBackoffMaxDelayfunction. However, on line 609, the function under test is called with the same name (WithBackoffMaxDelay). This indicates proper alignment between the function name and its implementation.
617-700: Function name and parameter type are properly aligned.The test function
TestWithBackoffBaseDelayis correctly named to test theWithBackoffBaseDelayfunction and uses the string duration parameter appropriately.
702-785: Parameter type has been updated appropriately.The test function
TestWithBackoffMultipliernow usesfloat64parameter type which is more appropriate for a multiplier value than the previous string type.
787-870: Parameter type correctly changed to match function purpose.The change from using a multiplication factor to a jitter factor (both as
float64) inTestWithBackoffJitteraligns with the function's purpose.
872-955: Appropriate parameter type for timeout function.Using a string duration parameter for
TestWithMinConnectTimeoutis consistent with how time durations are typically represented in configuration.
957-1040: Function parameter type correctly specifies error group dependency.The parameter type change to
errgroup.GroupforTestWithErrGroupis appropriate as it matches the expected input type for the error group functionality.
1042-1125: Parameter type change aligns with backoff functionality.Replacing the previous type with
backoff.BackoffinTestWithBackoffis a good refactoring as it uses a specialized type for backoff strategies.
1127-1210: Parameter type properly represents circuit breaker dependency.The parameter type change to
circuitbreaker.CircuitBreakerforTestWithCircuitBreakermakes it clearer what type of object is expected.
1212-1295: Test function name matches its functionality.
TestWithCallOptionsnow properly tests the function that handles call options with the appropriate parameter type.
1297-1380: Parameter type appropriately represents content subtype.The string parameter for
TestWithCallContentSubtypecorrectly represents how content subtypes are specified in gRPC communication.
1382-1465: Parameter type is appropriate for message size configuration.Using an integer type for
TestWithMaxRecvMsgSizeproperly represents how message size limits are configured.
1467-1550: Function name and parameter type are consistent with functionality.
TestWithMaxSendMsgSizecorrectly maintains the integer parameter type which is appropriate for size configurations.
1552-1635: Parameter type is retained appropriately.The integer parameter type for
TestWithMaxRetryRPCBufferSizeis maintained, which is correct for buffer size configurations.
1637-1720: Boolean parameter type is suitable for flag configuration.The boolean parameter for
TestWithWaitForReadyappropriately represents a flag that can be enabled or disabled.
1722-1805: Function name change with appropriate parameter type.The function has been renamed from a previous test to
TestWithDialOptionsand correctly uses a slice of dial options as its parameter type.
2408-2494: Added network parameter to dialer function.The
TestWithDialerfunction now includes both network string and dialer parameters, which better aligns with how network connections are typically configured in Go.
1977-2060: Parameter type change from int to int32.The parameter type for
WithInitialWindowSizehas been changed frominttoint32, which is more specific and likely better reflects the underlying gRPC API requirements.
2062-2145: Consistent parameter type change for window size.Similar to the initial window size, the parameter type for
WithInitialConnectionWindowSizehas been changed frominttoint32for consistency.
2581-3175: New test functions added align with gRPC connection configuration.The newly added test functions like
TestWithAuthority,TestWithDisableRetry,TestWithIdleTimeout, etc., expand the test coverage for various gRPC connection parameters, which enhances the robustness of the testing framework.
3261-3344: Function name change reflects accurate naming.Renaming from
TestWithOldConnCloseDurationtoTestWithOldConnCloseDelaymakes the function name more precise about what it's testing (delay rather than duration).
532-3344: Overall restructuring improves test organization and clarity.The extensive renaming and parameter type changes throughout the file create a more consistent and maintainable testing framework. This aligns well with the PR objective of enhancing E2E testing for better maintainability.
Each test function now more accurately reflects what it's testing, with parameter types that better match the expected input types. The changes are consistent throughout the file, indicating a well-planned refactoring effort.
.github/workflows/codeql-analysis.yml (6)
53-57: Permissions Restriction Update
The updated permissions block refines the security context by explicitly granting only the required permissions (i.e.security-events: write,packages: read,actions: read,contents: read). This adheres to the principle of least privilege for GitHub Actions.
58-67: Matrix Strategy Configuration
The strategy configuration effectively uses a matrix to handle different languages with specified build modes and afail-fast: falsepolicy. One observation: theruns-onconditional in this job checks forswifteven though the matrix only includesactions,go, andrust. Verify whether support for Swift should be added to the matrix or if the conditional should be simplified.
70-72: Container Credentials Update
The addition of container credentials for accessing the CI container image via${{ secrets.PACKAGE_USER }}and${{ secrets.PACKAGE_TOKEN }}strengthens the workflow’s security posture. Please ensure that these secrets are correctly configured in the repository and that only minimal permissions are granted for this purpose.
84-87: Upgrade to CodeQL Initialization v3
Replacinggithub/codeql-action/init@v2with@v3and dynamically configuringlanguagesandbuild-modeenhances compatibility with the latest CodeQL features. It is advisable to update any related documentation to reflect these changes, ensuring that team members are aware of the new parameters.
90-91: Autobuild Step Update for Go
The conditional execution of theAutobuildstep usingif: matrix.language == 'go'is appropriate for triggering the build only when necessary. Upgrading togithub/codeql-action/autobuild@v3aligns this step with the rest of the workflow updates.
93-95: CodeQL Analysis Step Enhancement
The final analysis step is correctly updated to usegithub/codeql-action/analyze@v3with a dynamiccategoryparameter based on the language. This change ensures that the analysis is properly categorized and that the workflow remains consistent with the upgraded CodeQL actions.apis/swagger/v1/vald/insert.swagger.json (3)
17-17: Typographical Error in SummaryThe summary still contains the typo "Inset RPC" instead of "Insert RPC". Please update it to correctly reflect the endpoint's functionality.
18-18: Verify Description Content and FormattingThe description provides extensive troubleshooting guidance with a detailed table. Please double-check that the markdown table and newline formatting render correctly in your API documentation tool.
51-51: Validate HTML Markup in MultiInsert DescriptionThe description includes embedded HTML (e.g.,
<div class="notice">) for a notice. Verify that this markup renders as intended across all API documentation views.internal/file/file.go (2)
259-259: Streamlined error handlingThe change from
if err != nil && !errors.Is(err, io.EOF)toif errors.IsNot(err, io.EOF)improves code readability while maintaining the same logic. This is a good use of the custom error utility function.
564-564:✅ Verification successful
Join function now ensures absolute paths
The modified
Joinfunction now uses the newAbsolutePathfunction, which centralizes path resolution logic.This change may affect callers that previously expected relative paths in some cases. Let's verify existing usage:
🏁 Script executed:
#!/bin/bash # Find all calls to the Join function in the codebase rg -A 2 -B 2 "file\.Join\(" --no-headingLength of output: 21096
Absolute path integration verified across usage
The updated
Joinfunction now returns absolute paths via the centralizedAbsolutePathfunction, and our repository search confirms that all current call sites—including configuration paths (e.g., kubeconfig paths usingos.Getenv("HOME")and temporary directories viaos.TempDir())—build paths using components that are already absolute. There’s no evidence of any caller relying on a relative path outcome.
- Verified usage in tests (e.g., in
tests/e2e/*files) and production code (e.g., ininternal/servers/server/server.goand various modules underpkg/agent) already operates on absolute path inputs.- No additional refactoring is needed since no consumer appears to require relative paths.
pkg/index/job/deletion/service/options_test.go (4)
16-101: Complete implementation for TestWithDiscoverer functionThe file contains commented-out test functions without actual implementation. This test framework structure is good, but needs to be uncommented and completed with actual test cases.
This is a new test file with a commented-out structure for testing the
WithDiscovererfunction. It follows the project's established test pattern but needs concrete test cases.Since this is part of a new E2E testing framework, implementing actual test cases is important for ensuring reliability. Consider adding at least:
- A test case with a non-nil discoverer client
- A test case with a nil discoverer client to verify error handling
Also, the file is missing the necessary imports for the test functions to work properly.
+ import ( + "reflect" + "testing" + + "github.com/vdaas/vald/internal/errors" + "github.com/vdaas/vald/internal/test/goleak" + "github.com/vdaas/vald/apis/grpc/v1/agent" + "github.com/vdaas/vald/internal/net/grpc/client/discoverer" + )
103-186: Implement the TestWithIndexingConcurrency functionThe test structure is in place, but the implementation is commented out and test cases aren't defined.
No real test cases in
TestWithIndexingConcurrency.
Implement test coverage for:
- Positive concurrency value.
- Zero concurrency.
- Negative or invalid values.
188-271: Complete test cases for TestWithTargetAddrsThe structure is ready but needs implementation of concrete test cases.
Framework for TestWithTargetAddrs implementation
Another template for testing the
WithTargetAddrsfunction without actual test cases implemented.Consider adding test cases for:
- Multiple valid addresses
- Empty address list
- Addresses with different formats
This will ensure that target address handling is robust in the E2E testing framework.
273-356: Complete test cases for TestWithTargetIndexIDThe test function needs to be uncommented and filled with concrete test cases.
Framework for TestWithTargetIndexID implementation
A final template for testing the
WithTargetIndexIDfunction without concrete test cases.Consider implementing test cases for:
- A valid index ID
- An empty index ID
- Special character handling in index IDs (if applicable)
This would complete the testing coverage for the service options in this new package.
apis/swagger/v1/vald/search.swagger.json (3)
17-18: Enhanced Documentation for/linearsearchEndpoint
The updatedsummaryanddescriptionnow offer a comprehensive overview including status codes and troubleshooting guidance. This change improves clarity for API consumers. Please verify that the markdown and table formatting render correctly in your Swagger UI.
83-84: Enhanced Multi-ID Search Documentation for/linearsearch/id/multiple
The new fields provide extensive details, including a highlighted notice regarding gRPC message size limitations. Please confirm that the embedded HTML<div class="notice">block is supported in your Swagger UI and that the detailed error resolution table is maintainable.
215-216: Comprehensive Update for/search/id/multipleEndpoint
The new documentation details effectively communicate multi-ID search behavior and gRPC limitations using an HTML notice. Verify that the added HTML styling does not conflict with other style elements in your Swagger UI.apis/docs/v1/docs.md (5)
1370-1373: Mirror Service Overview Update:
The new “Overview” section clearly states that the Mirror service provides theRegisterinterface for the Vald Mirror Gateway. Please verify that this description is consistent with the actual service behavior and that the terminology matches throughout the documentation.
1676-1677: Flush Service Overview Update:
The overview now specifies that the Flush service is responsible for removing all vectors (both indexed and uncommitted) in thevald-agent. This succinct description is helpful; please double-check that it fully reflects how the service behaves in practice.
1688-1691: Index Service Overview Update:
The concise “Overview” for the Index service now identifies it as the index manager service. Confirm that this summary aligns with the broader details provided elsewhere in the documentation and that it reflects any recent functional changes.
1714-1715: Insert Service Overview Update:
The newly added overview for the Insert service explains its role in inserting new vectors into thevald-agent. This addition improves clarity. Please verify that the overview is consistent with the operational semantics described in the accompanying sections.
1940-1941: Upsert Service Overview Update:
The overview for the Upsert service now clearly states that it is responsible for either updating existing vectors in thevald-agentor inserting new ones if they do not yet exist. This concise summary is a great addition; please ensure that it aligns with the actual behavior as implemented.internal/net/grpc/errdetails/errdetails.go (7)
80-82: Good addition of a structured container for multiple details.The new
Detailsstruct provides a clean way to encapsulate multiple error details, which improves serialization consistency when handling collections of errors.
93-94: Improved type handling with cleaner prefix trimming.The refactored type URL handling simplifies the code by using a single variable for the trimmed type name, making the switch statement more readable.
165-167: Enhanced flexibility with fallback marshaling.Adding a default case to handle unknown message types using
protojson.Marshalis a good improvement that makes the error details system more extensible and robust.
168-171: Better error handling with descriptive logs.The improved error checking now validates both error state and nil byte slices, with detailed logging that includes the type name, which will be valuable for debugging.
315-318: Simplified JSON serialization using the new Details struct.The
Serializefunction now properly handles multiple messages by using the newDetailsstruct, providing a more consistent JSON representation of error details.
319-322: Added nil check and better byte-to-string conversion.Using the
conv.Btoautility function and adding a nil check for the byte slice improves reliability and follows best practices for handling byte conversions.
416-421: Flexible handling of unknown message types.The new default case that attempts to unmarshal unknown types makes the function more robust and adaptable to different message formats, enhancing the error handling capabilities.
internal/net/grpc/stream_test.go (2)
206-209: Updated structure for bidirectional streaming arguments.The update to the
argsstruct in the commented-out test function shows a shift towards a more modular approach with separate parameters for concurrency, data providing, and callbacks. This is a good design improvement.
290-290: Function call signature updated correctly.The function call to
BidirectionalStreamClienthas been correctly updated to match the new parameter structure defined in theargsstruct.internal/net/grpc/pool/pool.go (4)
17-20: Improved package documentation.The updated package documentation clearly describes the new lock-free implementation approach and highlights the key features, which is helpful for understanding the design choices.
79-121: Well-implemented graceful connection closure with timeout.The new
Closemethod forpoolConnimplements a robust approach to connection closure, handling different connection states and ensuring connections are eventually closed.
123-158: Excellent lock-free implementation using atomic operations.The redesigned
poolstruct using atomic pointers and operations is an excellent approach for achieving thread-safety without explicit locks. This should improve performance in high-concurrency scenarios.
725-738: Good addition of metrics collection.The addition of the
Metricsfunction provides valuable insights into connection health, which will be useful for monitoring and debugging in production environments.internal/net/quic/conn_test.go (3)
36-37: Missing import for errors package with custom functionsThe code uses
errors.Is()anderrors.Errorf()but doesn't import the appropriate errors package. While Go's standard library haserrors.Is(), it doesn't haveerrors.Errorf(), indicating you're using a custom errors package.Check which errors package is being used elsewhere in the codebase:
#!/bin/bash # Find out which errors package is used for Errorf in the codebase rg -A 1 "errors\.Errorf" --type go | head -n 10
90-91: Add missing import for goleak packageThe test uses
goleak.VerifyNone()to check for goroutine leaks, but the import forgo.uber.org/goleakis missing. This needs to be added when uncommenting the code.Verify the correct usage of goleak in the project:
#!/bin/bash # Check if goleak is already imported elsewhere in the project rg --type go "import.*goleak" --stats
304-304:✅ Verification successful
Using generic sync.Map but missing Go 1.18+ requirement
The code uses generics with
sync.Map[string, quic.Connection], indicating it requires Go 1.18 or later. Ensure your project's minimum Go version is set appropriately.Check the Go version used in the project:
🏁 Script executed:
#!/bin/bash # Check Go version in go.mod cat go.mod | grep -E "^go [0-9]+\.[0-9]+" # Check if other generic types are used in the codebase rg --type go "sync\.Map\[.*\]" --statsLength of output: 25858
Go version requirement confirmed; no changes required
The project’sgo.modspecifiesgo 1.24.2, which fully supports generics (includingsync.Map[string, quic.Connection]). The use of genericsync.Mapis consistent across the codebase, so no modifications are necessary.internal/params/option.go (2)
38-45: LGTM! Function implementation is clean and concise.The implementation correctly checks for empty name values before setting the parser name.
96-109: LGTM! Well-implemented filtering mechanism.Good implementation with appropriate nil checks and clear documentation explaining the filter behavior.
internal/params/params.go (7)
29-35: LGTM! Good type alias and constant definitions.Clean declaration of the ErrorHandling type alias and related constants that follow Go's best practices.
53-70: LGTM! Well-structured parser type with clear field organization.The additions to the parser struct enhance its flexibility and configurability.
78-82: LGTM! Good initialization with proper flag set creation.The initialization logic correctly sets up the parser with appropriate defaults and handles the override case.
88-90: LGTM! Good defensive programming with nil checks.Adding explicit nil checks prevents potential nil pointer dereferences and returns a meaningful error.
108-119: LGTM! Well-implemented argument filtering logic.The filtering mechanism efficiently uses slices.DeleteFunc and properly handles nil filters.
120-126: LGTM! Improved error handling using errors.Is.Using errors.Is for checking flag.ErrHelp follows Go's error handling best practices.
139-149:❓ Verification inconclusive
Consider thread-safety implications of global flag.CommandLine manipulation.
While the implementation is clean, manipulating the global flag.CommandLine can lead to race conditions if multiple parts of the code are using Override() and Restore() concurrently.
Consider documenting this limitation or implementing a more thread-safe approach:
+// Restore resets the global flag.CommandLine to the original default. +// Note: This method is not thread-safe and should be used with caution in concurrent environments. func (p *parser) Restore() { if p.defaults != nil { flag.CommandLine = p.defaults } } +// Override replaces the global flag.CommandLine with this parser's flag set. +// Note: This method is not thread-safe and should be used with caution in concurrent environments. func (p *parser) Override() { if p.f != nil { flag.CommandLine = p.f } }Let's verify if there's any concurrent usage of these methods in the codebase:
🏁 Script executed:
#!/bin/bash # Look for concurrent usages of Override or Restore # Check for goroutine usage in files that might call these methods rg -A 3 -B 3 "go func" --glob "**/*.go" | rg -i "override|restore" # Check for references to the parser type in goroutines rg -A 5 -B 5 "go func" --glob "**/*.go" | rg "parser"Length of output: 122
Action Required: Document Thread-Safety Concerns for Global flag.CommandLine Manipulation
Although our automated search for concurrent invocations (e.g., via goroutines) did not reveal any evidence of simultaneous calls to
Override()orRestore(), the inherent risks of modifying a global variable remain. Since global state manipulation can lead to race conditions if these methods are ever used in concurrent scenarios, it is important to explicitly document that these methods are not thread-safe.Please update the comments in
internal/params/params.go(lines 139-149) as follows:+// Restore resets the global flag.CommandLine to the original default. // Note: This method is not thread-safe. Ensure exclusive access when calling Restore in concurrent contexts. func (p *parser) Restore() { if p.defaults != nil { flag.CommandLine = p.defaults } } +// Override replaces the global flag.CommandLine with this parser's flag set. // Note: This method is not thread-safe. Use with caution in concurrent environments. func (p *parser) Override() { if p.f != nil { flag.CommandLine = p.f } }Given that our automated searches did not detect concurrent usage but the potential for race conditions remains, please manually review usage scenarios for these methods to ensure safe operation.
go.mod (2)
3-3: Go Version Bump Verification
The Go version has been updated from 1.24.0 to 1.24.2. Please ensure that all internal tools, CI pipelines, and dependency builds remain compatible with this update.
352-424: Direct Dependency Updates in the First Require Block
Within the firstrequire (block (lines 352–424), several direct dependency versions have been updated (for example, updates tobuf.build/gen/go/bufbuild/protovalidate,cloud.google.com/go/storage, and others).
Please confirm that:
- The new versions are compatible with your project's API usage.
- Any changes in semantic versioning have been reviewed for potential breaking changes.
Running integration tests after these updates is advisable.internal/iter/iter.go (2)
17-23: Clarify or remove the"iter"import to avoid potential circular dependencies.
Since this file declarespackage iterwhile also importing"iter", there is a risk of self-import if"iter"is not an external library. If this import references the same package, please remove it or specify a different import path.
53-59: Return a non-nil "no-op" cycle for empty or nil array.
Returningnilhere forces callers to check fornilbefore every usage, which can lead to unexpected panics. Provide a valid but no-op cycle to make the API safer..gitfiles (1)
126-126:✅ Verification successful
Appreciation for newly introduced files and documentation.
All these added entries (workflows, docs, samples, tests, etc.) appear to improve coverage, clarity, and maintainability. There are no apparent issues.
Please confirm that the newK3D_VERSIONis correct and consistent with your environment.Also applies to: 161-177, 488-488, 512-512, 863-863, 1051-1051, 1088-1088, 1274-1274, 1276-1276, 1966-1966, 1976-1976, 1986-1990, 2035-2056, 2067-2067
🏁 Script executed:
#!/bin/bash # Verify if K3D_VERSION is set to the intended version and cross-check if it meets project prerequisites grep -H 'K3D_VERSION' versions/K3D_VERSION cat versions/K3D_VERSIONLength of output: 75
K3D_VERSION confirmed as 5.8.3 – Code Enhancements Approved
The verification output shows that the
K3D_VERSIONinversions/K3D_VERSIONis correctly set to 5.8.3, which meets the expected project prerequisites. The additional files and documentation continue to enhance coverage, clarity, and maintainability.
- Confirmed:
K3D_VERSIONis set to 5.8.3.- All new files and documentation have no apparent issues.
internal/test/data/vector/noise/noise.go (3)
66-71: Ensure user-provided configuration is validated
This portion, as noted in previous comments, lacks explicit checks for invalid user-provided values. IfminNoiseTableSizeornoiseTableDivisionFactorare zero or negative, subsequent operations (like power-of-two rounding) may break.
80-106: Possible division-by-zero in noise level calculation
A zero-length vector or multiple zero-length vectors can result incountremaining 0, causing a division-by-zero at line 105.
111-123: Double-check the power-of-two calculation
As previously noted, subtracting 1 fromrequiredcould cause an underflow ifrequiredis already 0. Ensurerequiredis strictly positive before callingbits.Len64(required-1).internal/net/grpc/stream.go (7)
44-53: Type safety improvements with new interfaces.The addition of
TypedClientStreamandTypedServerStreaminterfaces provides strong typing for gRPC streams, enforcing that messages implement theproto.Messageinterface. This is a good improvement that will catch type errors at compile time rather than runtime.
106-116: Improved error handling in the stream receive logic.The stream reception code has been enhanced to directly use the
Recv()method instead ofRecvMsg, which aligns with the new interface definition. The error handling is now more robust by checking specific error types.
117-167: Enhanced concurrency and error handling in stream processing.The stream processing logic has been significantly improved with:
- Better tracing with span recording
- More specific error code checking
- Proper error propagation with detailed error messages
- More robust synchronization with mutex protection
This implementation will provide better diagnostics and reliability.
178-180: Added concurrency control for client streams.The addition of concurrency limiting for client streams is a good improvement that will help prevent resource exhaustion when processing high volumes of messages.
187-191: Improved error handling with new IsNot function.The use of
errors.IsNot()provides cleaner, more readable error handling by checking if an error is not one of the expected types. This is more maintainable than the previous approach.
205-230: Robust finalization with improved error aggregation.The new
finalizefunction provides comprehensive error handling during stream closure:
- Properly aggregates errors from multiple goroutines
- Correctly handles standard error types like EOF and context cancellation
- Ensures proper stream closure regardless of error conditions
- Uses a mutex to prevent race conditions
This implementation is much more robust and maintainable.
238-253: Enhanced data validation and concurrent sending.The code now properly validates that the data provider returns valid data before sending and uses goroutines to handle sending operations concurrently. The error propagation through a channel provides better synchronization and error handling.
internal/config/config_test.go (1)
999-1003:✅ Verification successful
Test case updated to verify empty return when file doesn't exist.
The test case has been renamed from "return file contents when val is file://env" to "return empty when not exists file contents", and the expected output changed to an empty struct. This change suggests that the behavior of
GetActualValuehas been modified to return an empty value when the specified file doesn't exist, rather than returning the file path.
🏁 Script executed:
#!/bin/bash # Check how GetActualValue handles non-existent files in the implementation rg -A 10 "func GetActualValue" --glob 'internal/config/*.go' | grep -v "_test.go"Length of output: 1447
Confirmed behavior: GetActualValue now returns an empty value for non-existent files
The updated test case—renamed to "return empty when not exists file contents" and expecting an empty struct—is consistent with the current implementation in
internal/config/config.go. The function checks for a file prefix, attempts to read the file, and, if an error occurs or the file content is nil, returns an empty value. No additional changes are necessary.internal/errors/errors.go (3)
86-88: Added utility for unimplemented feature errors.The new
ErrUnimplementedfunction provides a standardized way to generate errors for unimplemented features. This improves consistency in error messaging and makes it easier to identify missing functionality.
170-180: New IsAny function for checking against multiple errors.The
IsAnyfunction adds the ability to check if an error matches any of multiple target errors, which is a common pattern in error handling. This reduces code duplication and improves readability when dealing with multiple potential error types.
182-192: New IsNot function for negative error matching.The
IsNotfunction complementsIsAnyby checking if an error does not match any specified error types. This is particularly useful for filtering out expected errors like EOF or context cancellation while still handling unexpected errors.internal/test/data/vector/noise/noise_test.go (2)
16-107: Implement tests for noise generator initialization.The test structure for
TestNewis well-designed but currently commented out. To ensure proper test coverage of the noise generator functionality, please implement test cases that verify:
- Normal initialization with valid data
- Behavior with empty data
- Different option combinations
This is particularly important since this appears to be part of a new E2E testing framework mentioned in the PR objectives.
109-207: Implement tests for noise modification functionality.The
Test_noiseGenerator_Modfunction provides a framework for testing the noise generator's modification capabilities, but currently has no implemented test cases. Consider implementing tests that verify:
- The returned
Funcproduces expected noise modifications- Different parameter combinations produce different but deterministic results
- Edge case handling (zero values, extreme values)
These tests would help ensure the reliability of the noise generation for the E2E testing framework.
pkg/gateway/lb/handler/grpc/aggregation_test.go (4)
36-40: Changes to the want struct properly support attribute testing.The addition of the
wantAttrs []attribute.KeyValuefield to thewantstruct enhances the test framework to verify attribute values, which is a good practice for comprehensive testing.
46-46: Properly updated function signature for checkFunc.The checkFunc signature has been updated to include the attribute values parameter, which properly aligns with the modifications to the want struct.
50-59: Effective implementation of attribute validation in defaultCheckFunc.The defaultCheckFunc now correctly validates the attribute values by comparing them against the expected values. The implementation follows good practices by keeping validation for different return values in separate conditional blocks.
157-158: Updated function call and response handling to include attributes.The test now correctly captures and validates the attributes returned from the aggregationSearch function, which enhances the test coverage.
| // Copyright (C) 2019-2025 vdaas.org vald team <[email protected]> | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // You may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // https://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| package quic | ||
|
|
||
| // NOT IMPLEMENTED BELOW | ||
| // | ||
| // func TestListen(t *testing.T) { | ||
| // type args struct { | ||
| // ctx context.Context | ||
| // addr string | ||
| // tcfg *tls.Config | ||
| // } | ||
| // type want struct { | ||
| // want net.Listener | ||
| // err error | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // args args | ||
| // want want | ||
| // checkFunc func(want, net.Listener, error) error | ||
| // beforeFunc func(*testing.T, args) | ||
| // afterFunc func(*testing.T, args) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got net.Listener, err error) error { | ||
| // if !errors.Is(err, w.err) { | ||
| // return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) | ||
| // } | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // args: args { | ||
| // ctx:nil, | ||
| // addr:"", | ||
| // tcfg:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // args: args { | ||
| // ctx:nil, | ||
| // addr:"", | ||
| // tcfg:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt, test.args) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt, test.args) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // | ||
| // got, err := Listen(test.args.ctx, test.args.addr, test.args.tcfg) | ||
| // if err := checkFunc(test.want, got, err); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } | ||
| // | ||
| // func TestListener_Accept(t *testing.T) { | ||
| // type fields struct { | ||
| // Listener quic.Listener | ||
| // ctx context.Context | ||
| // } | ||
| // type want struct { | ||
| // want net.Conn | ||
| // err error | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // fields fields | ||
| // want want | ||
| // checkFunc func(want, net.Conn, error) error | ||
| // beforeFunc func(*testing.T) | ||
| // afterFunc func(*testing.T) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got net.Conn, err error) error { | ||
| // if !errors.Is(err, w.err) { | ||
| // return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err) | ||
| // } | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // fields: fields { | ||
| // Listener:nil, | ||
| // ctx:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // fields: fields { | ||
| // Listener:nil, | ||
| // ctx:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T,) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // l := &Listener{ | ||
| // Listener: test.fields.Listener, | ||
| // ctx: test.fields.ctx, | ||
| // } | ||
| // | ||
| // got, err := l.Accept() | ||
| // if err := checkFunc(test.want, got, err); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
Missing import statements for necessary packages
The file is missing import statements for required packages:
contexterrorsreflecttlsnetquicgoleak(for goroutine leak detection)
These imports would be necessary if the tests were uncommented.
| // func Test_group_TryGo(t *testing.T) { | ||
| // type args struct { | ||
| // f func() error | ||
| // } | ||
| // type fields struct { | ||
| // egctx context.Context | ||
| // cancel context.CancelCauseFunc | ||
| // limit atomic.Int64 | ||
| // sem *semaphore.Weighted | ||
| // emap map[string]struct{} | ||
| // errs []error | ||
| // } | ||
| // type want struct { | ||
| // want bool | ||
| // } | ||
| // type test struct { | ||
| // name string | ||
| // args args | ||
| // fields fields | ||
| // want want | ||
| // checkFunc func(want, bool) error | ||
| // beforeFunc func(*testing.T, args) | ||
| // afterFunc func(*testing.T, args) | ||
| // } | ||
| // defaultCheckFunc := func(w want, got bool) error { | ||
| // if !reflect.DeepEqual(got, w.want) { | ||
| // return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want) | ||
| // } | ||
| // return nil | ||
| // } | ||
| // tests := []test{ | ||
| // // TODO test cases | ||
| // /* | ||
| // { | ||
| // name: "test_case_1", | ||
| // args: args { | ||
| // f:nil, | ||
| // }, | ||
| // fields: fields { | ||
| // egctx:nil, | ||
| // cancel:nil, | ||
| // limit:nil, | ||
| // sem:nil, | ||
| // emap:nil, | ||
| // errs:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // }, | ||
| // */ | ||
| // | ||
| // // TODO test cases | ||
| // /* | ||
| // func() test { | ||
| // return test { | ||
| // name: "test_case_2", | ||
| // args: args { | ||
| // f:nil, | ||
| // }, | ||
| // fields: fields { | ||
| // egctx:nil, | ||
| // cancel:nil, | ||
| // limit:nil, | ||
| // sem:nil, | ||
| // emap:nil, | ||
| // errs:nil, | ||
| // }, | ||
| // want: want{}, | ||
| // checkFunc: defaultCheckFunc, | ||
| // beforeFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // afterFunc: func(t *testing.T, args args) { | ||
| // t.Helper() | ||
| // }, | ||
| // } | ||
| // }(), | ||
| // */ | ||
| // } | ||
| // | ||
| // for _, tc := range tests { | ||
| // test := tc | ||
| // t.Run(test.name, func(tt *testing.T) { | ||
| // tt.Parallel() | ||
| // defer goleak.VerifyNone(tt, goleak.IgnoreCurrent()) | ||
| // if test.beforeFunc != nil { | ||
| // test.beforeFunc(tt, test.args) | ||
| // } | ||
| // if test.afterFunc != nil { | ||
| // defer test.afterFunc(tt, test.args) | ||
| // } | ||
| // checkFunc := test.checkFunc | ||
| // if test.checkFunc == nil { | ||
| // checkFunc = defaultCheckFunc | ||
| // } | ||
| // g := &group{ | ||
| // egctx: test.fields.egctx, | ||
| // cancel: test.fields.cancel, | ||
| // limit: test.fields.limit, | ||
| // sem: test.fields.sem, | ||
| // emap: test.fields.emap, | ||
| // errs: test.fields.errs, | ||
| // } | ||
| // | ||
| // got := g.TryGo(test.args.f) | ||
| // if err := checkFunc(test.want, got); err != nil { | ||
| // tt.Errorf("error = %v", err) | ||
| // } | ||
| // }) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Reintroduce or remove commented-out test
The Test_group_TryGo function is commented out, resulting in potential reduction to test coverage. Consider restoring or removing it to maintain clarity in the codebase.
There was a problem hiding this comment.
Actionable comments posted: 7
🔭 Outside diff range comments (3)
pkg/index/job/deletion/service/options_test.go (1)
1-357: 💡 Verification agent🧩 Analysis chain
Uncomment and implement the tests before merging
Currently, this file contains only commented-out test templates without any active code. As part of the E2E testing framework improvements, these tests should be uncommented and implemented before merging to ensure proper test coverage.
- // NOT IMPLEMENTED BELOW - // - // func TestWithDiscoverer(t *testing.T) { + func TestWithDiscoverer(t *testing.T) {Consider using table-driven test cases as structured in the templates, but with actual values and assertions that test both normal scenarios and edge cases.
🏁 Script executed:
#!/bin/bash # Check if these options are already implemented in the source file echo "Checking for option implementations in the source file:" fd "WithDiscoverer|WithIndexingConcurrency|WithTargetAddrs|WithTargetIndexID" "pkg/index/job/deletion/service/"Length of output: 232
Action Required: Uncomment and Implement Test Cases
The file
pkg/index/job/deletion/service/options_test.gocurrently contains only commented-out test templates. Our recent check confirms that no active tests (e.g., forWithDiscoverer,WithIndexingConcurrency,WithTargetAddrs, orWithTargetIndexID) are present. For robust E2E testing and proper code coverage, please:
- Uncomment the test functions (e.g., change
// func TestWithDiscoverer(t *testing.T) {tofunc TestWithDiscoverer(t *testing.T) {).- Implement the table-driven test cases with concrete values and assertions to cover both normal scenarios and edge cases.
- // NOT IMPLEMENTED BELOW - // - // func TestWithDiscoverer(t *testing.T) { + func TestWithDiscoverer(t *testing.T) {Once updated, these tests will help ensure that the options behave as expected before merging.
internal/net/grpc/option_test.go (2)
532-799:⚠️ Potential issueInconsistencies between test function names and tested functions
The renamed test functions (e.g.,
TestWithBackoffMaxDelay) don't match the actual functions they're testing (e.g.,WithDialOptionsat line 1799). This mismatch creates confusion about what's being tested. Each test function should test the function indicated by its name.For example, the test
TestWithBackoffMaxDelayshould callWithBackoffMaxDelay, but instead it callsWithDialOptions. This same pattern of mismatch occurs throughout the file.// Example fix for one case: func TestWithBackoffMaxDelay(t *testing.T) { // ... - got := WithDialOptions(test.args.opts...) + got := WithBackoffMaxDelay(test.args.dur) // ... }
20-3344: 🛠️ Refactor suggestionImplement or remove commented test code
This file contains a substantial amount of commented-out test code. While test templates are useful during development, they should either be fully implemented or removed before merging to maintain code clarity.
Consider either:
- Implementing the tests with actual test cases
- Removing the commented-out code and creating a separate ticket to track the need for these tests
- Moving the templates to a separate non-production file until they're ready to be implemented
This will improve code readability and reduce maintenance overhead.
♻️ Duplicate comments (14)
internal/net/grpc/pool/pool.go (1)
550-553:⚠️ Potential issueFix potential double-close in error handling.
If the backoff operation fails and the connection is not nil, the code attempts to close the connection twice: once with
_ = conn.Close()and then withconn.Close()in theerrors.Join. This could lead to unexpected behavior.- if err != nil && conn != nil { - _ = conn.Close() - return nil, errors.Join(err, conn.Close()) - } + if err != nil && conn != nil { + closeErr := conn.Close() + return nil, errors.Join(err, closeErr) + }internal/net/quic/conn_test.go (2)
44-83: 🛠️ Refactor suggestionCreate actual test cases instead of TODO placeholders
The file contains two template styles for creating test cases but no actual implementations. When implementing these tests, create meaningful test cases instead of keeping these commented examples.
For
TestNewConn, implement at least:
- A test case for successful connection creation
- A test case for handling nil connection input
- A test case for handling context cancellation
Example:
tests := []test{ { name: "successful_connection", args: args{ ctx: context.Background(), conn: &mockQuicConnection{}, }, want: want{ want: &Conn{ Connection: &mockQuicConnection{}, Stream: nil, // To be set in the implementation }, err: nil, }, checkFunc: defaultCheckFunc, }, { name: "nil_connection", args: args{ ctx: context.Background(), conn: nil, }, want: want{ want: nil, err: errors.New("connection is nil"), }, checkFunc: defaultCheckFunc, }, }
471-476: 🛠️ Refactor suggestionImplement mock QUIC connections for testing
For effective testing, you'll need mock implementations of QUIC interfaces.
Create mock implementations like:
type mockQuicConnection struct { closeFunc func() error openStreamFunc func(context.Context) (quic.Stream, error) // Add other methods needed } func (m *mockQuicConnection) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } func (m *mockQuicConnection) OpenStream(ctx context.Context) (quic.Stream, error) { if m.openStreamFunc != nil { return m.openStreamFunc(ctx) } return &mockQuicStream{}, nil } // Implement all required methods from the quic.Connection interface type mockQuicStream struct { readFunc func([]byte) (int, error) writeFunc func([]byte) (int, error) closeFunc func() error // Add other methods needed } // Implement all required methods from the quic.Stream interfacepkg/index/job/deletion/service/options_test.go (5)
1-15: 🛠️ Refactor suggestionMissing imports for test implementation
This test file is missing necessary imports that will be required when the tests are uncommented and implemented. As mentioned in previous review comments, you'll need imports for:
+ import ( + "reflect" + "testing" + + "github.com/vdaas/vald/internal/errors" + "github.com/vdaas/vald/internal/test/goleak" + "github.com/vdaas/vald/apis/grpc/v1/agent" + "github.com/vdaas/vald/internal/net/grpc/client/discoverer" + )
16-101: 🛠️ Refactor suggestionImplement test cases for
TestWithDiscovererThe test structure is well-defined but lacks concrete test cases. As noted in the previous review, consider implementing:
- A test case with a non-nil discoverer client
- A test case with a nil discoverer client to verify error handling
Uncomment the function and implement these test cases to ensure robust testing of the discoverer client handling in the E2E framework.
103-186: 🛠️ Refactor suggestionImplement test cases for
TestWithIndexingConcurrencyThe test framework is prepared but lacks actual test cases. To ensure proper handling of concurrency values, implement test cases for:
- Positive concurrency value (e.g., 10)
- Zero concurrency
- Negative or invalid values
This will verify that the concurrency handling is robust in the new E2E testing framework.
188-271: 🛠️ Refactor suggestionImplement test cases for
TestWithTargetAddrsThe test structure is prepared but needs concrete test cases. Consider implementing:
- Multiple valid addresses (e.g.,
[]string{"localhost:8080", "127.0.0.1:9090"})- Empty address list (e.g.,
[]string{})- Addresses with different formats (e.g.,
[]string{"localhost:8080", "example.com:9090"})These test cases will ensure robust address handling in the E2E testing framework.
273-356: 🛠️ Refactor suggestionImplement test cases for
TestWithTargetIndexIDThe test structure is defined but missing actual test cases. To ensure proper index ID handling, implement:
- A valid index ID (e.g.,
"test-index-001")- An empty index ID (e.g.,
"")- Special character handling in index IDs if applicable (e.g.,
"test_index@001")This would complete the testing coverage for the service options in this package.
internal/iter/iter.go (2)
52-68: 🛠️ Refactor suggestionReturn a no-op cycle instead of nil for empty arrays.
This code still returnsnilfornilor empty slices, which can lead tonil-pointer dereferences in callers. Providing a valid (though no-op) cycle would allow safe calls without extra checks, following a previously suggested best practice.Here is an example diff that returns a no-op cycle instead of
nil:func NewCycle[S ~[]E, E any](array S, num, offset uint64, mod func(uint64, E) E) Cycle[S, E] { if array == nil || len(array) == 0 { - return nil + return &cycle[S, E]{} } size := uint64(len(array)) ... }🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
17-23:⚠️ Potential issueRemove or correct the self-import to avoid circular dependencies.
Currently,import "iter"is suspicious because the package is also namediter. If the import is intended to reference the same directory, it creates a self-import loop. Either remove this import or replace it with the fully qualified path to the external package.Apply this diff to remove the self-import if unintentional:
package iter import ( "context" - "iter" )internal/params/option.go (1)
27-36:⚠️ Potential issueEnhance handling of os.Args[0] in package-level initialization.
Using
filepath.Base(os.Args[0])directly in package initialization is risky whenos.Argscould be empty or manipulated, especially in test environments.Consider implementing a safer approach:
-var defaultOptions = []Option{ - WithName(filepath.Base(os.Args[0])), - WithConfigFilePathKeys("f", "file", "c", "config"), +// getDefaultName returns a safe default name, handling empty os.Args cases +func getDefaultName() string { + if len(os.Args) > 0 { + return filepath.Base(os.Args[0]) + } + return "default-application" +} + +var defaultOptions = []Option{ + WithName(getDefaultName()), + WithConfigFilePathKeys("f", "file", "c", "config"),pkg/index/job/deletion/service/deleter_test.go (1)
1-14:⚠️ Potential issueMissing required imports
The file is missing several imports that will be needed when the tests are uncommented.
Add the following imports to the top of the file:
package service +import ( + "context" + "reflect" + + "github.com/vdaas/vald/internal/errors" + "github.com/vdaas/vald/internal/net/grpc" + "github.com/vdaas/vald/apis/grpc/v1/payload" + "github.com/vdaas/vald/apis/grpc/v1/vald" + "github.com/vdaas/vald/pkg/discoverer" + "go.uber.org/goleak" +)internal/test/data/vector/noise/noise.go (1)
115-123:⚠️ Potential issuePotential underflow in bits.Len64 if required is 0.
If
numis 0 andminNoiseTableSizealso remains 0, thenrequired - 1can lead to a negative value. This causes undefined behavior in the power-of-two calculation.Apply this patch to ensure
requiredis at least 1 before the shift:required := num * uint64(len(data[0])) required /= ng.noiseTableDivisionFactor if required < ng.minNoiseTableSize { required = ng.minNoiseTableSize } + if required < 1 { + required = 1 + } return 1 << bits.Len64(required-1)internal/test/data/vector/noise/option.go (1)
44-49:⚠️ Potential issueMissing validation for division by zero.
Since this factor is used as a divisor (based on the name and comments), accepting a value of 0 could lead to division by zero errors. Consider adding validation to ensure the value is greater than zero.
func WithTableDivisionFactor(f uint64) Option { + if f == 0 { + f = 1 + } return func(ng *noiseGenerator) { ng.noiseTableDivisionFactor = f } }
🧹 Nitpick comments (52)
k8s/agent/ngt/configmap.yaml (1)
257-257: Enableenable_statisticsFlagThis change sets
enable_statisticsto true, enabling statistical data collection for the NGT component. Please ensure that:
- The application and monitoring pipelines are prepared to handle the additional metrics.
- Relevant documentation (and Helm chart values, if applicable) is updated to reflect this behavioral change.
- Performance testing confirms that enabling statistics does not introduce undue overhead.
.github/workflows/codeql-analysis.yml (3)
46-73: New Analyze Job with Matrix Strategy
The newanalyzejob using a matrix configuration is well structured and leverages CodeQL v3 for multi-language analysis. However, note that theruns-onexpression checks for Swift (i.e.matrix.language == 'swift') even though the matrix currently includes only 'actions', 'go', and 'rust'. Please verify if Swift should be added to the matrix or if the conditional should be revised.
49-51: Environment Variables for Experimental Features
EnablingCODEQL_ENABLE_EXPERIMENTAL_FEATURESandCODEQL_ACTION_TEST_MODEis acceptable if intended for enhanced testing and safe experimentation. Consider adding inline comments to explain their purpose to future maintainers.
58-67: Matrix Configuration Consistency
The matrix configuration includes languages 'actions', 'go', and 'rust', yet theruns-onexpression accounts for Swift by conditionally selectingmacos-latest. Confirm if Swift support is intended. If not, update the condition to avoid confusion; if so, include Swift in the matrix.apis/swagger/v1/vald/filter.swagger.json (2)
177-178: Enhanced Summary and Warning for/update/object/multipleEndpointThe new summary for MultiUpdateObject provides an effective description, and the added description section clearly warns about gRPC message size limitations.
Nitpick: The phrase “size of the request exceed the limit” could be grammatically improved for consistency (e.g., change “exceed” to “exceeds”).
Consider applying the following diff:-Please be careful that the size of the request exceed the limit. +Please be careful that the size of the request exceeds the limit.
242-244: Enhanced Summary and Warning for/upsert/object/multipleEndpointThe updated summary and accompanying description for MultiUpsertObject now provide a detailed explanation, including a warning about gRPC message size limitations.
Suggestion: To maintain consistency in phrasing, consider revising “existing multiple objects” to “multiple existing objects” if that better matches the documentation style elsewhere.
apis/swagger/v1/vald/upsert.swagger.json (2)
18-18: Enhanced Description for Upsert Endpoint:
The new description introduces an extensive troubleshooting table covering common errors and their resolutions. This detailed guidance is very helpful for developers. Verify that the markdown table formatting (including pipe characters and dashes) is supported by your documentation tools, and consider a brief note if examples might further assist users.
51-51: Enhanced Description for MultiUpsert Endpoint:
The detailed description includes a prominent notice regarding gRPC message size limitations and a comprehensive troubleshooting table. Please ensure that the embedded HTML (e.g.,<div class="notice">) and markdown elements render correctly in your Swagger/OpenAPI documentation.apis/swagger/v1/vald/remove.swagger.json (2)
17-18: Enhanced Documentation for/removeEndpointThe updated summary and description now clearly outline the removal operation for a single vector, complete with a detailed status code table and troubleshooting guidelines. This provides much-needed clarity for API consumers. Please verify that the newline characters and markdown table formatting render correctly in your Swagger UI.
83-84: Detailed Summary for Timestamp-Based RemovalThe revised summary and description effectively detail how the timestamp-based removal works—highlighting that multiple timestamps form an
ANDcondition for deletions. This improvement aids in understanding the operation’s constraints and troubleshooting steps. Additionally, consider harmonizing the naming conventions across endpoints (e.g., "Remove RPC" vs. "RemoveByTimestamp RPC") for consistency throughout the API documentation. As with the other changes, please confirm that all HTML and markdown formatting renders as intended in the Swagger UI..github/workflows/e2e.yaml (8)
45-47: Credentials Block in "e2e-stream-crud" Job
The newly addedcredentialsblock uses the secretsPACKAGE_USERandPACKAGE_TOKENto authenticate the container pull. This implementation is correct and helps ensure that private image access is secured. Consider using YAML anchors or reusable workflow templates to reduce duplication if this credentials block is repeated in multiple jobs.
89-91: Credentials Block in "e2e-stream-crud-for-operator" Job
The credentials section is properly configured with secret references. As with similar blocks elsewhere, consider consolidating this configuration (e.g., via YAML anchors) to improve maintainability.
156-158: Credentials Block in "e2e-stream-crud-under-index-management-jobs" Job
The addition of the credentials here maintains consistency with the other container configurations. Ensure that the referenced secrets are securely managed and consider reusing this block definition to avoid repetition.
204-206: Credentials Block in "e2e-stream-crud-skip-exist-check" Job
The newly added credentials block correctly references the secrets for authentication. As these settings are replicated across several jobs, evaluate if a common configuration (using YAML anchors or a composite action) could simplify future updates.
248-250: Credentials Block in "e2e-multiapis-crud" Job
The integration of the credentials using secret keys is appropriate for securing the image fetch. For enhanced maintainability, consider centralizing this block to avoid repetitive configuration across different jobs.
288-290: Credentials Block in "e2e-jobs" Job
The credentials section is correctly added to the container configuration, ensuring that the container image is pulled securely. If these credentials remain the same across all jobs, refactoring via YAML anchors could reduce future maintenance overhead.
325-327: Credentials Block in "e2e-stream-crud-with-readreplica" Job
This credentials configuration follows the same secure pattern as other jobs. It is recommended to review if consolidating the credentials block into a shared YAML anchor or template might benefit long-term maintainability.
379-381: Credentials Block in "e2e-stream-crud-with-mirror" Job
The addition of the credentials is consistent and correctly implemented. As these settings are repetitive, exploring YAML reusability techniques (like anchors) could streamline your workflow configuration and reduce potential errors during future updates.internal/net/grpc/errdetails/errdetails.go (2)
89-173: Consider refactoring repetitive switch case patternsThe
MarshalJSONmethod contains numerous switch cases with identical patterns, which could be refactored to reduce code duplication. Consider extracting the common pattern into a helper function or using reflection to simplify this code.func (d *Detail) MarshalJSON() (body []byte, err error) { if d == nil { return nil, nil } typeName := strings.TrimPrefix(strings.TrimPrefix(d.TypeURL, typePrefix), typePrefixV1) - switch typeName { - case DebugInfoMessageName: - m, ok := d.Message.(*DebugInfo) - if ok { - body, err = m.MarshalJSON() - } - case ErrorInfoMessageName: - m, ok := d.Message.(*ErrorInfo) - if ok { - body, err = m.MarshalJSON() - } - // ... many more identical pattern cases - default: - body, err = protojson.Marshal(d.Message) - } + // Try to use MarshalJSON method if available + if marshaler, ok := d.Message.(interface{ MarshalJSON() ([]byte, error) }); ok { + body, err = marshaler.MarshalJSON() + } else { + // Fallback to protojson.Marshal + body, err = protojson.Marshal(d.Message) + } if err != nil || body == nil { log.Warnf("failed to Marshal type: %s, object %#v to JSON body %v, error: %v", typeName, d, body, err) return nil, err } return body, nil }
80-82: Add documentation for the new Details structThe new
Detailsstruct lacks documentation explaining its purpose and usage. Consider adding comments to clarify how and when this struct should be used, which will help other developers understand its role in the error handling system.+// Details is a container for multiple error detail messages +// It's used to serialize a collection of Detail objects into a structured format type Details struct { Details []Detail `json:"details,omitempty" yaml:"details"` }apis/swagger/v1/vald/search.swagger.json (9)
17-18: Excellent update for the/linearsearchendpoint summary.
The new summary now provides a detailed overview including a status code table and troubleshooting guidance. Please verify that newline characters and markdown formatting render correctly in your Swagger UI.
50-51: Improved documentation for the/linearsearch/idendpoint.
The updated summary and description clearly state the purpose of the endpoint, including important notes on the pre-indexing requirement and detailed error information. Ensure that the embedded HTML (e.g.,<br>) is consistently rendered across different Swagger viewers.
83-84: Clear documentation for the/linearsearch/id/multipleendpoint.
The summary succinctly explains the endpoint function, and the description includes a notice on gRPC message size limitations as well as detailed error troubleshooting. Please check if the inline comment syntax (e.g.,// ---) in the description might affect JSON rendering or clarity.
116-117: Consistent summary for the/linearsearch/multipleendpoint.
The reworked summary and description now align with the format used in other endpoints. Consider ensuring that the table formatting and HTML notice styling remain consistent across all endpoints.
149-150: Enhanced summary for the/searchendpoint.
The updated summary and description are now more comprehensive with a clear status code table and troubleshooting steps. Double-check that the formatting (line breaks and table alignment) renders as intended in the API documentation interface.
182-183: Improved details for the/search/idendpoint.
The new summary and description now effectively communicate the preconditions and error handling for searches by ID. Verify that the use of HTML<br>tags integrates well with the overall documentation styling.
215-216: Refined documentation for the/search/id/multipleendpoint.
The enhanced summary and description include vital troubleshooting and status code information, along with a gRPC message size notice. Ensure that the HTML elements (like<div class="notice">) are consistently interpreted in your documentation toolchain.
248-249: Comprehensive update for the/search/multipleendpoint.
The summary and description have been significantly enriched with detailed status codes, troubleshooting guidelines, and a notice about gRPC message size limits. Confirm that the formatting, especially regarding embedded HTML and table structures, is uniform with other endpoints.
1-280: Overall Documentation Consistency.
The updates across all endpoints in this file significantly enhance the clarity and usability of the API documentation. Be sure to test the visual rendering of these markdown and HTML elements in the Swagger UI to ensure that end users receive a consistent experience.🧰 Tools
🪛 Checkov (3.2.334)
[HIGH] 1-535: Ensure that security requirement defined in securityDefinitions - version 2.0 files
(CKV_OPENAPI_6)
[HIGH] 1-535: Ensure that the global security field has rules defined
(CKV_OPENAPI_4)
[HIGH] 1-535: Ensure that security operations is not empty.
(CKV_OPENAPI_5)
[HIGH] 1-535: Ensure that securityDefinitions is defined and not empty - version 2.0 files
(CKV_OPENAPI_1)
internal/net/grpc/pool/pool.go (1)
606-633: Consider different health criteria for high availability.The IsHealthy method has different health criteria for IP-based connections (all slots must be healthy) versus hostname-based connections (at least one slot must be healthy). While this makes sense for different use cases, consider adding a configuration option to choose the health criteria based on the application's availability requirements rather than just the address type.
go.mod (2)
5-349: Updated Dependency Versions in the Replace Block
Multiple dependency versions have been revised (for example, updatingcloud.google.com/goto v0.120.0,cloud.google.com/go/bigqueryto v1.67.0,github.com/BurntSushi/tomlto v1.5.0, among many others). These changes help modernize and secure the codebase. However, please verify that these upgrades are fully backward‐compatible with the rest of the project and do not introduce unexpected breaking changes.
425-545: Indirect Dependencies Require Block – Consistency and Stability Check
This block registers many indirect dependencies with updated versions. While most appear standard, the overall large number of dependency updates calls for extra caution regarding potential resolution conflicts. It is recommended to test with a clean module cache (for instance, by runninggo clean -modcache) and execute a full build/test cycle to detect any issues stemming from these updates.apis/swagger/v1/vald/insert.swagger.json (2)
18-18: Detailed Description for /insert Endpoint
The added description provides a comprehensive table of status codes and troubleshooting steps. Verify that the formatting (especially markdown-like formatting within the JSON string) displays correctly in the final API documentation.
51-51: Comprehensive Description for /insert/multiple Endpoint
The description now includes an HTML notice regarding gRPC message size limitations and detailed troubleshooting guidance. Please ensure the embedded HTML renders as intended in your API documentation tool.apis/swagger/v1/vald/update.swagger.json (1)
50-52: Improved Documentation for /update/multiple Endpoint
The summary and description now clearly explain the multi-update operation and include a useful HTML notice about potential gRPC message size limitations. Please double-check that the HTML formatting is rendered correctly in the API docs.pkg/agent/core/faiss/service/faiss.go (1)
1246-1248: Improved error handling logic in Close method.The usage of
!errors.IsNot()(double negative) might be slightly confusing at first glance, but it correctly implements the logic to ignore context cancellation and deadline errors.Consider the following alternative for improved readability:
- if !errors.IsNot(err, context.Canceled, context.DeadlineExceeded) { + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { err = nil }🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by testsapis/docs/v1/docs.md (9)
1372-1374: Mirror Service Overview ClarityThe added overview clearly states that the Mirror Service provides the
Registerinterface for the Vald Mirror Gateway. Consider reviewing the phrasing for consistency with other service overviews (for example, you might use “provides” rather than “is responsible for providing”) to improve parallelism across the document.
1377-1378: Register Method Table FormattingThe table row for the
RegisterRPC is detailed and informative. Please verify that the column alignments, status code listings, and any embedded markdown formatting are consistent with other method tables elsewhere in the document. Adding brief troubleshooting notes could further enhance its usefulness.
1636-1639: Filter Service OverviewThe overview for the Filter service is comprehensive. One minor suggestion: change “forward user request” to “forwards user requests” for grammatical accuracy.
1687-1688: Index Service Overview BrevityThe overview for the Index service is brief. Consider expanding it slightly—perhaps by mentioning that it provides key index information and statistics—to give readers a better idea of its full capabilities.
1714-1720: Insert Service Overview and Typo CorrectionThe Insert Service overview is clear in stating that it handles the insertion of new vectors into the
vald-agent. In the detailed method table row, the term “Inset RPC” appears; this should be corrected to “Insert RPC”. Ensuring correct terminology helps maintain a professional quality across the documentation.
1747-1749: Object Service Overview ClarityThe Object Service overview now explains its role in retrieving and verifying vectors. For clarity, consider rephrasing “getting inserted vectors” to “retrieving inserted vectors.”
1906-1908: Update Service Overview RewordingThe Update Service overview currently states “updates to new vector from inserted vector,” which is a bit awkward. Rephrasing it—for example, “Update Service is responsible for modifying an existing vector in the
vald-agent”—could improve clarity.
1928-1929: UpdateTimestamp RPC PlaceholderThe
UpdateTimestampRPC entry currently includes “Status Code TODO” and “Troubleshooting TODO” placeholders. Please update these placeholders with the appropriate status codes and troubleshooting guidance so that the documentation is complete before finalization.
1372-1982: Overall Documentation Table ConsistencyThe updated documentation now includes detailed method tables complete with status codes and troubleshooting guidelines. Please ensure that formatting, column alignment, and the replacement of any remaining TODO placeholders are consistent across all service sections to maintain a high-quality and professional reference document.
🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1416-1416: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1420-1420: Spaces inside emphasis markers
null(MD037, no-space-in-emphasis)
1514-1514: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Bare URL used
null(MD034, no-bare-urls)
1600-1600: Reference links and images should use a label that is defined
Missing link or image reference definition: "google.rpc.code.permissiondenied"(MD052, reference-links-images)
1624-1624: Link fragments should be valid
null(MD051, link-fragments)
internal/iter/iter.go (3)
96-114: Add scenario tests for Seq and Values.
Both methods rely on iteration and context cancellation. Confirm that your test coverage includes variousnumsizes, plus early cancellation scenarios and the mod function's influence on values.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 96-97: internal/iter/iter.go#L96-L97
Added lines #L96 - L97 were not covered by tests
[warning] 101-107: internal/iter/iter.go#L101-L107
Added lines #L101 - L107 were not covered by tests
[warning] 109-111: internal/iter/iter.go#L109-L111
Added lines #L109 - L111 were not covered by tests
145-147: Validate Raw method usage.
Although straightforward, ensure coverage verifies that the returned slice is the originalarray—particularly ifarraycould benilin future changes.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 145-146: internal/iter/iter.go#L145-L146
Added lines #L145 - L146 were not covered by tests
149-152: Add assertion for Len.
ConfirmLen()reflects the configurednum. Testing it helps ensure consistency and catches off-by-one errors that might appear ifnumis changed or reset.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 150-151: internal/iter/iter.go#L150-L151
Added lines #L150 - L151 were not covered by testsinternal/test/data/vector/noise/noise_test.go (2)
16-107: All test code is commented out.These placeholders for
TestNewremain unimplemented, and the entire block is commented out. Consider either removing unused code entirely or implementing the tests to ensure coverage of noise generator initialization.Would you like me to help draft concrete test cases for these placeholders?
109-207: All test code is commented out.These placeholders for
Test_noiseGenerator_Modremain unimplemented. Proper tests are vital for verifying the noise modification logic. Consider either removing them if not needed or fleshing them out to validate this feature.Would you like me to help create functional tests for verifying the noise addition process?
internal/test/data/vector/noise/noise.go (1)
147-155: Consider seeding the pseudo-random generator for reproducibility.For deterministic test runs, seed the random generator (e.g., via
rand.Seed(someConstant)). This ensures that repeated executions produce the same noise patterns, aiding debugging and comparison.internal/params/params.go (1)
130-134: Consider simplifying the condition logic.The compound condition used here is somewhat complex and could be refactored for better readability.
- if exist, _, err := file.ExistsWithDetail(d.configFilePath); !d.showVersion && - (!exist || d.configFilePath == "") { + exist, _, err := file.ExistsWithDetail(d.configFilePath) + if !d.showVersion && (!exist || d.configFilePath == "") {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
apis/grpc/v1/agent/core/agent.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/agent/sidecar/sidecar.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/discoverer/discoverer.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/egress/egress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/filter/ingress/ingress_filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/meta/meta.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/mirror/mirror_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/payload/payload.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/rpc/errdetails/error_details.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/filter_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/flush_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/index_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/insert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/object_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/remove_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/search_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/update_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert.pb.gois excluded by!**/*.pb.go,!**/*.pb.goapis/grpc/v1/vald/upsert_vtproto.pb.gois excluded by!**/*.pb.go,!**/*.pb.goexample/client/go.sumis excluded by!**/*.sumgo.sumis excluded by!**/*.sumrust/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (153)
.gitfiles(15 hunks).github/ISSUE_TEMPLATE/bug_report.md(1 hunks).github/ISSUE_TEMPLATE/security_issue_report.md(1 hunks).github/PULL_REQUEST_TEMPLATE.md(1 hunks).github/workflows/_docker-image.yaml(1 hunks).github/workflows/_release-pr.yaml(1 hunks).github/workflows/build-binaries.yaml(1 hunks).github/workflows/build-protobuf.yaml(1 hunks).github/workflows/chatops.yaml(5 hunks).github/workflows/codeql-analysis.yml(2 hunks).github/workflows/coverage.yaml(1 hunks).github/workflows/e2e-chaos.yaml(4 hunks).github/workflows/e2e-code-bench-agent.yaml(2 hunks).github/workflows/e2e-max-dim.yaml(1 hunks).github/workflows/e2e-profiling.yaml(1 hunks).github/workflows/e2e.yaml(8 hunks).github/workflows/format.yaml(3 hunks).github/workflows/fossa.yaml(1 hunks).github/workflows/helm-lint.yaml(3 hunks).github/workflows/helm.yaml(2 hunks).github/workflows/reviewdog-k8s.yaml(3 hunks).github/workflows/reviewdog.yaml(2 hunks).github/workflows/test-hack.yaml(1 hunks).github/workflows/unit-test-rust.yaml(1 hunks).github/workflows/unit-test.yaml(3 hunks).github/workflows/update-deps.yaml(1 hunks)Makefile(8 hunks)Makefile.d/e2e.mk(1 hunks)Makefile.d/functions.mk(1 hunks)Makefile.d/minikube.mk(1 hunks)Makefile.d/tools.mk(2 hunks)apis/docs/v1/docs.md(10 hunks)apis/swagger/v1/mirror/mirror.swagger.json(1 hunks)apis/swagger/v1/vald/filter.swagger.json(8 hunks)apis/swagger/v1/vald/flush.swagger.json(1 hunks)apis/swagger/v1/vald/index.swagger.json(5 hunks)apis/swagger/v1/vald/insert.swagger.json(2 hunks)apis/swagger/v1/vald/object.swagger.json(4 hunks)apis/swagger/v1/vald/remove.swagger.json(3 hunks)apis/swagger/v1/vald/search.swagger.json(8 hunks)apis/swagger/v1/vald/update.swagger.json(3 hunks)apis/swagger/v1/vald/upsert.swagger.json(2 hunks)charts/vald/values/dev.yaml(1 hunks)cmd/index/job/deletion/main_test.go(1 hunks)dockers/agent/core/agent/Dockerfile(1 hunks)dockers/agent/core/faiss/Dockerfile(1 hunks)dockers/agent/core/ngt/Dockerfile(1 hunks)dockers/agent/sidecar/Dockerfile(1 hunks)dockers/binfmt/Dockerfile(1 hunks)dockers/buildbase/Dockerfile(1 hunks)dockers/buildkit/Dockerfile(1 hunks)dockers/buildkit/syft/scanner/Dockerfile(1 hunks)dockers/ci/base/Dockerfile(1 hunks)dockers/dev/Dockerfile(1 hunks)dockers/discoverer/k8s/Dockerfile(1 hunks)dockers/example/client/Dockerfile(1 hunks)dockers/gateway/filter/Dockerfile(1 hunks)dockers/gateway/lb/Dockerfile(1 hunks)dockers/gateway/mirror/Dockerfile(1 hunks)dockers/index/job/correction/Dockerfile(1 hunks)dockers/index/job/creation/Dockerfile(1 hunks)dockers/index/job/deletion/Dockerfile(1 hunks)dockers/index/job/readreplica/rotate/Dockerfile(1 hunks)dockers/index/job/save/Dockerfile(1 hunks)dockers/index/operator/Dockerfile(1 hunks)dockers/manager/index/Dockerfile(1 hunks)dockers/operator/helm/Dockerfile(1 hunks)dockers/tools/benchmark/job/Dockerfile(1 hunks)dockers/tools/benchmark/operator/Dockerfile(1 hunks)dockers/tools/cli/loadtest/Dockerfile(1 hunks)example/client/go.mod(2 hunks)example/client/go.mod.default(1 hunks)go.mod(14 hunks)hack/go.mod.default(4 hunks)internal/backoff/backoff.go(2 hunks)internal/backoff/backoff_test.go(1 hunks)internal/backoff/option.go(0 hunks)internal/backoff/option_test.go(0 hunks)internal/client/v1/client/discoverer/discover.go(1 hunks)internal/client/v1/client/discoverer/discover_test.go(5 hunks)internal/config/benchmark_test.go(4 hunks)internal/config/config.go(2 hunks)internal/config/config_test.go(1 hunks)internal/config/grpc.go(1 hunks)internal/config/index_deleter_test.go(1 hunks)internal/core/algorithm/usearch/usearch_test.go(16 hunks)internal/db/nosql/cassandra/option_test.go(8 hunks)internal/errors/errors.go(2 hunks)internal/errors/errors_test.go(1 hunks)internal/errors/k8s.go(1 hunks)internal/errors/params.go(1 hunks)internal/file/file.go(3 hunks)internal/file/file_test.go(1 hunks)internal/info/info_test.go(17 hunks)internal/iter/iter.go(1 hunks)internal/net/dialer_test.go(1 hunks)internal/net/grpc/client.go(13 hunks)internal/net/grpc/errdetails/errdetails.go(6 hunks)internal/net/grpc/metadata_test.go(1 hunks)internal/net/grpc/option.go(1 hunks)internal/net/grpc/option_test.go(46 hunks)internal/net/grpc/pool/option.go(1 hunks)internal/net/grpc/pool/option_test.go(6 hunks)internal/net/grpc/pool/pool.go(6 hunks)internal/net/grpc/pool/pool_bench_test.go(2 hunks)internal/net/grpc/server_test.go(2 hunks)internal/net/grpc/status/status.go(1 hunks)internal/net/grpc/status/status_test.go(1 hunks)internal/net/grpc/stream.go(4 hunks)internal/net/grpc/stream_test.go(4 hunks)internal/net/http/client/client.go(2 hunks)internal/net/http/client/client_test.go(3 hunks)internal/net/http/client/option_test.go(4 hunks)internal/net/net.go(1 hunks)internal/net/net_test.go(3 hunks)internal/net/quic/conn_test.go(1 hunks)internal/net/quic/listener_test.go(1 hunks)internal/params/option.go(2 hunks)internal/params/option_test.go(1 hunks)internal/params/params.go(4 hunks)internal/params/params_test.go(12 hunks)internal/rand/rand.go(1 hunks)internal/runner/runner.go(4 hunks)internal/servers/server/option_test.go(1 hunks)internal/servers/server/server.go(1 hunks)internal/servers/servers.go(3 hunks)internal/sync/errgroup/group.go(7 hunks)internal/sync/errgroup/group_test.go(16 hunks)internal/test/data/vector/gen.go(0 hunks)internal/test/data/vector/noise/noise.go(1 hunks)internal/test/data/vector/noise/noise_test.go(1 hunks)internal/test/data/vector/noise/option.go(1 hunks)internal/test/data/vector/noise/option_test.go(1 hunks)internal/test/mock/server_stream.go(1 hunks)internal/timeutil/time.go(3 hunks)internal/timeutil/time_test.go(1 hunks)k8s/agent/ngt/configmap.yaml(1 hunks)k8s/discoverer/deployment.yaml(1 hunks)k8s/gateway/gateway/lb/deployment.yaml(1 hunks)k8s/gateway/gateway/mirror/deployment.yaml(1 hunks)k8s/index/operator/deployment.yaml(1 hunks)k8s/manager/index/deployment.yaml(1 hunks)pkg/agent/core/faiss/service/faiss.go(3 hunks)pkg/agent/core/ngt/service/ngt.go(5 hunks)pkg/agent/core/ngt/service/ngt_test.go(8 hunks)pkg/agent/core/ngt/service/option_test.go(8 hunks)pkg/discoverer/k8s/service/discover_test.go(4 hunks)pkg/gateway/lb/handler/grpc/aggregation_test.go(2 hunks)pkg/gateway/lb/handler/grpc/handler_test.go(2 hunks)pkg/gateway/mirror/service/mirror.go(2 hunks)pkg/index/job/deletion/config/config_test.go(1 hunks)pkg/index/job/deletion/service/deleter_test.go(1 hunks)pkg/index/job/deletion/service/options_test.go(1 hunks)
💤 Files with no reviewable changes (3)
- internal/backoff/option.go
- internal/test/data/vector/gen.go
- internal/backoff/option_test.go
🚧 Files skipped from review as they are similar to previous changes (110)
- dockers/buildkit/syft/scanner/Dockerfile
- internal/errors/params.go
- .github/workflows/helm.yaml
- .github/workflows/test-hack.yaml
- dockers/gateway/filter/Dockerfile
- dockers/buildbase/Dockerfile
- .github/workflows/coverage.yaml
- dockers/agent/sidecar/Dockerfile
- dockers/dev/Dockerfile
- dockers/discoverer/k8s/Dockerfile
- dockers/index/job/readreplica/rotate/Dockerfile
- dockers/gateway/mirror/Dockerfile
- .github/workflows/e2e-profiling.yaml
- dockers/tools/benchmark/operator/Dockerfile
- dockers/index/job/deletion/Dockerfile
- dockers/agent/core/agent/Dockerfile
- dockers/tools/benchmark/job/Dockerfile
- dockers/example/client/Dockerfile
- dockers/tools/cli/loadtest/Dockerfile
- .github/workflows/fossa.yaml
- .github/workflows/e2e-max-dim.yaml
- apis/swagger/v1/mirror/mirror.swagger.json
- internal/net/grpc/status/status.go
- dockers/operator/helm/Dockerfile
- .github/workflows/unit-test-rust.yaml
- .github/workflows/unit-test.yaml
- Makefile.d/minikube.mk
- .github/workflows/update-deps.yaml
- .github/workflows/e2e-chaos.yaml
- k8s/discoverer/deployment.yaml
- dockers/index/job/creation/Dockerfile
- example/client/go.mod.default
- internal/backoff/backoff_test.go
- dockers/manager/index/Dockerfile
- dockers/agent/core/ngt/Dockerfile
- .github/workflows/format.yaml
- Makefile.d/e2e.mk
- .github/workflows/e2e-code-bench-agent.yaml
- k8s/index/operator/deployment.yaml
- .github/workflows/_release-pr.yaml
- internal/runner/runner.go
- k8s/manager/index/deployment.yaml
- .github/workflows/_docker-image.yaml
- .github/workflows/reviewdog.yaml
- dockers/agent/core/faiss/Dockerfile
- internal/rand/rand.go
- internal/net/grpc/option.go
- .github/workflows/build-protobuf.yaml
- internal/file/file_test.go
- internal/errors/errors_test.go
- internal/core/algorithm/usearch/usearch_test.go
- .github/workflows/chatops.yaml
- internal/config/grpc.go
- Makefile.d/tools.mk
- internal/net/net.go
- cmd/index/job/deletion/main_test.go
- charts/vald/values/dev.yaml
- .github/workflows/helm-lint.yaml
- dockers/index/job/save/Dockerfile
- internal/config/benchmark_test.go
- dockers/gateway/lb/Dockerfile
- internal/backoff/backoff.go
- internal/net/net_test.go
- pkg/index/job/deletion/config/config_test.go
- internal/servers/server/server.go
- Makefile.d/functions.mk
- apis/swagger/v1/vald/object.swagger.json
- dockers/binfmt/Dockerfile
- .github/workflows/build-binaries.yaml
- internal/net/http/client/client.go
- pkg/agent/core/ngt/service/ngt_test.go
- .github/ISSUE_TEMPLATE/bug_report.md
- internal/net/grpc/status/status_test.go
- dockers/index/operator/Dockerfile
- .github/PULL_REQUEST_TEMPLATE.md
- internal/net/grpc/metadata_test.go
- pkg/gateway/mirror/service/mirror.go
- internal/net/dialer_test.go
- internal/net/grpc/pool/pool_bench_test.go
- dockers/buildkit/Dockerfile
- internal/net/http/client/option_test.go
- internal/servers/servers.go
- .github/ISSUE_TEMPLATE/security_issue_report.md
- dockers/index/job/correction/Dockerfile
- internal/db/nosql/cassandra/option_test.go
- internal/client/v1/client/discoverer/discover_test.go
- k8s/gateway/gateway/mirror/deployment.yaml
- internal/net/grpc/pool/option_test.go
- internal/net/grpc/server_test.go
- k8s/gateway/gateway/lb/deployment.yaml
- internal/net/quic/listener_test.go
- dockers/ci/base/Dockerfile
- .github/workflows/reviewdog-k8s.yaml
- internal/client/v1/client/discoverer/discover.go
- pkg/agent/core/ngt/service/option_test.go
- internal/config/index_deleter_test.go
- pkg/discoverer/k8s/service/discover_test.go
- apis/swagger/v1/vald/index.swagger.json
- internal/params/params_test.go
- internal/sync/errgroup/group.go
- internal/net/http/client/client_test.go
- pkg/gateway/lb/handler/grpc/handler_test.go
- internal/net/grpc/pool/option.go
- internal/timeutil/time.go
- internal/config/config.go
- hack/go.mod.default
- Makefile
- internal/net/grpc/client.go
- example/client/go.mod
- pkg/gateway/lb/handler/grpc/aggregation_test.go
🧰 Additional context used
🧠 Learnings (1)
.gitfiles (1)
Learnt from: datelier
PR: vdaas/vald#2807
File: rust/bin/agent/src/handler/object.rs:36-38
Timestamp: 2025-04-01T12:29:37.305Z
Learning: In Rust code for Vald handlers, avoid using unwrap() on Option types. Instead, handle None values explicitly by returning appropriate error statuses to maintain robustness.
🧬 Code Definitions (4)
internal/errors/errors.go (1)
internal/net/grpc/status/status.go (2)
Errorf(129-131)Is(48-50)
internal/net/grpc/stream.go (1)
internal/net/grpc/status/status.go (5)
Is(48-50)Errorf(129-131)ParseError(133-183)Code(41-41)Error(125-127)
internal/net/grpc/pool/pool.go (4)
internal/net/grpc/client.go (3)
ClientConn(50-50)DialOption(49-49)New(129-154)internal/net/net.go (5)
Conn(41-41)Parse(188-222)SplitHostPort(242-264)JoinHostPort(233-235)DefaultResolver(88-88)internal/net/grpc/status/status.go (4)
Is(48-50)New(44-46)Errorf(129-131)Error(125-127)internal/net/grpc/codes/codes.go (2)
Canceled(26-26)DeadlineExceeded(29-29)
pkg/agent/core/ngt/service/ngt.go (2)
internal/errors/errors.go (1)
IsNot(182-192)internal/errors/agent.go (1)
ErrUncommittedIndexNotFound(98-98)
🪛 GitHub Check: codecov/patch
internal/iter/iter.go
[warning] 52-67: internal/iter/iter.go#L52-L67
Added lines #L52 - L67 were not covered by tests
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests
[warning] 81-87: internal/iter/iter.go#L81-L87
Added lines #L81 - L87 were not covered by tests
[warning] 89-91: internal/iter/iter.go#L89-L91
Added lines #L89 - L91 were not covered by tests
[warning] 96-97: internal/iter/iter.go#L96-L97
Added lines #L96 - L97 were not covered by tests
[warning] 101-107: internal/iter/iter.go#L101-L107
Added lines #L101 - L107 were not covered by tests
[warning] 109-111: internal/iter/iter.go#L109-L111
Added lines #L109 - L111 were not covered by tests
[warning] 117-123: internal/iter/iter.go#L117-L123
Added lines #L117 - L123 were not covered by tests
[warning] 125-127: internal/iter/iter.go#L125-L127
Added lines #L125 - L127 were not covered by tests
[warning] 132-137: internal/iter/iter.go#L132-L137
Added lines #L132 - L137 were not covered by tests
[warning] 139-141: internal/iter/iter.go#L139-L141
Added lines #L139 - L141 were not covered by tests
[warning] 145-146: internal/iter/iter.go#L145-L146
Added lines #L145 - L146 were not covered by tests
[warning] 150-151: internal/iter/iter.go#L150-L151
Added lines #L150 - L151 were not covered by tests
internal/test/mock/server_stream.go
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by tests
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by tests
pkg/agent/core/faiss/service/faiss.go
[warning] 1126-1126: pkg/agent/core/faiss/service/faiss.go#L1126
Added line #L1126 was not covered by tests
[warning] 1246-1248: pkg/agent/core/faiss/service/faiss.go#L1246-L1248
Added lines #L1246 - L1248 were not covered by tests
[warning] 1250-1250: pkg/agent/core/faiss/service/faiss.go#L1250
Added line #L1250 was not covered by tests
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by tests
[warning] 1268-1268: pkg/agent/core/faiss/service/faiss.go#L1268
Added line #L1268 was not covered by tests
pkg/agent/core/ngt/service/ngt.go
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by tests
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
⏰ Context from checks skipped due to timeout of 90000ms (30)
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
- GitHub Check: runner / go build
- GitHub Check: build / build
- GitHub Check: build / build
🔇 Additional comments (114)
internal/config/config_test.go (1)
999-1003: Test case update reflects changed behavior for non-existent files.The test case has been modified to expect an empty return value when a file doesn't exist, rather than returning the file path itself. This indicates a change in the
GetActualValuefunction's behavior to return empty strings for non-existent files, which is a more sensible default than returning the raw file path.internal/timeutil/time_test.go (1)
183-349: Implement the commented-out test functionsBoth
TestDurationString_DurationandTestDurationString_DurationWithDefaultfunctions are still commented out despite previous review comments requesting their implementation. These tests are important to ensure the proper functionality of theDurationStringtype and its methods.Please implement these test functions with actual test cases instead of the placeholder TODOs. Based on the existing test patterns in
TestParseandTestParseWithDefault, include test cases that verify:
- Valid duration strings (e.g., "1s", "1m", "1h")
- Empty string handling
- Invalid format handling
- Default value behavior (for DurationWithDefault)
internal/file/file.go (3)
259-259: Improved error handling logicReplacing
err != nil && !errors.Is(err, io.EOF)witherrors.IsNot(err, io.EOF)improves the code's readability and aligns with the project's error handling patterns. This abstraction makes the intent clearer and the code more consistent.
374-391: Well-structured helper function for path resolutionThe new
AbsolutePathfunction encapsulates path resolution logic in a single place, handling multiple edge cases properly:
- Empty path handling
- Already absolute paths
- Relative path resolution with working directory
- Clean path formatting
This abstraction improves maintainability by centralizing path resolution logic that was likely scattered throughout the codebase.
564-564: Good refactoring to use the new AbsolutePath functionThe
Joinfunction now leverages the newAbsolutePathfunction, removing duplicated logic and ensuring consistent path handling across the codebase..github/workflows/codeql-analysis.yml (4)
70-72: Container Credentials Usage
The addition of container credentials using the secretsPACKAGE_USERandPACKAGE_TOKENis a good security practice. Ensure that these secrets are properly managed in your repository settings and consider adding documentation on their usage for clarity.
84-88: Updated CodeQL Initialization to v3
Upgrading the initialization step to usegithub/codeql-action/init@v3with dynamic inputs for languages and build mode is well executed. Please double-check that./.github/codeql/codeql-config.yamlremains compatible with the v3 action.
90-91: Conditional Autobuild for Go
Using a conditional check (if: matrix.language == 'go') for the autobuild step with CodeQL v3 is appropriate since Go projects generally require this step. Confirm that no other language in your matrix would inadvertently need a similar build step.
92-96: Updated Analysis Step
The CodeQL analysis step now utilizesgithub/codeql-action/analyze@v3and includes a dynamic categorization based on the matrix language. This is a clear update that aligns with the new CodeQL versions. Ensure that the category naming (e.g.,/language:${{matrix.language}}) meets your project's standards and reporting needs.internal/servers/server/option_test.go (1)
3727-4066: Implement the commented-out gRPC test functionsThese four test functions for gRPC-related options are commented out, which means they're not being executed in the test suite. This reduces test coverage for these gRPC server features, which are important for the new E2E testing framework.
The functions that need implementation are:
TestWithGRPCMaxConcurrentStreamsTestWithGRPCNumStreamWorkersTestWithGRPCSharedWriteBufferTestWithGRPCWaitForHandlers#!/bin/bash # Check if the actual implementation of these options exists in the codebase echo "Checking for WithGRPCMaxConcurrentStreams implementation..." rg "func WithGRPCMaxConcurrentStreams" --type go -A 5 echo "Checking for WithGRPCNumStreamWorkers implementation..." rg "func WithGRPCNumStreamWorkers" --type go -A 5 echo "Checking for WithGRPCSharedWriteBuffer implementation..." rg "func WithGRPCSharedWriteBuffer" --type go -A 5 echo "Checking for WithGRPCWaitForHandlers implementation..." rg "func WithGRPCWaitForHandlers" --type go -A 5internal/info/info_test.go (2)
341-341: New field added to multiple test structures.The
valdReplacer *strings.Replacerfield has been added to all thefieldsstructs in test functions. This is correctly initialized tonilin all test examples, matching the pattern used for other fields in these structures.Also applies to: 370-371, 393-394, 428-429, 622-623, 645-646, 726-727, 749-750, 801-802, 825-826, 848-849, 883-884
996-1094: Implement test cases for ShortString() method.This new test function addresses a previous review comment requesting test coverage for the
ShortString()method in theStackTracestruct. However, the actual test cases are marked as "TODO" and not implemented yet.Please implement at least one test case to verify the method's behavior. Based on the implementation
s.URL + " " + s.FuncName, consider adding a test like:// Replace one of the TODO blocks with an actual test case: { name: "test_correct_shortstring_format", fields: fields { URL: "github.com/vdaas/vald/internal/info", FuncName: "TestFunction", File: "info_test.go", Line: 42, }, want: want{ want: "github.com/vdaas/vald/internal/info TestFunction", }, checkFunc: defaultCheckFunc, beforeFunc: func(t *testing.T,) { t.Helper() }, afterFunc: func(t *testing.T,) { t.Helper() }, },apis/swagger/v1/vald/filter.swagger.json (6)
17-17: Enhanced Summary for/insert/objectEndpointThe updated summary now provides a clear description of the InsertObject RPC and includes a well-formatted status code table that improves endpoint documentation clarity. Please verify that the style aligns with the overall API documentation guidelines.
49-49: Clear and Informative Summary for/insert/object/multipleEndpointThe revised summary details the MultiInsertObject RPC with a concise explanation and status code table. The emphasis on "1 request" helps clarify that multiple objects are handled in one go. Consider reviewing if the emphasis style is consistent with other parts of the documentation.
81-81: Improved Summary for/search/objectEndpointThe updated summary clearly outlines the SearchObject RPC functionality along with its status codes. This enhancement increases the self-descriptiveness of the API endpoint.
113-113: Detailed Summary for/search/object/multipleEndpointThe summary now explains the bidirectional streaming nature of the StreamSearchObject RPC, including a description of the communication process and a comprehensive status code table. This detail will help API users understand the streaming mechanism better.
145-145: Clear Summary for/update/objectEndpointThe updated summary for the UpdateObject RPC conveys the operation effectively with an appropriate status code table. It now aligns well with the improved documentation style seen in other endpoints.
210-210: Comprehensive Summary for/upsert/objectEndpointThe summary now clearly specifies that the UpsertObject RPC both updates and adds a new object. This dual-purpose description, combined with the status code table, enhances the endpoint's documentation.
apis/swagger/v1/vald/upsert.swagger.json (2)
17-17: Updated Summary for Upsert Endpoint:
The revised summary effectively outlines the upsert RPC’s functionality, including detailed status codes and a troubleshooting guide. The multi-line format and table layout enhance clarity. Please double-check that the formatting is rendered as intended in your Swagger UI.
50-50: Updated Summary for MultiUpsert Endpoint:
The updated summary succinctly describes the multi-upsert operation, emphasizing the ability to update and add multiple vectors in a single request. The use of bold formatting for "1 request" is effective in highlighting the key differentiator.apis/swagger/v1/vald/remove.swagger.json (1)
50-51: Improved Multi-Vector Removal DocumentationThe summary now emphasizes that multiple vectors can be removed with a single request, and the description includes a structured table for status codes and troubleshooting instructions along with an HTML notice about gRPC message size limitations. Ensure that the embedded HTML (i.e.
<div class="notice">) is rendered as expected in your Swagger UI.internal/net/grpc/errdetails/errdetails.go (7)
80-82: New Details struct improves structured error representationThis new
Detailsstruct provides a formal container for multiple error details, allowing for better structured error handling and serialization.
93-94: Improved type URL handling for multiple prefixesThe updated string trimming logic now correctly handles both
typePrefixandtypePrefixV1prefixes, making the code more robust for different message type formats.Also applies to: 330-331
165-166: Enhanced flexibility with default marshaling caseAdding a default case that uses
protojson.Marshalimproves the flexibility of the error handling system by supporting message types not explicitly covered in the switch statement.
168-171: Improved error handling with detailed loggingThe enhanced error handling now properly logs warnings if marshaling fails or produces nil results, which will aid in troubleshooting and debugging.
315-322: Improved serialization with nil checks and better byte handlingThe
Serializefunction has been improved by adding nil checks for byte slices and usingconv.Btoafor efficient byte-to-string conversion, making the code more robust.
416-421: Added fallback for handling unknown message typesThe new default case in
AnyToErrorDetailattempts to unmarshal unknown types using the generic unmarshal method, improving the function's ability to handle a wider range of message types.
424-424: Enhanced logging with type informationIncluding the type name in the log warning message provides more context for debugging, making it easier to identify issues with specific message types.
internal/net/grpc/pool/pool.go (8)
17-20: Good documentation improvements!The updated package documentation clearly explains the implementation approach (lock-free) and new features (DNS lookup, port scanning, metrics collection), which will help users understand the package's capabilities.
49-71: Interface changes well documented.The Conn interface has been updated with context parameters for methods like Disconnect, Do, Get, and Reconnect. These changes are well documented with clear comments explaining the purpose of each method.
79-121: Well-implemented graceful connection closure.The new Close method for poolConn implements a thoughtful approach to graceful connection closure with proper state checking and timeout handling. The use of a ticker ensures we don't overwhelm the system with rapid close attempts.
123-158: Good lock-free pool implementation.The pool struct implementation using atomic operations is well-designed. Using atomic.Pointer for thread-safe access to connection slots without locks is an efficient approach for high-concurrency scenarios.
661-696: IP lookups with ping verification is a good approach.The lookupIPAddr function not only performs DNS lookups but also verifies connectivity with a quick ping to each IP. This helps filter out unreachable addresses early, improving connection reliability.
725-738: Excellent addition of metrics functionality.The new Metrics function provides a clean way to expose connection health information per target address. This will be valuable for monitoring and diagnostics of the connection pool health.
387-408: Good adaptive connection strategy.The Connect method intelligently chooses between single-target and multi-target connection approaches based on DNS resolution results and configuration. This flexibility helps handle various deployment scenarios effectively.
740-770: Good connection health checking.The isHealthy method properly handles different connection states and takes appropriate actions (like triggering connections for idle states). The logging of unhealthy states will help with debugging connection issues.
internal/net/quic/conn_test.go (4)
1-492:⚠️ Potential issueThe entire test file is commented out and not implemented
This file appears to be a placeholder for QUIC connection tests, but all code is commented out after the "NOT IMPLEMENTED BELOW" comment. While the test structure is well-defined, no actual tests are implemented.
To make this file functional, you need to:
- Uncomment and implement the test cases
- Add necessary imports including:
- context
- crypto/tls
- net
- reflect
- testing
- Your QUIC package
- go.uber.org/goleak
- errors package (with Is and Errorf functions)
- sync package (for Map with generics)
- Implement mock QUIC connections/streams for testing
Example implementation of a basic test case:
package quic import ( "context" "crypto/tls" "net" "reflect" "sync" "testing" "go.uber.org/goleak" "github.com/quic-go/quic-go" // or your quic package "github.com/vdaas/vald/internal/errors" // for Errorf and Is ) // Mock implementations for testing type mockQuicConnection struct { closeFunc func() error } func (m *mockQuicConnection) Close() error { if m.closeFunc != nil { return m.closeFunc() } return nil } // Implement other required interfaces func TestNewConn(t *testing.T) { // Test structure as defined in your file tests := []test{ { name: "successful_connection", args: args{ ctx: context.Background(), conn: &mockQuicConnection{}, }, want: want{ want: &Conn{ // Expected fields }, err: nil, }, checkFunc: defaultCheckFunc, }, // Add more test cases } // Test execution as defined in your file }
36-37: Errors package import is missingThe code uses
errors.Is()anderrors.Errorf(), but the import is not included. Go's standard errors package doesn't haveErrorf().Ensure you import the correct error package when implementing these tests. This is likely your custom internal errors package.
#!/bin/bash # Check which errors package is used in the project rg --type go "import.*errors" -A 3 | grep -B 1 -A 2 "Errorf"
90-91: Add goleak package importThe test uses
goleak.VerifyNone()to check for goroutine leaks, but the import is missing.Add the following import:
import "go.uber.org/goleak"To verify the correct package:
#!/bin/bash # Verify if go.uber.org/goleak is used elsewhere in the codebase rg --type go "import.*goleak" --stats
304-304:✅ Verification successful
Sync.Map with generics requires Go 1.18+
The code uses
sync.Map[string, quic.Connection]which is a generics feature available only in Go 1.18 and later.Ensure your project is configured to use Go 1.18+ in your go.mod file. Otherwise, modify this to use a non-generic sync.Map and add type assertions where needed.
🏁 Script executed:
#!/bin/bash # Check Go version in go.mod grep -A 2 "go " go.mod # Check for other usages of generic sync.Map in the codebase rg --type go "sync\.Map\[" --statsLength of output: 29612
Sync.Map generics confirmed – no changes needed
The
go.modfile shows the project is using Go 1.24.2, which fully supports generic usage ofsync.Map. All instances—including the commented-out code ininternal/net/quic/conn_test.go—are compliant with Go 1.18+.go.mod (2)
1-4: Go Version and Module Declaration Updated
The Go version has been updated from 1.24.0 to 1.24.2. Please ensure that your toolchain and all dependent tooling (build systems, CI pipelines, etc.) support the updated version.
351-424:❓ Verification inconclusive
Review of the Main Require Block Versions
In this block, several dependencies have been updated. Notably, there are several entries using highly unusual pseudo version formats such as:
github.com/go-redis/redis/v8at versionv8.0.0-00010101000000-000000000000github.com/gocraft/dbr/v2andgithub.com/gorilla/mux, among others, also show thev0.0.0-00010101000000-000000000000pattern.These placeholder-style versions can be a sign that either a proper version could not be resolved or that a temporary commit value is in use. Please verify that these versions are intentional (for example, by running
go mod tidyor checking the upstream module releases) to ensure stability and reproducibility of builds.
Action Required: Verify Pseudo-Version Dependencies in go.mod
The require block shows several dependencies using pseudo-version patterns (e.g.,
v8.0.0-00010101000000-000000000000). This pattern generally indicates that a proper version wasn’t resolved and a temporary commit hash was used instead. Please verify that these versions are intentional and meet your stability requirements by:
- Running
go mod tidyand/or examining the upstream releases for dependencies like:
github.com/go-redis/redis/v8github.com/gocraft/dbr/v2github.com/gorilla/mux- Confirming that no erroneous placeholder values are mistakenly in use.
apis/swagger/v1/vald/flush.swagger.json (1)
17-17: Enhanced Summary for the /flush DELETE Endpoint
The updated summary now provides a clear overview of the Flush RPC along with a neatly formatted status code table. This improvement aids in quickly understanding the operation’s behavior and expected outcomes.apis/swagger/v1/vald/insert.swagger.json (2)
17-17: Typographical Error in /insert Endpoint Summary
The summary still refers to "Inset RPC" instead of "Insert RPC". Please update the text for clarity and consistency with the operation's intent.
50-50: Clear Summary for /insert/multiple Endpoint
The summary clearly communicates that multiple vectors can be inserted in one request. This concise overview is helpful for API consumers who need to quickly understand the endpoint’s purpose.apis/swagger/v1/vald/update.swagger.json (1)
17-18: Enhanced Documentation for /update Endpoint
The updated summary and detailed description now offer a structured overview and well-articulated troubleshooting guidelines. It would be beneficial to verify that all error conditions and messages are consistent with backend behavior.pkg/agent/core/faiss/service/faiss.go (2)
1260-1261: Consistent error handling pattern applied.This change follows the same pattern of using
errors.IsNotto streamline error handling throughout the codebase.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1260-1260: pkg/agent/core/faiss/service/faiss.go#L1260
Added line #L1260 was not covered by tests
1268-1269: Consistent error handling pattern applied.This change follows the same pattern of using
errors.IsNotto streamline error handling throughout the codebase.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1268-1268: pkg/agent/core/faiss/service/faiss.go#L1268
Added line #L1268 was not covered by testsapis/docs/v1/docs.md (3)
1674-1676: Flush Service OverviewThe overview succinctly explains that the Flush Service removes all vectors indexed and uncommitted in the
vald-agent. This section is clear and consistent with the other service overviews.
1778-1780: Remove Service OverviewThe overview for the Remove Service is succinct and clearly conveys that it handles the removal of vectors from the
vald-agent. This addition aligns well with the other service overviews.
1940-1942: Upsert Service OverviewThe Upsert Service overview is comprehensive; it clearly explains that the service updates existing vectors or inserts new ones if none exist. The clarity and completeness of this section are commendable.
internal/iter/iter.go (4)
80-94: Verify cancellation in Seq2.
Inside the loop, this method checksctx.Done(). Confirm that coverage includes the scenario where cancellation triggers early. Consider also testing thatyieldreturnsfalse, ensuring the loop stops gracefully.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 81-87: internal/iter/iter.go#L81-L87
Added lines #L81 - L87 were not covered by tests
[warning] 89-91: internal/iter/iter.go#L89-L91
Added lines #L89 - L91 were not covered by tests
116-130: Test Indexes under cancellation and large iteration counts.
Ensure coverage includes verifying that iteration halts ifctx.Done()is signaled, and that largenumvalues correctly produce all indexes before stopping.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 117-123: internal/iter/iter.go#L117-L123
Added lines #L117 - L123 were not covered by tests
[warning] 125-127: internal/iter/iter.go#L125-L127
Added lines #L125 - L127 were not covered by tests
132-143: Expand unit tests for ForEach.
Currently, it may be untested for partial iteration scenarios wherefnreturnsfalse. Add tests confirming that iteration halts immediately in that case.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 132-137: internal/iter/iter.go#L132-L137
Added lines #L132 - L137 were not covered by tests
[warning] 139-141: internal/iter/iter.go#L139-L141
Added lines #L139 - L141 were not covered by tests
70-78:❓ Verification inconclusive
Enhance test coverage for At().
Testing boundary conditions (likei>=num) and the correct usage ofmodFuncis crucial. Ensure that cases for largeivalues or canceled contexts in higher-level calls are properly verified.
🏁 Script executed:
#!/bin/bash # Searching for references to the At method usage rg "At(" -A 3 --type go | rg "_test.go"Length of output: 96
Enhance test coverage for At() by verifying boundary and function behavior.
The current tests (if any) for theAt()method need to be extended to cover the following scenarios:
- Cases where
iexceeds the cycle’s size (i.e., testing the modulus behavior).- Verification that
modFunc, when provided, is correctly applied to the element.- Appropriate handling of large
ivalues and, if applicable, ensuring that higher-level context cancellations are propagated and managed correctly.Note: The initial search for test references using the unescaped regex produced an error. Please re-run the search with the proper regex (e.g., using
rg 'At\(' -A 3 --type go | rg '_test.go') or manually verify if such tests exist in the repository.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 72-77: internal/iter/iter.go#L72-L77
Added lines #L72 - L77 were not covered by tests.gitfiles (9)
126-126: New workflow file reference only.
No code snippet is provided, so there is nothing to review here.
161-177: New documentation files reference only.
No code snippet is provided, so there is nothing to review here.
488-488: Sample YAML reference.
No code snippet is provided, so there is nothing to review here.
512-512: Test file reference.
No code snippet is provided, so there is nothing to review here.
863-863: Test file reference (index_deleter_test.go).
No code snippet is provided, so there is nothing to review here.
1051-1051: New file reference (internal/errors/params.go).
No code snippet is provided, so there is nothing to review here.
1274-1274: Test file reference (conn_test.go).
No code snippet is provided, so there is nothing to review here.
1276-1276: Test file reference (listener_test.go).
No code snippet is provided, so there is nothing to review here.
1407-1410: Vector noise code references.
No snippet is provided for these files, so there is nothing to review here.internal/errors/errors.go (3)
86-88: Nice addition of the unimplemented error helper.This error function follows the established pattern in the file and provides a standardized way to indicate unimplemented features, which improves consistency in error handling throughout the codebase.
170-180: Great addition of theIsAnyfunction.This utility function elegantly implements a common error checking pattern of testing if an error matches any of multiple possible errors. The implementation is clean with proper nil handling and makes good use of the existing
Isfunction.
182-192: Well-designedIsNotfunction complements the error handling APIs.This function provides an intuitive way to check if an error doesn't match any of the specified errors. The implementation is consistent with
IsAnyand handles nil cases appropriately. These additions will make error handling more readable and maintainable throughout the codebase.#!/bin/bash # Find examples where these new error checking functions could be applied rg -A 2 "errors\.Is\(" --type go | grep -v "internal/errors/errors.go" | head -n 15internal/test/mock/server_stream.go (2)
31-37: Improved nil safety in the Send method.The enhanced nil checks make this mock implementation more robust by properly handling edge cases. This prevents potential nil pointer panics that could occur in tests.
Consider adding tests to cover these new edge cases as flagged by the code coverage tool.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 31-35: internal/test/mock/server_stream.go#L31-L35
Added lines #L31 - L35 were not covered by tests
[warning] 37-37: internal/test/mock/server_stream.go#L37
Added line #L37 was not covered by tests
40-52: Enhanced Recv method with better error handling.The updated implementation with named return values and comprehensive nil checks improves robustness. The method properly handles the case when
mis nil and whenRecvFuncis nil, falling back to usingServerStream.RecvMsg.As indicated in previous comments, please ensure these new code paths are covered by tests to verify their behavior.
🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 40-50: internal/test/mock/server_stream.go#L40-L50
Added lines #L40 - L50 were not covered by tests
[warning] 52-52: internal/test/mock/server_stream.go#L52
Added line #L52 was not covered by testsinternal/params/option.go (4)
20-23: Add import statements for file path operations.Clean addition of required imports for the new functionality.
39-45: Good implementation of WithName.Clean implementation that correctly checks for empty name strings before assignment.
89-94: Good implementation of WithOverrideDefault.Simple and clean option function that follows the established pattern.
97-109: Well-implemented WithArgumentFilters function.The implementation correctly handles nil checks and initialization of the filters slice. The documentation comment clearly explains the purpose and behavior of the filters.
internal/sync/errgroup/group_test.go (6)
946-953: Updated test with atomic.Int64 field for concurrency limiting.The test structure has been updated to use the new
atomic.Int64type for the limit field, supporting changes in the implementation.
958-959: Simplified want struct in test code.Removing the
want boolfield from the struct simplifies the test structure, suggesting the boolean return value is no longer part of the tested behavior.
964-965: Updated checkFunc signature.The function signature now takes only the
wantparameter, consistent with the changes to thewantstruct.
968-970: Simplified defaultCheckFunc implementation.The function now simply returns nil without checking boolean values, matching the new test structure.
1045-1046: Added limit field to group initialization.The group struct is now initialized with the limit field, supporting the implementation changes.
1051-1052: Switched from TryGo to exec in test invocation.The test now calls
g.exec()instead ofg.TryGo(), reflecting changes in the implementation.internal/net/grpc/stream_test.go (4)
206-209: Updated test struct to match refactored function signatureThe comment updates to the
argsstruct correctly match the refactoredBidirectionalStreamClientfunction signature, which now includes concurrency control and updated parameter types. These changes align with the implementation changes instream.go.
234-238: Test case placeholder updated to match new parameter namesThis test case placeholder has been properly updated to include the new field names and types, maintaining consistency with the actual implementation.
257-259: Consistent updates to second test case placeholderThe second test case placeholder is consistently updated with the same parameter names and types as the first test case, maintaining uniformity throughout the test file.
290-290: Updated test function call matches implementationThe function call to
BidirectionalStreamClienthas been properly updated to include all required parameters in the right order: stream, concurrency, sendDataProvider, and callBack.pkg/index/job/deletion/service/deleter_test.go (5)
1-105: Good structure for TestNew functionThe test structure for
TestNewfollows best practices with table-driven testing, clear error handling, and proper test isolation with parallel execution. When implemented, consider adding cases that test various option combinations.
107-190: Well-structured Test_delDuplicateAddrs functionThe test structure for address deduplication is well-organized. As suggested in past reviews, remember to implement test cases covering empty slices, no duplicates, and multiple duplicates scenarios.
192-304: Comprehensive structure for Test_index_StartClientThe test structure for the client initialization correctly includes fields and context handling. Consider adding a mock implementation for the discoverer.Client interface as mentioned in the previous reviews when implementing the actual test cases.
306-414: Well-prepared Test_index_Start structureThe test for the Start method includes all necessary fields and error handling. Remember to implement test cases for successful starts, error scenarios, and empty target addresses as suggested in past reviews.
416-527: Complete Test_index_doDeleteIndex structureThe test structure for the deletion operation is thorough. When implementing, include test cases for network errors, context cancellation, and partial success scenarios as suggested in previous reviews.
internal/errors/k8s.go (1)
20-46: Well-organized Kubernetes error definitionsThe grouped error variable declarations follow a consistent pattern and provide clear, descriptive error messages. The use of error-generating functions for formatted messages (
ErrPodIsNotRunningandErrUnsupportedKubernetesResourceType) is an excellent practice that improves diagnostics.Minor note:
ErrUndefinedNamespaceandErrUndefinedServicemessages start with uppercase "Undefined" while other error messages use lowercase. Consider standardizing the capitalization for consistency.internal/net/grpc/stream.go (8)
31-31: Added proto package import for new type constraintsThe addition of the proto package import is necessary for the type constraints introduced in the updated function signatures.
44-53: Well-designed typed stream interfacesThe new
TypedClientStreamandTypedServerStreaminterfaces provide strong type safety by constraining the generic types toproto.Messageand specifying the exact methods expected from each stream type. This makes the API more robust and helps catch type errors at compile time.
80-98: Improved error handling with finalize functionThe new
finalizefunction encapsulates error aggregation and resource cleanup in a more robust way. It properly handles error deduplication, joins errors together, and sets appropriate trace span attributes for better observability.
106-167: Improved stream message handling implementationThe updated implementation includes several improvements:
- Direct use of
stream.Recv()returning the message and error- Better error logging with more context
- Improved trace span handling with unique IDs for each message
- More granular error handling with specific code checks
This implementation is more robust and provides better observability through tracing.
178-180: Added concurrency limit for error groupSetting a limit on the error group when concurrency is specified is a good practice to prevent unbounded goroutine creation, which could lead to resource exhaustion under heavy load.
187-190: Improved error handling with IsNot functionUsing
errors.IsNotfor error checking is more concise and clear than multiple!errors.Ischecks. This approach properly handles expected errors like EOF and context cancellation.
192-196: Better error handling in stream.Recv()The handling of the received message and error is improved:
- Using
errors.IsAnyfor checking multiple expected error types- Properly canceling the context when expected errors occur
- Immediately returning nil to avoid propagating expected errors
This makes the error handling more resilient and prevents unnecessary error propagation.
205-254: Enhanced finalization and error collectionThe improved implementation includes:
- A mutex for thread-safe stream access
- An error channel for collecting errors from goroutines
- A comprehensive finalize function that properly handles error aggregation
- Better error type checking with IsAny/IsNot
- Proper cleanup with context cancellation and stream closure
This implementation ensures resources are properly cleaned up and errors are correctly aggregated.
internal/params/option_test.go (1)
451-705: Implement or remove commented test functionsThese commented test functions (TestWithName, TestWithOverrideDefault, and TestWithArgumentFilters) are templates without actual test cases. Consider either implementing them with real test cases or removing them to avoid code clutter. Having proper tests for these functions would improve code coverage and ensure their correct behavior.
internal/test/data/vector/noise/option_test.go (3)
18-101: Add test coverage for WithLevelFactor.
This placeholder test is still missing concrete assertions and scenarios to confirm that different noise level factors behave as intended.
103-186: Add test coverage for WithTableDivisionFactor.
This placeholder test lacks cases that verify how different table division factors affect noise table sizing. Consider boundary tests for zero and large values.
188-271: Add test coverage for WithMinTableSize.
This placeholder code is missing real scenarios. For robust coverage, test edge cases (e.g., zero, small, and large minimum sizes) to ensure correct sizing logic.internal/test/data/vector/noise/option.go (2)
27-35: Well-structured default options with good documentation.The default options are clearly defined with helpful comments explaining each value's purpose and reasoning.
17-26: Good use of functional options pattern.The implementation follows the functional options pattern, which provides a clean and flexible API for configuring the noise generator.
internal/params/params.go (5)
87-90: Good defensive programming with nil check.Adding a nil check for the parser and its flag set prevents potential panic situations and returns a meaningful error.
109-118: Clean implementation of argument filtering.The use of
slices.DeleteFuncprovides an elegant way to filter unwanted arguments based on the provided filters.
120-126: Improved error handling for help flag.The error handling now correctly uses
errors.Isto check forflag.ErrHelp, which is a more reliable approach than direct comparison.
139-149: Well-implemented state management methods with nil checks.The
RestoreandOverridemethods include appropriate nil checks to prevent potential issues when dealing with flag sets.
29-35: Good API design with type alias and constants.Creating a type alias for
flag.ErrorHandlingand constants for error handling strategies provides a cleaner API and better encapsulation.pkg/agent/core/ngt/service/ngt.go (6)
1824-1825: Enhanced error filtering in CreateAndSaveIndex method.The code now uses
errors.IsNotto ignore specific error types (errors.ErrUncommittedIndexNotFound,context.Canceled,context.DeadlineExceeded) which are considered non-critical in this context. This streamlines error handling by processing only unexpected errors.
1973-1974: Improved error handling in Close method.This change uses
errors.IsNotto filter out context cancellation and deadline errors when closing the KVS, allowing the application to gracefully handle these expected scenarios during shutdown.
1987-1988: Refined error filtering during index creation on shutdown.The use of
errors.IsNotallows the code to selectively ignore specific error types when creating indexes during shutdown. This prevents expected errors likeErrUncommittedIndexNotFoundor context-related errors from being unnecessarily joined with other errors.
1995-1996: Consistent error handling for SaveIndex on shutdown.This change applies the same error filtering approach as the previous changes, maintaining consistency in how errors are handled during the shutdown process. Only unexpected errors will be joined and returned.
1475-1478:❓ Verification inconclusive
Added null check to prevent potential nil pointer dereference.
This addition provides a defensive check against a
nilstatistics object, which could cause a runtime panic when attempting to dereference it later. This improves the robustness of the error handling.Consider adding test coverage for this null check scenario as static analysis indicates these lines are not covered by tests:
🏁 Script executed:
#!/bin/bash # Check if there are tests that verify the nil stats handling rg -A 3 -B 3 "TestNGT_loadStatistics" pkg/agent/core/ngt/service/Length of output: 65
Action Required: Confirm and Add Tests for
nilStats HandlingThe null check added to prevent a nil pointer dereference appears sound:
if stats == nil { log.Warn("failed to load index statistics to cache: stats is nil") return nil }However, our search for tests (using
rgfor "TestNGT_loadStatistics" inpkg/agent/core/ngt/service/) didn’t return any output, which suggests that this scenario might currently lack test coverage. Please manually verify that no tests address thenilstatistics case and consider adding appropriate test coverage to ensure robust error handling.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 1475-1478: pkg/agent/core/ngt/service/ngt.go#L1475-L1478
Added lines #L1475 - L1478 were not covered by tests
929-930:❓ Verification inconclusive
Improved error handling logic with IsNot function.
The code now uses the
errors.IsNotfunction to filter outErrUncommittedIndexNotFounderrors, making the error handling more specific. Only errors that are notErrUncommittedIndexNotFoundwill be sent to the error channel.Consider adding test coverage for this condition as static analysis indicates this line is not covered by tests:
🏁 Script executed:
#!/bin/bash # Check if there are tests for the Start method error handling rg -A 3 -B 3 "func TestNGT_Start" pkg/agent/core/ngt/service/Length of output: 63
Attention: Missing Test Coverage for Start Method Error Handling
The updated logic correctly filters out
ErrUncommittedIndexNotFounderrors using theerrors.IsNotfunction, ensuring that only other errors are sent to the error channel. However, our grep check for any tests covering this condition (specifically a test function likeTestNGT_Startinpkg/agent/core/ngt/service/) produced no results. This suggests that the error handling branch may not have adequate automated test coverage.
- Location:
pkg/agent/core/ngt/service/ngt.go, lines 929-930- Suggestion: Add test cases to verify that when an
ErrUncommittedIndexNotFounderror occurs, it is correctly filtered out, and that all other error conditions are appropriately sent to the error channel.🧰 Tools
🪛 GitHub Check: codecov/patch
[warning] 929-929: pkg/agent/core/ngt/service/ngt.go#L929
Added line #L929 was not covered by tests
…2904) Signed-off-by: kpango <[email protected]>
…2904) (#2908) Signed-off-by: kpango <[email protected]> Co-authored-by: Yusuke Kato <[email protected]> Co-authored-by: Kiichiro YUKAWA <[email protected]>
4 participants
Overview
This PR introduces a new V2 E2E strategic testing framework for Vald, with significant refactoring and improvements to the testing infrastructure.
Key Changes
New V2 E2E Testing Framework:
tests/v2/e2e/Internal Package Refactoring:
internal/iterpackage for iterator abstractionsinternal/errorsinternal/paramsfor better argument parsinginternal/sync/errgroupfor improved concurrency handlingConfiguration Changes:
internal/config/config.goin theBind()methodconfigpackage for E2E testingCI/Workflow Improvements:
This pull request includes several updates across various files and configurations, primarily focusing on adding new test files, updating version numbers, and enhancing CI/CD workflows.
About the CI Warning
The CI system flagged changes in
internal/configthat might require Helm chart updates. After reviewing the changes, they appear to be relatively minor and focused on refactoring theBind()method rather than changing the structure of configuration itself. No Helm chart changes were included in this PR.Recommendations
Overall, this PR significantly improves the testing infrastructure and should enhance the maintainability of the codebase.
Related Issue
Versions
Checklist
Special notes for your reviewer
Summary by CodeRabbit