Merge branch 2025-04 to main for release
#112
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Problem `2025-04` has been released and needs to be incorporated into the Go client. ## Solution - Bump `justfile` -> `api_version := "2025-04", run `just gen`. - Tweak inline declarations to get things building again. ## Type of Change - [X] New feature (non-breaking change which adds functionality) ## Test Plan Functionally nothing has changed - make sure CI unit & integration tests are passing. I've run them locally.
## Problem
Inference Models and Namespaces are new API resources available in
`2025-04`. They need to be implemented in the Go client.
Additionally, there are a few client bugs which need to be fixed:
- The `Embed` method and it's return type `EmbedResponse` need to be
refactored to support both sparse and dense embedding responses, rather
than just dense. Currently, embedding with a model that returns sparse
values will cause errors.
- The `IndexConnection` struct is not safe to reuse for performing
operations across namespaces while reusing the existing gRPC connection
for the index. This is because `IndexConnection.Namespace` is publicly
exposed, and could be updated at any point.
## Solution
Implement new namespaces and models API operations:
- Namespace operations have been implemented on `IndexConnection`.
They're exposed as methods which you can call via
`IndexConnection.ListNamespaces`, `IndexConnection.DescribeNamespace`,
or `IndexConnection.DeleteNamespace`.
- Hosted model operations can be performed using the `Client.Inference`
namespace (`InferenceService` struct) by calling
`client.Inference.DescribeModel` or `client.Inference.ListModels`.
The `InferenceService.Embed` method now returns a different `Embedding`
inside of `EmbedResponse`. `Embedding` has been refactored into a tagged
union type which is basically just a struct that wraps either a
`SparseEmbedding` or `DenseEmbedding` pointers. This seemed to be the
best way to manage this type of thing in Go given the lack of explicit
union types. I'm open to suggestions if anyone has something that may be
a bit more ergonomic.
`IndexConnection` has been refactored to no longer expose
`IndexConnection.Namespace` directly. Instead, `Namespace` is now a
method which allows checking the currently targeted namespace. Users can
now call `IndexConnection.WithNamespace` which will return a copy of the
`IndexConnection` targeting the new namespace, but sharing the
underlying gRPC connection. Again, I think this is a reasonable way of
approaching this and allowing safely targeting multiple namespaces
within an index, but I'm open to feedback.
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- Breaking change for `EmbedResponse` on `InferenceService.Embed`
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)
## Test Plan
CI - unit tests & integration tests
If you'd like to test the new APIs out yourself you can check the
integration tests or README for more detailed examples:
Models:
```go
ctx := context.Background()
pc, err := pinecone.NewClient(pinecone.NewClientParams{
ApiKey: "YOUR_API_KEY",
})
if err != nil {
log.Fatalf("Failed to create Client: %v", err)
}
embed := "embed"
rerank := "rerank"
embedModels, err := pc.Inference.ListModels(ctx, &pinecone.ListModelsParams{
Type: &embed,
})
if err != nil {
log.Fatalf("Failed to list embedding models: %v", err)
}
rerankModels, err := pc.Inference.ListModels(ctx, &pinecone.ListModelsParams{
Type: &rerank,
})
if err != nil {
log.Fatalf("Failed to list reranking models: %v", err)
}
multilingualModel, := pc.Inference.DescribeModel(ctx, "multilingual-e5-large")
if err != nil {
log.Fatalf("Failed to describe models: %v", err)
}
```
Namespaces:
```go
ctx := context.Background()
pc, err := pinecone.NewClient(pinecone.NewClientParams{
ApiKey: "YOUR_API_KEY",
})
if err != nil {
log.Fatalf("Failed to create Client: %v", err)
}
idx, err := pc.DescribeIndex(ctx, "example-index")
if err != nil {
log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
}
idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
if err != nil {
log.Fatalf("Failed to create IndexConnection for Host: %v: %v", idx.Host, err)
}
// list namespaces
limit := uint32(10)
namespaces, err := idxConnection.ListNamespaces(ctx, &pinecone.ListNamespacesParams{
Limit: &limit,
})
if err != nil {
log.Fatalf("Failed to list namespaces for Host: %v: %v", idx.Host, err)
}
// describe a namespace
namespace1, err := idxConnection.DescribeNamespace(ctx, "my-namespace-1")
if err != nil {
log.Fatalf("Failed to describe namespace: %v: %v", "my-namespace-1", err)
}
// delete a namespace
err := idxConnection.DeleteNamespace("my-namespace-1")
if err != nil {
log.Fatalf("Failed to delete namespace: %v: %v", "my-namespace-1", err)
}
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210238631289243
- https://app.asana.com/0/0/1209828518477630
## Problem
Backups/Restore are a part of the `2025-04` spec and need to be
implemented in the Go client.
## Solution
Add new types for working with `Backups` and `RestoreJob` methods on
`Client`:
- `Backup`
- `BackupList`
- `RestoreJob`
- `RestoreJobList`
- `CreateBackupParams`
- `CreateIndexFromBackupParams`
- `CreateIndexFromBackupResponse`
- `ListBackupsParams`
- `ListRestoreJobsParams`
Add new methods for working with backups as a part of `Client`:
- `CreateBackup`
- `CreateIndexFromBackup`
- `DescribeBackup`
- `DeleteBackup`
- `DescribeRestoreJob`
- `ListBackups`
- `ListRestoreJobs`
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)
## Test Plan
CI - unit & integration tests
New integration tests have been added to validate the create backup ->
list/describe backups -> create index from backup -> list/describe
restore jobs flow.
You can also test using the `README` or the integration tests themselves
as example code:
```go
package main
import (
"context"
"fmt"
"log"
"os"
"time"
"github.com/pinecone-io/go-pinecone/v3/pinecone"
)
ctx := context.Background()
clientParams := pinecone.NewClientParams{
ApiKey: os.Getenv("PINECONE_API_KEY"),
}
pc, err := pinecone.NewClient(clientParams)
if err != nil {
log.Fatalf("Failed to create Client: %w", err)
}
indexName := "my-index"
backupName := fmt.Sprintf("backup-%s", )
backupDesc := fmt.Sprintf("Backup created for index %s", indexName)
fmt.Printf("Creating backup: %s for index: %s\n", backupName, indexName)
backup, err := pc.CreateBackup(ctx, &pinecone.CreateBackupParams{
IndexName: indexName,
Name: &backupName,
Description: &backupDesc,
})
if err != nil {
log.Fatalf("Failed to create backup: %w", err)
}
backup, err = pc.DescribeBackup(ctx, backup.BackupId)
if err != nil {
log.Fatalf("Failed to describe backup: %w", err)
}
// wait for backup to be "Complete" before triggering a restore job
log.Printf("Backup status: %v", backup.Status)
limit := 10
backups, err := pc.ListBackups(ctx, &pinecone.ListBackupsParams{
Limit: &limit,
IndexName: &indexName,
})
if err != nil {
log.Fatalf("Failed to list backups: %w", err)
}
// create a new serverless index from the backup
restoredIndexName := indexName + "-from-backup"
restoredIndexTags := IndexTags{"restored_on": time.Now().Format("2006-01-02 15:04")}
createIndexFromBackupResp, err := pc.CreateIndexFromBackup(context.Background(), &CreateIndexFromBackupParams{
BackupId: ts.backupId,
Name: restoredIndexName,
Tags: &restoredIndexTags,
})
// check the status of the index restoration
restoreJob, err := pc.DescribeRestoreJob(ctx, restoreJob.RestoreJobId)
if err != nil {
log.Fatalf("Failed to describe restore job: %w", err)
}
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1209571416750587
## Problem When releasing a new major version of a Go package, you need to update the `go.mod` file and associated imports. Here are examples of where this was done previously: - #91 - #97 ## Solution Update for `v4`. ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [X] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) ## Test Plan CI - unit + integration
There was a problem hiding this comment.
Pull Request Overview
Merge the 2025-04 release branch into main, introducing v4 module, namespace handling, and backup support.
- Bumped module path and API version to v4.
- Added first-class namespace APIs (
WithNamespace,List/Describe/DeleteNamespace) and multi-namespace test support. - Introduced backup creation, listing, and restore operations with tests and docs updates.
Reviewed Changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pinecone/test_suite.go | Added multi-namespace upsert loops and backup setup/teardown |
| pinecone/index_connection.go | Renamed Namespace field, added Namespace() accessor, WithNamespace, and namespace RPC methods |
| pinecone/index_connection_test.go | Updated tests to use unexported namespace field and added namespace tests |
| pinecone/models.go | Added NamespaceDescription, ListNamespacesResponse, backup & restore types, and JSON unmarshal for parameter values |
| README.md | Documented namespace and backup usage with code examples |
| internal/gen/api_version.go | Bumped PineconeApiVersion to 2025-04 |
| internal/useragent/useragent.go | Updated import path for v4 |
| go.mod | Updated module to v4 and bumped dependencies |
Comments suppressed due to low confidence (2)
README.md:675
- The example call to
DeleteNamespaceis missing the requiredctxparameter. It should beidxConnection.DeleteNamespace(ctx, "my-namespace-1").
err := idxConnection.DeleteNamespace("my-namespace-1")
README.md:1650
- The
fmt.Sprintfcall is missing an argument for%s. Ensure you pass a unique identifier, for exampleuuid.New().String().
backupName := fmt.Sprintf("backup-%s", )
Dependabot opened a PR for bumping the `golang.org/x/net` dependency from 0.33.0 -> 0.38.0. However, in order to make that jump, we'd need to bump the Go version our module supports from go 1.21 -> go 1.24. For consumers, this would be a breaking change and they'd need to upgrade their version of Go to use the SDK. At the moment, I don't think is necessary for this release. For now I've updated to `golang.org/x/net v0.35.0` which is the last version which supports < `go v1.23`: - https://cs.opensource.google/go/x/net/+/refs/tags/v0.35.0:go.mod - https://cs.opensource.google/go/x/net/+/refs/tags/v0.36.0:go.mod Bump `golang.org/x/net` -> `v0.35.0`. - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [X] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) CI
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Features and changes for the next major release have been staged in the
2025-04branch. Development is complete, and we need to merge all of the relevant PRs over to main in order to release the client.Solution
Merge branch
2025-04intomain. This branch includes a set of PRs which have already been tested / reviewed individually:2025-04#106StartImportfunction #108go.modto support v4.0.0 major version update #110golang.org/x/[email protected]#111Type of Change
Test Plan
Testing has been done against each PR contained in the branch individually. CI unit and integration tests will run as a part of this PR as well.