feat: add gemini vertex api support #21
Conversation
|
Caution Review failedThe pull request is closed. WalkthroughThe pull request introduces support for the Google Cloud Vertex AI API. New functions and type definitions have been added to convert JSON schemas and classes into Vertex AI tool and response schema formats. Documentation in the CHANGELOG and README has been updated to reflect these changes, and the package version has been incremented. Existing export lists have been enhanced by integrating the new functions alongside legacy functionality. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Dev as Developer
participant SF as @firefliesai/schema-forge
participant LLM as llm-formats Module
Dev->>SF: call classToGeminiVertexTool(UserInput, options)
SF->>LLM: Delegate conversion from class to tool format
LLM-->>SF: Return GeminiVertexToolFunction instance
SF-->>Dev: Return tool instance
sequenceDiagram
autonumber
participant Dev as Developer
participant SF as @firefliesai/schema-forge
participant JSON as JSON Converter
Dev->>SF: call jsonSchemaToGeminiVertexTool(schema, metadata)
SF->>JSON: Convert JSON schema to tool format
JSON-->>SF: Return conversion result
SF-->>Dev: Return GeminiVertexToolFunction instance
Possibly related PRs
Suggested reviewers
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (1)
src/lib/index.ts (1)
75-112: 💡 Verification agent🧩 Analysis chain
Consider updating Schema namespace with new exports
The
Schemanamespace at the bottom of the file hasn't been updated to include the newly added functions. This could lead to inconsistency in the API for users who use the namespace-style imports.export const Schema = { // Decorators ToolMeta, ToolProp, // Core JSON Schema generation classToJsonSchema, prepareForOpenAIStructuredOutput, // Schema modification updateSchemaProperty, addSchemaProperty, // LLM-specific class converter functions classToOpenAITool, classToOpenAIResponseFormatJsonSchema, classToOpenAIResponseApiTool, classToOpenAIResponseApiTextSchema, classToGeminiTool, classToGeminiResponseSchema, + classToGeminiVertexTool, + classToGeminiVertexResponseSchema, + classToGeminiOldTool, + classToGeminiOldResponseSchema, classToAnthropicTool, // Direct JSON Schema converter functions jsonSchemaToOpenAITool, jsonSchemaToOpenAIResponseFormat, jsonSchemaToOpenAIResponseApiTool, jsonSchemaToOpenAIResponseApiTextSchema, jsonSchemaToGeminiTool, jsonSchemaToGeminiResponseSchema, + jsonSchemaToGeminiVertexTool, + jsonSchemaToGeminiVertexResponseSchema, + jsonSchemaToGeminiOldTool, + jsonSchemaToGeminiOldResponseSchema, jsonSchemaToAnthropicTool, // Extract JSON Schema from LLM formats openAIToolToJsonSchema, openAIResponseApiToolToJsonSchema, // Legacy name classToLLMTool: classToOpenAITool, } as const;To ensure this is implemented correctly, we would also need to import these functions in the imports section around line 56-72.
🏁 Script executed:
#!/bin/bash # Check if classToGeminiVertexTool and other new functions are defined in the llm-formats.ts file echo "Searching for implementation of new Vertex API functions..." rg -A 1 "export (const|function) (classToGeminiVertex|jsonSchemaToGeminiVertex|classToGeminiOld|jsonSchemaToGeminiOld)" src/lib/llm-formats.tsLength of output: 1018
Action Required: Extend Schema Namespace with New Vertex API Exports
The new functions (e.g.
classToGeminiVertexTool,classToGeminiVertexResponseSchema,classToGeminiOldTool,classToGeminiOldResponseSchema, and their JSON Schema converter counterparts) have been verified to exist insrc/lib/llm-formats.ts. However, theSchemanamespace insrc/lib/index.tshasn’t been updated to expose these exports.Please update the
Schemaobject as follows and ensure that the corresponding import statements (around lines 56–72 insrc/lib/index.ts) are also updated to import these new functions:
- Add the following to the LLM-specific class converter functions section:
classToGeminiVertexToolclassToGeminiVertexResponseSchemaclassToGeminiOldToolclassToGeminiOldResponseSchema- Add the following to the direct JSON Schema converter functions section:
jsonSchemaToGeminiVertexTooljsonSchemaToGeminiVertexResponseSchemajsonSchemaToGeminiOldTooljsonSchemaToGeminiOldResponseSchemaDiff Suggestion:
export const Schema = { // Decorators ToolMeta, ToolProp, // Core JSON Schema generation classToJsonSchema, prepareForOpenAIStructuredOutput, // Schema modification updateSchemaProperty, addSchemaProperty, // LLM-specific class converter functions classToOpenAITool, classToOpenAIResponseFormatJsonSchema, classToOpenAIResponseApiTool, classToOpenAIResponseApiTextSchema, classToGeminiTool, classToGeminiResponseSchema, + classToGeminiVertexTool, + classToGeminiVertexResponseSchema, + classToGeminiOldTool, + classToGeminiOldResponseSchema, classToAnthropicTool, // Direct JSON Schema converter functions jsonSchemaToOpenAITool, jsonSchemaToOpenAIResponseFormat, jsonSchemaToOpenAIResponseApiTool, jsonSchemaToOpenAIResponseApiTextSchema, jsonSchemaToGeminiTool, jsonSchemaToGeminiResponseSchema, + jsonSchemaToGeminiVertexTool, + jsonSchemaToGeminiVertexResponseSchema, + jsonSchemaToGeminiOldTool, + jsonSchemaToGeminiOldResponseSchema, jsonSchemaToAnthropicTool, // Extract JSON Schema from LLM formats openAIToolToJsonSchema, openAIResponseApiToolToJsonSchema, // Legacy name classToLLMTool: classToOpenAITool, } as const;After updating the export object, please double-check that the new functions are being imported correctly from
src/lib/llm-formats.ts. This will ensure consistency in the API for namespace-style imports.
🧹 Nitpick comments (3)
src/lib/vertex-api.type.ts (1)
1-4: Consider adding more documentation for the SchemaType enumThis enum appears to be used for Vertex AI schema type definitions. Consider adding:
- JSDoc comments to explain the purpose of this enum
- Documentation about whether more values might be added in the future
- Why only
OBJECTis defined currently// @google-cloud/vertexai +/** + * Schema types supported by the Google Cloud Vertex AI API. + * Currently only OBJECT schema type is supported. + */ export enum SchemaType { OBJECT = 'OBJECT', }CHANGELOG.md (1)
8-15: Fix capitalization of "Google" in changelogThe company name "Google" should be capitalized.
## [1.0.2] - 2025-03-25 -1. Publish `jsonSchemaToGeminiOldTool`, `jsonSchemaToGeminiOldResponseSchema` that whould be published in 1.0.0. -2. Support the tool format and responseSchema format of google vertex API, `@google-cloud/vertexai`. Supply the following functions +1. Publish `jsonSchemaToGeminiOldTool`, `jsonSchemaToGeminiOldResponseSchema` that should be published in 1.0.0. +2. Support the tool format and responseSchema format of Google Vertex API, `@google-cloud/vertexai`. Supply the following functions 1. `jsonSchemaToGeminiVertexTool` 2. `jsonSchemaToGeminiVertexResponseSchema` 3. `classToGeminiVertexResponseSchema` 4. `classToGeminiVertexTool`Also note there's a typo: "whould" should be "should".
🧰 Tools
🪛 LanguageTool
[grammar] ~11-~11: “Google” is a proper noun and needs to be capitalized.
Context: ...ool format and responseSchema format of google vertex API,@google-cloud/vertexai. S...(A_GOOGLE)
src/lib/types.ts (1)
132-145: Fix typo in comment.There's a typo in the comment documentation.
// json schema like structure, - // Google Gemini (@google/genai) prefers capital 'OBJECT' insteadac of 'object' + // Google Gemini (@google/genai) prefers capital 'OBJECT' instead of 'object' // but 'object' is also accepted by Gemini API if we use any to force triggering api
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
CHANGELOG.md(1 hunks)README.md(4 hunks)package.json(2 hunks)src/lib/index.ts(1 hunks)src/lib/llm-formats.ts(5 hunks)src/lib/types.ts(3 hunks)src/lib/vertex-api.type.ts(1 hunks)
🧰 Additional context used
🧬 Code Definitions (1)
src/lib/llm-formats.ts (1)
src/lib/types.ts (5)
JSONSchemaDefinition(54-60)GeminiVertexToolFunction(133-145)GeminiVertexResponseSchema(165-170)GeminiToolOptions(222-225)GeminiResponseSchemaOptions(231-234)
🪛 LanguageTool
CHANGELOG.md
[grammar] ~11-~11: “Google” is a proper noun and needs to be capitalized.
Context: ...ool format and responseSchema format of google vertex API, @google-cloud/vertexai. S...
(A_GOOGLE)
🔇 Additional comments (21)
package.json (1)
3-3: Version bump is appropriate for feature additionThe version increment from 1.0.1 to 1.0.2 follows semantic versioning principles for adding new features without breaking changes.
src/lib/index.ts (3)
23-24: New Vertex API class converters added appropriatelyThe Vertex API-specific class converters are correctly exported from the module.
31-32: Legacy Gemini converters exported correctlyThe "Old" Gemini converters that should have been in v1.0.0 are now properly exported.
35-36: New Vertex API JSON Schema converters added correctlyThe direct JSON Schema converters for Vertex API are properly exported.
src/lib/types.ts (3)
4-4: Looking good! Appropriate import and type aliasing.The import and renaming of
SchemaTypetoSchemaVertexTypecreates a clear distinction for Vertex API types.
132-145: Good interface definition that maintains consistency with existing patterns.The
GeminiVertexToolFunctioninterface follows the same structure as the other Gemini tool interfaces, which maintains consistency across the codebase. UsingSchemaVertexType.OBJECThelps distinguish this implementation for Vertex AI.
164-170: Well-structured response schema interface.The
GeminiVertexResponseSchemainterface appropriately mirrors the pattern of other response schema interfaces while using the Vertex-specific type.src/lib/llm-formats.ts (5)
14-15: Good import of new types.Properly importing the new Vertex-specific types to be used in the new functions.
285-294: Consistent implementation pattern for response schema conversion.The implementation follows the same pattern as the tool function conversion, leveraging existing code with type casting.
559-565: Similar implementation pattern for class-based conversion.The approach of leveraging existing functions with type casting is maintained consistently across all Vertex AI conversion functions.
675-681: Consistent implementation completing the set of conversion functions.The implementation completes the set of conversion functions needed for Vertex AI support, following the same pattern as the other functions.
236-242:✅ Verification successful
Implementation leverages existing code with type casting.
The implementation reuses the existing
jsonSchemaToGeminiToolfunction, avoiding code duplication while maintaining API-specific types through casting. This approach is efficient if the underlying formats are indeed identical.Since the implementation assumes structural equivalence between Gemini and Vertex formats, verify this assumption remains valid if either API changes:
🏁 Script executed:
#!/bin/bash # Check if there are any structural differences in type definitions grep -r "export interface Gemini.*ToolFunction" src/ grep -r "export interface Gemini.*ResponseSchema" src/Length of output: 465
Verified Structural Equivalence between Gemini and Vertex APIs
The implementation in
src/lib/llm-formats.tsefficiently leverages the existingjsonSchemaToGeminiToolfunction by casting its output to aGeminiVertexToolFunction. A review of type definitions insrc/lib/types.tsconfirms that the expected interfaces (includingGeminiVertexToolFunction,GeminiOldToolFunction, and their corresponding response interfaces) are present and consistent. This suggests that the assumption of structural equivalence between Gemini and Vertex formats currently holds. However, please remain alert to any future changes in these API definitions that might affect this equivalence.README.md (9)
290-290: Clear import statement including new Vertex tool function.The import statement now includes
classToGeminiVertexToolalongside the other Gemini conversion functions.
293-293: Good variable name clarification.Renaming from
tooltogeminiToolimproves clarity, especially as multiple tool variables are now defined in the example.
298-299: Proper formatting for tool configuration.The example correctly shows how to use the tool function in the configuration object.
303-304: Clear identification of API client.The comment properly identifies which Google API client is being used for this example (
@google/generative-ai).
310-316: Good example for Vertex AI implementation.The example demonstrates how to use the new Vertex AI support with the appropriate API client.
689-690: Transparent documentation with appropriate cautionary note.The documentation clearly indicates that while the Vertex API functionality is implemented, it hasn't been verified with real API calls yet. This transparency is important for users.
703-704: Consistent documentation for JSON Schema converters.The documentation for the JSON Schema converters is consistent with the class-based converters, including the same cautionary note about verification.
838-840: Important implementation notes for users.The notes provide important context about the implementation status and potential future API changes, which is valuable information for users deciding which API to use.
845-845: Example code matches API documentation.The example code correctly uses the
classToGeminiVertexToolfunction as described in the API reference.
What does this PR do?
Type of change
This pull request is a
Which introduces
How should this be manually tested?
xxx
What are the requirements to deploy to production?
Any background context you want to provide beyond Shortcut?
xxx
Screenshots (if appropriate)
xxx
Loom Video (if appropriate)
xxx
Any Security implications
xxx
Summary by CodeRabbit
New Features
Documentation
Chores