feat: add usage accounting support to OpenRouter

[ImBIOS]

Close #56

• Updated README.md to include usage accounting use cases and examples.
• Enhanced OpenRouterChatLanguageModel to support usage accounting settings and metadata extraction.
• Implemented detailed usage information tracking in responses.
• Added tests for usage accounting functionality, ensuring correct metadata inclusion based on settings.

Tested

• Verified that usage accounting data is included in responses when enabled.
• Confirmed that provider-specific metadata is omitted when usage accounting is disabled.

Added

• Added support for OpenRouter usage accounting feature
  • Allows tracking token usage details directly in API responses
  • Includes cost, cached tokens, and reasoning tokens information
  • Available in both streaming and non-streaming APIs

Changed

• Updated schema to support OpenRouter usage accounting response format

[ImBIOS]

I probably need to create a PR to vercel/ai too to make us able to pass the detailed usage to the user.

UPDATE:
Nope, these changes are enough to achieve the goal.

[ImBIOS]

If you guys want to test this feature before it's officially merged, replace your deps to be like this:

"@openrouter/ai-sdk-provider": "github:ImBIOS/OpenRouterTeam-ai-sdk-provider#feat/usage-accounting"

When using a Git repository as a dependency with pnpm, you'll need to regularly update to get the latest changes from the branch. Here are your options:

1. Use pnpm update for specific updates:

pnpm update "@openrouter/ai-sdk-provider"

2. For CI environments, clear cache before install:

pnpm store prune
pnpm install

3. Force reinstall with this command:

pnpm install --force

Note that for production use, it's better to use specific commits or version tags rather than branches to ensure reproducible builds.

[ImBIOS]

Tested locally

[ImBIOS]

This particular guide/comment/document/tutorial is generated by claude-3.7-sonnet-thinking; it might not be syntactically correct, and I don't see the need to invest time to review it, but the goal is achieved, so that's what's important. PLEASE READ THIS PARTICULAR COMMENT WITH A GRAIN OF SALT!

---

OpenRouter Usage Accounting

This document explains how to use the OpenRouter usage accounting feature with the AI SDK provider. This feature allows you to track token usage details directly in your API responses, without making additional API calls.

Enabling Usage Accounting

You can enable usage accounting at different levels:

1. When creating the provider instance

import { createOpenRouter } from '@openrouter/ai-sdk-provider';

const openrouter = createOpenRouter({
  apiKey: 'your-api-key',
  usage: { include: true }, // Enable usage accounting for all models
});

2. For a specific model

import { createOpenRouter } from '@openrouter/ai-sdk-provider';

const openrouter = createOpenRouter({
  apiKey: 'your-api-key',
});

// Enable usage accounting just for this model
const model = openrouter.chat('anthropic/claude-3-sonnet', {
  usage: { include: true },
});

3. For a specific request using providerOptions

import { createOpenRouter } from '@openrouter/ai-sdk-provider';
import { generateText } from 'ai';

const openrouter = createOpenRouter({
  apiKey: 'your-api-key',
});

const model = openrouter('anthropic/claude-3-sonnet');

const result = await generateText({
  model,
  prompt: 'Hello, how are you?',
  providerOptions: {
    openrouter: {
      usage: { include: true },
    },
  },
});

Accessing Usage Information

When usage accounting is enabled, the provider will include detailed usage information in the response.

For non-streaming responses

const response = await model.doGenerate({
  inputFormat: 'messages',
  mode: { type: 'regular' },
  prompt: [
    { role: 'user', content: [{ type: 'text', text: 'Hello!' }] },
  ],
});

// Basic usage information (available with or without usage accounting)
console.log('Prompt tokens:', response.usage.promptTokens);
console.log('Completion tokens:', response.usage.completionTokens);

// Detailed usage information (only with usage accounting enabled)
if (response.providerMetadata?.openrouter?.usage) {
  const usage = response.providerMetadata.openrouter.usage;

  console.log('Total tokens:', usage.totalTokens);
  console.log('Cost in USD:', usage.cost);

  if (usage.promptTokensDetails) {
    console.log('Cached tokens:', usage.promptTokensDetails.cachedTokens);
  }

  if (usage.completionTokensDetails) {
    console.log('Reasoning tokens:', usage.completionTokensDetails.reasoningTokens);
  }
}

For streaming responses

When streaming, the usage information is included in the finish event:

const { stream } = await model.doStream({
  inputFormat: 'messages',
  mode: { type: 'regular' },
  prompt: [
    { role: 'user', content: [{ type: 'text', text: 'Hello!' }] },
  ],
});

for await (const chunk of stream) {
  if (chunk.type === 'finish') {
    // Basic usage information
    console.log('Prompt tokens:', chunk.usage.promptTokens);
    console.log('Completion tokens:', chunk.usage.completionTokens);

    // Detailed usage information
    if (chunk.providerMetadata?.openrouter?.usage) {
      const usage = chunk.providerMetadata.openrouter.usage;

      console.log('Total tokens:', usage.totalTokens);
      console.log('Cost in USD:', usage.cost);

      if (usage.promptTokensDetails) {
        console.log('Cached tokens:', usage.promptTokensDetails.cachedTokens);
      }

      if (usage.completionTokensDetails) {
        console.log('Reasoning tokens:', usage.completionTokensDetails.reasoningTokens);
      }
    }
  }
}

Usage Information Fields

The following fields are available in the usage information:

Field	Description
totalTokens	Total number of tokens used (prompt + completion)
promptTokens	Number of tokens in the prompt
completionTokens	Number of tokens in the completion
cost	Cost of the request in USD
promptTokensDetails.cachedTokens	Number of tokens that were cached
completionTokensDetails.reasoningTokens	Number of tokens used for reasoning

Benefits of Usage Accounting

• Real-time cost tracking: Monitor the cost of each request directly.
• Cached tokens insights: See how many tokens were retrieved from cache.
• Reasoning tokens: View the number of tokens used for reasoning steps.
• No additional API calls: Get usage information directly in the response.

Performance Considerations

Enabling usage accounting does not impact performance significantly, as OpenRouter already tracks this information internally. The only difference is that it's included in the API response.

Further Resources

For more information about OpenRouter usage accounting, visit the OpenRouter documentation.

[ImBIOS]

To the maintainer, feel free to take over or edit, or merge this PR, since I might not be asynchronously responsive due to priority.

[louisgv]

Thanks for the PR @ImBIOS :)

[abromberg]

Thanks for putting this together @ImBIOS — I was looking for a solution to this!

However: I am having a tough time getting this PR to work on streaming responses. I'm using Vercel AI SDK's streamText. On non-streaming, this works well with generateObject and I get the full usage data. But I can't seem to get streamText or streamObject (which I believe are the normal ways to access streaming responses) to output the full usage data. Not sure if you have any ideas on this, or perhaps I'm doing something wrong.

[ImBIOS]

Thanks for putting this together @ImBIOS — I was looking for a solution to this!

However: I am having a tough time getting this PR to work on streaming responses. I'm using Vercel AI SDK's streamText. On non-streaming, this works well with generateObject and I get the full usage data. But I can't seem to get streamText or streamObject (which I believe are the normal ways to access streaming responses) to output the full usage data. Not sure if you have any ideas on this, or perhaps I'm doing something wrong.

@abromberg try this

onFinish: async ({ response, providerMetadata, usage }) => {
            logUsageAccounting({
              providerMetadata,
              usage,
            });

/**
 * Logs the usage accounting information for the given provider metadata and usage.
 * @param options - The options.
 * @param options.providerMetadata - The provider metadata.
 * @param options.usage - The usage.
 */
export const logUsageAccounting = ({
  providerMetadata,
  usage,
}: {
  providerMetadata: ProviderMetadata | undefined;
  usage: LanguageModelUsage;
}) => {
  // Basic usage information
  console.log('Prompt tokens:', usage.promptTokens);
  console.log('Completion tokens:', usage.completionTokens);

  // Detailed usage information
  if (providerMetadata?.openrouter?.usage) {
    const usage = providerMetadata.openrouter
      .usage as OpenRouterUsageAccounting;

    console.log('Total tokens:', usage.totalTokens);
    console.log('Cost in USD:', usage.cost);

    if (usage.promptTokensDetails) {
      console.log('Cached tokens:', usage.promptTokensDetails.cachedTokens);
    }

    if (usage.completionTokensDetails) {
      console.log(
        'Reasoning tokens:',
        usage.completionTokensDetails.reasoningTokens,
      );
    }
  } else {
    console.log('No usage information available');
  }
};

[abromberg]

@ImBIOS that code you just shared is working for you on streamText? It does not seem to be for me. Still just yields the basic usage accounting. If it's working for you I will continue to try to debug...

const result = streamText({
    model: modelInstance,
    system: systemPrompt,
    messages,
    tools,
    onError: err => {
      console.error("streamText error:", err);
    },
    async onFinish({ response, providerMetadata, usage }) {
      try {
        console.log("providerMetadata", providerMetadata);
        console.log("usage", usage);
        await saveFinalAssistantMessage(
          supabase,
          chatId,
          response.messages,
          modelId,
          selectedUserSystemPromptId,
          providerMetadata?.openrouter?.usage || usage
        );

[louisgv]

BTW will remove CHANGELOG in favor of using github release: https://github.com/OpenRouterTeam/ai-sdk-provider/releases

It automatically tracks the PRs will be closer to the npm deployment.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	dotenv@​16.5.0
	vite-tsconfig-paths@​5.1.4
	@​biomejs/​biome@​1.9.4

View full report

[louisgv]

@abromberg I added an e2e test to the repo -- it works, you just need to add the flag:

  const model = openrouter('anthropic/claude-3.7-sonnet:thinking', {
    usage: {
      include: true,
    },
  });

[louisgv]

Cooked the PR to add e2e tests + biome