Description
Description
I'm using the chatStream
function from the SDK (v2), and I’d like to access the response headers from the streaming API call—particularly the headers related to rate limiting, such as:
x-endpoint-monthly-call-limit: 1000
x-trial-endpoint-call-limit: 40
x-trial-endpoint-call-remaining: 39
This information is valuable in programmatically managing API usage and staying within rate limits.
Proposed Solution
Could the SDK expose the original Response
object or the relevant headers alongside the streamed chat events returned by chatStream
? For example, a wrapper like:
const stream = await cohere.chatStream({ ... });
console.log(stream.headers.get("x-trial-endpoint-call-remaining")); // hypothetical
for await (const chat of stream) {
...
}
Alternatively, any method to retrieve the headers right after initiating the stream would also work great.
Why this matters
- Helps prevent rate-limit issues proactively.
- Enables better observability and monitoring.
- Allows developers to build smarter clients that react to quota thresholds.
Notes
I understand this SDK is generated with Fern. If changes need to happen on the generator side, I'm happy to start the discussion here or file a related request elsewhere as advised.
Thank you for this awesome SDK!