Skip to content

chore: bump grpc limits to 50MB #5212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 19, 2025
Merged

chore: bump grpc limits to 50MB #5212

merged 1 commit into from
Apr 19, 2025

Conversation

mudler
Copy link
Owner

@mudler mudler commented Apr 18, 2025

Description

This pull request introduces changes to ensure consistent gRPC message size limits across various backends and client implementations. The updates set maximum message sizes to 50MB for sending, receiving, and overall message length. These changes improve the handling of large payloads and prevent errors caused by exceeding default gRPC limits.

Backend Updates

  • C++ Backend (grpc-server.cpp): Added SetMaxMessageSize, SetMaxSendMessageSize, and SetMaxReceiveMessageSize to set the maximum gRPC message size to 50MB.
  • Python Backends:
    • Updated serve functions in multiple files (e.g., autogptq, bark, coqui, diffusers, exllama2, faster-whisper, kokoro, rerankers, transformers, vllm) to include gRPC options for max_message_length, max_send_message_length, and max_receive_message_length, all set to 50MB. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Client Updates

  • Go Client (client.go):
    • Updated multiple methods (e.g., HealthCheck, Embeddings, Predict, LoadModel, GenerateImage, TTS, AudioTranscription, etc.) to include grpc.WithDefaultCallOptions for setting MaxCallRecvMsgSize and MaxCallSendMsgSize to 50MB. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Ettore Di Giacinto <[email protected]>
Copy link

netlify bot commented Apr 18, 2025

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 217c241
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/6802bcf662016700087bc417
😎 Deploy Preview https://deploy-preview-5212--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@mudler mudler merged commit 8abecb4 into master Apr 19, 2025
25 checks passed
@mudler mudler deleted the chore/bump_grpc_limits branch April 19, 2025 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant