Deploy spacy serverless #11494
-
|
Hi all, I'm wondering what my best bet is for deploying spacy in the cloud for real-time inference, with a serverless solution? Basically, I want an endpoint similar to what AWS can offer with an EC2 or SageMaker instance, but at the smallest possible cost and with autoscaling (to zero in times of no traffic). I can tolerate a few seconds delay for cold starts (when loading the model), but after that inference should be at basically no latency (hence AWS Lambda is not an option). I have looked into SageMaker Serverless Inference, but find it quite poorly documented, not sure if applicable for my use case? Looking forward to your input! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Serverless use of spaCy is pretty common, but we don't actually have any particular recommendations for cloud providers. spaCy's needs shouldn't be too idiosyncratic, so it sounds like for your goals you might be better off asking at a Stack Exchange site (not sure exactly which one) or Reddit or something. Sorry we can't provide more help! |
Beta Was this translation helpful? Give feedback.
-
|
Gotcha, thanks! |
Beta Was this translation helpful? Give feedback.
Serverless use of spaCy is pretty common, but we don't actually have any particular recommendations for cloud providers. spaCy's needs shouldn't be too idiosyncratic, so it sounds like for your goals you might be better off asking at a Stack Exchange site (not sure exactly which one) or Reddit or something. Sorry we can't provide more help!