Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Inference]: Mistral7b on GPUs with JARK stack with Ray Serve #497

Open
vara-bonthu opened this issue Apr 8, 2024 · 3 comments
Open

[Inference]: Mistral7b on GPUs with JARK stack with Ray Serve #497

vara-bonthu opened this issue Apr 8, 2024 · 3 comments
Assignees
@sheetaljoshi
Labels
enhancement New feature or request gen-ai pattern Distributed Training and Inference Patterns for Various Generative AI Large Language Models (LLMs)

Comments

@vara-bonthu
Copy link
Collaborator

No description provided.

@vara-bonthu vara-bonthu self-assigned this Apr 8, 2024
@vara-bonthu vara-bonthu added the gen-ai pattern Distributed Training and Inference Patterns for Various Generative AI Large Language Models (LLMs) label Apr 14, 2024
@vara-bonthu vara-bonthu changed the title Mistral7b on GPUs with JARK stack [Inference]: Mistral7b on GPUs with JARK stack with Ray Serve Apr 14, 2024
@github-actions GitHub Actions
Copy link
Contributor

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

@github-actions github-actions bot added the stale label May 16, 2024
@askulkarni2 askulkarni2 added enhancement New feature or request and removed stale labels May 16, 2024
@sheetaljoshi
Copy link

I am working on this issue, can you please assign this roadmap item to me.

@ratnopamc
Copy link
Collaborator

There's an issue with using the HF Transformer versions with mistral-7b-intsruct-v0.2 model.
https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/discussions/148

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sheetaljoshi sheetaljoshi

Labels
enhancement New feature or request gen-ai pattern Distributed Training and Inference Patterns for Various Generative AI Large Language Models (LLMs)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@askulkarni2 @sheetaljoshi @vara-bonthu @ratnopamc