garak audit for large offline models that wont fit in one gpu #1090
Unanswered
eivorwolfkissed
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am able to run garak on a smaller model. how would i do it for a bigger model say llama370B that requires multiple gpus to start inferencing?
Beta Was this translation helpful? Give feedback.
All reactions