-
Notifications
You must be signed in to change notification settings - Fork 48
Description

Steps:
conda activate chatunivi
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
pip install --upgrade pip
##"deepspeed==0.9.5"
pip install -e .
pip install bitsandbytes-windows
CUDA setup:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # Set GPU if available, otherwise CPU
print(f"Using device: {device}")
Output: Using device: cuda
Warning: "A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.2 as it may crash."
Warning: INFO: 127.0.0.1:58341 - "POST /run/predict HTTP/1.1" 422 Unprocessable Entity
Discussion Reference: pip install fastapi==0.95.2
Error Message:
Something went wrong
Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.
ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.
INFO: 127.0.0.1:56546 - "POST /run/predict HTTP/1.1" 500 Internal Server Error