Open
Description
The test suite assumes the server's environment with 2 available GPUs. As a result, tests will fail on insufficient hardware.
- Extend the
vllm_modified
file to detect what environment we are on and automatically switch to a CPU-based LLM implementation so that tests can still be adequately ran. - Create a separate requirements file for development; as
vllm
only supports Linux. - To manage separate requirements file, may as well set up the entire repository as a package that can be self-imported (
pip install -e .
), allowing us to remove these pesky relative imports as well (and run anything from anywhere in the package).