Skip to content

Support for running integration tests locally #15

Open
@Ar4l

Description

@Ar4l

The test suite assumes the server's environment with 2 available GPUs. As a result, tests will fail on insufficient hardware.

  • Extend the vllm_modified file to detect what environment we are on and automatically switch to a CPU-based LLM implementation so that tests can still be adequately ran.
  • Create a separate requirements file for development; as vllm only supports Linux.
  • To manage separate requirements file, may as well set up the entire repository as a package that can be self-imported (pip install -e .), allowing us to remove these pesky relative imports as well (and run anything from anywhere in the package).

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions