Fresh docker install not finishing

[ajaxdude]

I'm on Fedora/gnome/docker/compose/amd ( 7900xt )

I tried the launch script for rocm and AOE/Vulkan. It just won't start.

My dream is to get docker+vLLm+localai+ either rocm or vulkan working and connected to crush CLI. Any tips?

[ajaxdude]

here is the error for: docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas

Status: Downloaded newer image for localai/localai:latest-aio-gpu-hipblas
docker: Error response from daemon: failed to set up container networking: driver failed programming external connectivity on endpoint local-ai (458b52fb590693a5ee5532eedaac46323b6180c23f58e557d9d4adcfb7813d7b): failed to bind host port for 0.0.0.0:8080:172.17.0.2:8080/tcp: address already in use

Run 'docker run --help' for more information

[ajaxdude]

fix the port conflict but now get this:

CPU: AVX found OK
CPU: AVX2 found OK
CPU: AVX512 found OK
8:56PM INF Starting LocalAI using 6 threads, with models path: //models
8:56PM INF LocalAI version: v3.5.4 (f7f26b8)
8:56PM WRN Failed to read system backends, proceeding with user-managed backends error="open /usr/share/localai/backends: no such file or directory"
8:56PM INF Using forced capability run file () capability="amd\n" capabilityRunFile=/run/localai/capability
8:56PM WRN Failed to read system backends, proceeding with user-managed backends error="open /usr/share/localai/backends: no such file or directory"
8:56PM INF Using forced capability run file () capability="amd\n" capabilityRunFile=/run/localai/capability
8:56PM INF Using forced capability run file () capability="amd\n" capabilityRunFile=/run/localai/capability

[dennisvanderpool]

Mount the folder /usr/share/localai/backends into a volume or host dir