Skip to content

Error! mthreads-gmi did not output anything! Please check the underlying drivers and container tools #79

Open
@GT-610

Description

@GT-610

I run the docker command in the README, but failed to pass the check:

sudo docker run --rm -it --privileged --pull always --network=host --name=torch_musa_dev --env MTHREADS_VIS
IBLE_DEVICES=all --shm-size=80g registry.mthreads.com/mcconline/musa-pytorch-dev-public:rc3.1.0-v1.3.0-S80-py310 /bin/bas
h
rc3.1.0-v1.3.0-S80-py310: Pulling from mcconline/musa-pytorch-dev-public
Digest: sha256:70bc3e7494ac4538867706bdf7e0b996fa8ab35fcc08d257da6346c22e55118c
Status: Image is up to date for registry.mthreads.com/mcconline/musa-pytorch-dev-public:rc3.1.0-v1.3.0-S80-py310
Error! mthreads-gmi did not output anything! Please check the underlying drivers and container tools
/usr/bin/which: this version of `which' is deprecated; use `command -v' in scripts instead.
If the size of mthreads-gmi is 0, musa container tool is abnormal.
Error! In the output of clinfo, the value of 'Platform Vendor' is not 'MooreThreads'!
/usr/bin/ld: warning: libdrm.so.2, needed by /usr/lib/x86_64-linux-gnu/musa/libsrv_um_MUSA.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVReleaseDeviceMappingDirectX'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVMapToDeviceDirect'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVDevmemXAllocUnifiedVirtualRange'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/musa/libsrv_um_MUSA.so: undefined reference to `drmFreeVersion'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `musa_KickCETQ'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVDevMemXFreeUnifiedVirtualRange'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVGetFabricType'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVGetVersion'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/musa/libsrv_um_MUSA.so: undefined reference to `drmGetDevices2'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/musa/libsrv_um_MUSA.so: undefined reference to `drmGetVersion'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `RGXDecodeHWRInfo'
/usr/bin/ld: /usr/local/musa/lib/libmusa.so.1.0: undefined reference to `PVRSRVGetHeapTotalSize'
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
MCC failed to compile a simple demo musa program. The compilation command is:mcc test_musa.mu -o test_musa -mtgpu -O2 -lmusart -L/lib
 __  __  ___   ___  ____  _____   _____ _   _ ____  _____    _    ____  ____
|  \/  |/ _ \ / _ \|  _ \| ____| |_   _| | | |  _ \| ____|  / \  |  _ \/ ___|
| |\/| | | | | | | | |_) |  _|     | | | |_| | |_) |  _|   / _ \ | | | \___ \
| |  | | |_| | |_| |  _ <| |___    | | |  _  |  _ <| |___ / ___ \| |_| |___) |
|_|  |_|\___/ \___/|_| \_\_____|   |_| |_| |_|_| \_\_____/_/   \_\____/|____/

 _____     _ _          _ _
|  ___|_ _(_) | ___  __| | |
| |_ / _` | | |/ _ \/ _` | |
|  _| (_| | | |  __/ (_| |_|
|_|  \__,_|_|_|\___|\__,_(_)

(py310) root@localhost:/home#

I'm sure everything is ok. I run docker run --rm --env MTHREADS_VISIBLE_DEVICES=all ubuntu:20.04 mthreads-gmi to check if container toolkit works, and shows this:

sudo docker run --rm --env MTHREADS_VISIBLE_DEVICES=all ubuntu:20.04 mthreads-gmi
Wed Apr  9 11:32:44 2025
---------------------------------------------------------------
    mthreads-gmi:1.8.2           Driver Version:N/A
---------------------------------------------------------------
ID   Name           |PCIe                |%GPU  Mem
     Device Type    |Pcie Lane Width     |Temp  MPC Capable
+-------------------------------------------------------------+
0    MTT S80        |00000000:01:00.0    |0%    0MiB(16384MiB)
     Physical       |16x(16x)            |48C   YES
---------------------------------------------------------------

---------------------------------------------------------------
Processes:
ID   PID       Process name                         GPU Memory
                                                         Usage
+-------------------------------------------------------------+
   No running processes found
---------------------------------------------------------------

So it is probably a problem of torch-musa container.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions