Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

❓ [Question] Jetson AGX Orin Install in Jetpack 6.1 Build did NOT complete successfully #3317

Open
breknddone opened this issue Dec 11, 2024 · 5 comments
Open

❓ [Question] Jetson AGX Orin Install in Jetpack 6.1 Build did NOT complete successfully #3317

breknddone opened this issue Dec 11, 2024 · 5 comments
Labels
question Further information is requested

Comments

@breknddone
Copy link

breknddone commented Dec 11, 2024 •
edited
Loading

❓ Question

I follow this tutorial to install Torch-TensorRT, but in the last step:

# build and install torch_tensorrt wheel file
python setup.py --use-cxx11-abi install --user

some errors happened:

using CXX11 ABI build
Jetpack version: 6.1
building libtorchtrt cmd=['/usr/bin/bazel', 'build', '//:libtorchtrt', '--compilation_mode=opt', '--distdir=third_party/dist_dir/x86_64-linux-gnu', '--config=linux', '--platforms=//toolchains:jetpack_6.1']
DEBUG: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/external/rules_python~/python/private/python.bzl:46:10: WARNING: Ignoring toolchain 'python_3_11' from module 'rules_pkg': Toolchain 'python_3_11' from module 'torch_tensorrt' already registered Python version 3.11 and has precedence
INFO: Analyzed target //:libtorchtrt (127 packages loaded, 13849 targets configured).
ERROR: /home/lab223/TensorRT/core/util/BUILD:60:11: Compiling core/util/Exception.cpp failed: (Exit 1): gcc failed: error executing CppCompile command (from target //core/util:exception) /home/lab223/anaconda3/envs/rnw/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 25 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
gcc: fatal error: cannot execute 'cc1plus': execvp: No such file or directory
compilation terminated.
Target //:libtorchtrt failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 8.444s, Critical Path: 4.05s
INFO: 329 processes: 329 internal.
ERROR: Build did NOT complete successfully

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • PyTorch Version (e.g., 1.0): 2.5.0
  • CPU Architecture: arm64(Jetson AGX Orin)
  • OS (e.g., Linux): Linux
  • How you installed PyTorch: pip
  • Build command you used (if compiling from source): python setup.py --use-cxx11-abi install --user
  • Are you using local sources or building from archives: building from archives
  • Python version: 3.10.15
  • CUDA version: 12.6
  • GPU models and configuration: -
  • Any other relevant information: Install torch_tensorrt in the model's anaconda virtual environment

Additional context

please help me!thanks!!!!
@breknddone breknddone added the question Further information is requested label Dec 11, 2024
@narendasan
Copy link
Collaborator

Can you provide the results of adding --sandbox_debug and --verbose_failures to the build command in setup.py? Seems like from what is here there is a toolchain issue with g++.

@breknddone
Copy link
Author

breknddone commented Dec 12, 2024 •
edited
Loading

Can you provide the results of adding --sandbox_debug and --verbose_failures to the build command in setup.py? Seems like from what is here there is a toolchain issue with g++.

I add --sandbox_debug and --verbose_failures to setup.py, and the debug info shows:

using CXX11 ABI build
Jetpack version: 6.1
building libtorchtrt cmd=['/usr/bin/bazel', 'build', '//:libtorchtrt', '--compilation_mode=opt', '--distdir=third_party/dist_dir/x86_64-linux-gnu', '--config=linux', '--platforms=//toolchains:jetpack_6.1', '--sandbox_debug', '--verbose_failures']
INFO: Analyzed target //:libtorchtrt (0 packages loaded, 0 targets configured).
DEBUG: Sandbox debug output for CppCompile //core/util:exception:
1733989836.324773440: src/main/tools/linux-sandbox.cc:156: calling pipe(2)...
1733989836.324872248: src/main/tools/linux-sandbox.cc:165: Netns is 0
1733989836.324877687: src/main/tools/linux-sandbox.cc:176: calling clone(2)...
1733989836.325472613: src/main/tools/linux-sandbox.cc:185: linux-sandbox-pid1 has PID 52445
1733989836.325506626: src/main/tools/linux-sandbox-pid1.cc:700: Pid1Main started
1733989836.325653557: src/main/tools/linux-sandbox.cc:202: done manipulating pipes
1733989836.325868995: src/main/tools/linux-sandbox-pid1.cc:293: bind mount: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/_hermetic_tmp -> /tmp
1733989836.325927262: src/main/tools/linux-sandbox-pid1.cc:311: writable: /dev/shm
1733989836.325944541: src/main/tools/linux-sandbox-pid1.cc:311: writable: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main
1733989836.325960347: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp
1733989836.325968923: src/main/tools/linux-sandbox-pid1.cc:327: working dir: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main
1733989836.326134893: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /
1733989836.326149963: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc
1733989836.326158059: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1733989836.326176361: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /proc/sys/fs/binfmt_misc, nullptr, 2101281, nullptr) failure (Operation not permitted) ignored
1733989836.326184968: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1733989836.326191816: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys
1733989836.326197703: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/security
1733989836.326210598: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/selinux
1733989836.326220869: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/cgroup
1733989836.326227653: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/pstore
1733989836.326236900: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/firmware/efi/efivars
1733989836.326250755: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/bpf
1733989836.326257762: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/debug
1733989836.326264962: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/tracing
1733989836.326272545: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/fuse/connections
1733989836.326339195: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/config
1733989836.326350362: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev
1733989836.326355770: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1733989836.326362713: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/pts
1733989836.326368921: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/hugepages
1733989836.326375480: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/mqueue
1733989836.326381368: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run
1733989836.326386967: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/lock
1733989836.326393399: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/credentials/systemd-sysusers.service
1733989836.326458641: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000
1733989836.326478543: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/gvfs
1733989836.326492142: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/doc
1733989836.326502797: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/snapd/ns
1733989836.326508973: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /run/snapd/ns, nullptr, 4135, nullptr) failure (Permission denied) ignored
1733989836.326560169: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/bare/5
1733989836.326601637: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/1666
1733989836.326610404: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/firefox/5271
1733989836.326618244: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/firefox/5360
1733989836.326624707: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/gnome-42-2204/178
1733989836.326631746: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/gtk-common-themes/1535
1733989836.326639298: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/21761
1733989836.326646337: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/23259
1733989836.326652193: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /boot/efi
1733989836.326660000: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/1720
1733989836.326666656: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /media/lab223/6a0b1392-a961-47f9-ae83-1c4702c6aca2
1733989836.326676863: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1733989836.326682750: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1733989836.326693149: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main
1733989836.326707772: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1733989836.326712764: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main
1733989836.326861551: src/main/tools/linux-sandbox-pid1.cc:496: calling fork...
1733989836.327124665: src/main/tools/linux-sandbox-pid1.cc:533: child started with PID 2
1733989836.343405748: src/main/tools/linux-sandbox-pid1.cc:550: wait returned pid=2, status=0x100
1733989836.343437649: src/main/tools/linux-sandbox-pid1.cc:568: child exited normally with code 1
1733989836.344321766: src/main/tools/linux-sandbox.cc:243: child exited normally with code 1

Run this command to start an interactive shell in an identical sandboxed environment:
(exec env - \
	LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64: \
	PATH=/home/lab223/.cache/bazelisk/downloads/sha256/5a4cc979353671e438b9469b833924c2361e25a580cc278a75877aedc27c1c53/bin:/home/lab223/anaconda3/envs/rnw/bin:/home/lab223/anaconda3/condabin:/usr/local/cuda-12.6/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin \
	PWD=/proc/self/cwd \
	TMPDIR=/tmp \
  /home/lab223/.cache/bazel/_bazel_lab223/install/128438993754f9753a1e4f56fdd76124/linux-sandbox -t 15 -w /dev/shm -w /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main -w /tmp -M /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/_hermetic_tmp -m /tmp -S /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/stats.out -D /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/debug.out -- /bin/sh -i)
ERROR: /home/lab223/TensorRT/core/util/BUILD:60:11: Compiling core/util/Exception.cpp failed: (Exit 1): linux-sandbox failed: error executing CppCompile command
  (cd /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main && \
  exec env - \
	LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64: \
	PATH=/home/lab223/.cache/bazelisk/downloads/sha256/5a4cc979353671e438b9469b833924c2361e25a580cc278a75877aedc27c1c53/bin:/home/lab223/anaconda3/envs/rnw/bin:/home/lab223/anaconda3/condabin:/usr/local/cuda-12.6/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin \
	PWD=/proc/self/cwd \
	TMPDIR=/tmp \
  /home/lab223/.cache/bazel/_bazel_lab223/install/128438993754f9753a1e4f56fdd76124/linux-sandbox -t 15 -w /dev/shm -w /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/execroot/_main -w /tmp -M /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/_hermetic_tmp -m /tmp -S /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/stats.out -D /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/4/debug.out -- /home/lab223/anaconda3/envs/rnw/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++14' -MD -MF bazel-out/aarch64-opt/bin/core/util/_objs/exception/Exception.d '-frandom-seed=bazel-out/aarch64-opt/bin/core/util/_objs/exception/Exception.o' -iquote . -iquote bazel-out/aarch64-opt/bin '-std=c++17' '-fdiagnostics-color=always' '-std=c++17' '-fdiagnostics-color=always' -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c core/util/Exception.cpp -o bazel-out/aarch64-opt/bin/core/util/_objs/exception/Exception.o)
gcc: fatal error: cannot execute 'cc1plus': execvp: No such file or directory
compilation terminated.
Target //:libtorchtrt failed to build
INFO: Elapsed time: 0.358s, Critical Path: 0.06s
INFO: 4 processes: 4 internal.
ERROR: Build did NOT complete successfully

And my gcc/g++ location and version：

(rnw) lab223@ubuntu:~/TensorRT$ which gcc
/home/lab223/anaconda3/envs/rnw/bin/gcc
(rnw) lab223@ubuntu:~/TensorRT$ which g++
/usr/bin/g++
(rnw) lab223@ubuntu:~/TensorRT$ gcc --version
gcc (conda-forge gcc 12.1.0-17) 12.1.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(rnw) lab223@ubuntu:~/TensorRT$ g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

how can I solve this ?

@narendasan
Copy link
Collaborator

this seems relevant since you are using conda pytorch/vision#8540

@breknddone
Copy link
Author

breknddone commented Dec 13, 2024 •
edited
Loading

this seems relevant since you are using conda pytorch/vision#8540

I follow this method, it seems worked.

conda install -c conda-forge gxx_linux-64
cd /home/ms/anaconda3/envs/NAME/bin
ln -s /home/ms/anaconda3/envs/NAME/bin/aarch64-conda_cos7-linux-gnu-g++ g++

But some new errors happend:

using CXX11 ABI build
Jetpack version: 6.1
building libtorchtrt cmd=['/usr/bin/bazel', 'build', '//:libtorchtrt', '--compilation_mode=opt', '--distdir=third_party/dist_dir/x86_64-linux-gnu', '--config=linux', '--platforms=//toolchains:jetpack_6.1', '--sandbox_debug', '--verbose_failures']
INFO: Analyzed target //:libtorchtrt (0 packages loaded, 0 targets configured).
DEBUG: Sandbox debug output for CppLink //cpp/lib:libtorchtrt_plugins.so:
1734072959.687516534: src/main/tools/linux-sandbox.cc:156: calling pipe(2)...
1734072959.687671851: src/main/tools/linux-sandbox.cc:165: Netns is 0
1734072959.687678509: src/main/tools/linux-sandbox.cc:176: calling clone(2)...
1734072959.688578559: src/main/tools/linux-sandbox.cc:185: linux-sandbox-pid1 has PID 77219
1734072959.689758736: src/main/tools/linux-sandbox-pid1.cc:700: Pid1Main started
1734072959.689900160: src/main/tools/linux-sandbox.cc:202: done manipulating pipes
1734072959.690371328: src/main/tools/linux-sandbox-pid1.cc:293: bind mount: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/_hermetic_tmp -> /tmp
1734072959.690450395: src/main/tools/linux-sandbox-pid1.cc:311: writable: /dev/shm
1734072959.690468641: src/main/tools/linux-sandbox-pid1.cc:311: writable: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main
1734072959.690484710: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp
1734072959.690493769: src/main/tools/linux-sandbox-pid1.cc:327: working dir: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main
1734072959.690668293: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /
1734072959.690681193: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc
1734072959.690688395: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1734072959.690709074: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /proc/sys/fs/binfmt_misc, nullptr, 2101281, nullptr) failure (Operation not permitted) ignored
1734072959.690718582: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1734072959.690724824: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys
1734072959.690731034: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/security
1734072959.690744510: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/selinux
1734072959.690755426: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/cgroup
1734072959.690762501: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/pstore
1734072959.690770855: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/firmware/efi/efivars
1734072959.690783340: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/bpf
1734072959.690790574: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/debug
1734072959.690798641: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/tracing
1734072959.690806067: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/fuse/connections
1734072959.690871882: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/config
1734072959.690882157: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev
1734072959.690887343: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1734072959.690893777: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/pts
1734072959.690900916: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/hugepages
1734072959.690906709: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/mqueue
1734072959.690912247: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run
1734072959.690917785: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/lock
1734072959.690923835: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/credentials/systemd-sysusers.service
1734072959.690931646: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000
1734072959.690941089: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/gvfs
1734072959.690948452: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/doc
1734072959.690954630: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/snapd/ns
1734072959.690959880: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /run/snapd/ns, nullptr, 4135, nullptr) failure (Permission denied) ignored
1734072959.691003574: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/bare/5
1734072959.691048037: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/1666
1734072959.691056328: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/firefox/5271
1734072959.691063915: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/firefox/5360
1734072959.691071149: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/gnome-42-2204/178
1734072959.691077744: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/gtk-common-themes/1535
1734072959.691084466: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/21761
1734072959.691091604: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/23259
1734072959.691097462: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /boot/efi
1734072959.691105593: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/1720
1734072959.691111483: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /media/lab223/6a0b1392-a961-47f9-ae83-1c4702c6aca2
1734072959.691120990: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1734072959.691126016: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1734072959.691134819: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main
1734072959.691146823: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1734072959.691153001: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main
1734072959.691290776: src/main/tools/linux-sandbox-pid1.cc:496: calling fork...
1734072959.691626410: src/main/tools/linux-sandbox-pid1.cc:533: child started with PID 2
1734072959.804221858: src/main/tools/linux-sandbox-pid1.cc:550: wait returned pid=2, status=0x100
1734072959.804259631: src/main/tools/linux-sandbox-pid1.cc:568: child exited normally with code 1
1734072959.805278921: src/main/tools/linux-sandbox.cc:243: child exited normally with code 1

Run this command to start an interactive shell in an identical sandboxed environment:
(exec env - \
	LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64: \
	PATH=/home/lab223/.cache/bazelisk/downloads/sha256/5a4cc979353671e438b9469b833924c2361e25a580cc278a75877aedc27c1c53/bin:/home/lab223/anaconda3/envs/rnw/bin:/home/lab223/anaconda3/condabin:/usr/local/cuda-12.6/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin \
	PWD=/proc/self/cwd \
	TMPDIR=/tmp \
	ZERO_AR_DATE=1 \
  /home/lab223/.cache/bazel/_bazel_lab223/install/128438993754f9753a1e4f56fdd76124/linux-sandbox -t 15 -w /dev/shm -w /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main -w /tmp -M /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/_hermetic_tmp -m /tmp -S /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/stats.out -D /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/debug.out -- /bin/sh -i)
ERROR: /home/lab223/TensorRT/cpp/lib/BUILD:5:10: Linking cpp/lib/libtorchtrt_plugins.so failed: (Exit 1): linux-sandbox failed: error executing CppLink command
  (cd /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main && \
  exec env - \
	LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64: \
	PATH=/home/lab223/.cache/bazelisk/downloads/sha256/5a4cc979353671e438b9469b833924c2361e25a580cc278a75877aedc27c1c53/bin:/home/lab223/anaconda3/envs/rnw/bin:/home/lab223/anaconda3/condabin:/usr/local/cuda-12.6/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin \
	PWD=/proc/self/cwd \
	TMPDIR=/tmp \
	ZERO_AR_DATE=1 \
  /home/lab223/.cache/bazel/_bazel_lab223/install/128438993754f9753a1e4f56fdd76124/linux-sandbox -t 15 -w /dev/shm -w /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/execroot/_main -w /tmp -M /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/_hermetic_tmp -m /tmp -S /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/stats.out -D /home/lab223/.cache/bazel/_bazel_lab223/3fb6c16c20f38dfc11e57e77e6eea473/sandbox/linux-sandbox/153/debug.out -- /home/lab223/anaconda3/envs/rnw/bin/x86_64-conda-linux-gnu-cc @bazel-out/aarch64-opt/bin/cpp/lib/libtorchtrt_plugins.so-2.params)
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~tensorrt_S_S_Cnvinferplugin___Ulib_Saarch64-linux-gnu/libnvinfer_plugin.so while searching for nvinfer_plugin
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -lnvinfer_plugin
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~tensorrt_S_S_Cnvinfer_Ulib___Ulib_Saarch64-linux-gnu/libnvinfer.so while searching for nvinfer
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -lnvinfer
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~cuda_S_S_Ccudart___Ulib64/libcudart.so while searching for cudart
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -lcudart
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Ctorch___Ulib/libtorch.so while searching for torch
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -ltorch
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Ctorch___Ulib/libtorch_cpu.so while searching for torch_cpu
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -ltorch_cpu
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Ctorch___Ulib/libtorch_cuda.so while searching for torch_cuda
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -ltorch_cuda
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Ctorch___Ulib/libtorch_global_deps.so while searching for torch_global_deps
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -ltorch_global_deps
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Cc10_Ucuda___Ulib/libc10_cuda.so while searching for c10_cuda
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -lc10_cuda
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: warning: skipping incompatible bazel-out/aarch64-opt/bin/_solib_aarch64/_U@@_Umain~_Urepo_Urules~libtorch_S_S_Cc10___Ulib/libc10.so while searching for c10
/home/lab223/anaconda3/envs/rnw/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld.gold: error: cannot find -lc10
collect2: error: ld returned 1 exit status
Target //:libtorchtrt failed to build
INFO: Elapsed time: 0.546s, Critical Path: 0.15s
INFO: 13 processes: 13 internal.
ERROR: Build did NOT complete successfully

@narendasan
Copy link
Collaborator

hmm, can you share the module.bzl file you are using?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@narendasan @breknddone