Cannot compile: gnu/stubs-ilp32.h: No such file or directory

[capiman]

I am using a Raspberry Pi 5 (64 bit) with OS release bookworm.

I first tried this:

git clone https://github.com/riscv/riscv-gnu-toolchain
./configure --prefix=/opt/riscv
make linux

Compile worked. But I had an error, when compiling my app source code:
(from https://github.com/hydrausb3 )

/opt/riscv/bin/riscv64-unknown-linux-gnu-gcc -march=rv32imac_zicsr -mabi=ilp32 -msmall-data-limit=8 -O3 -fmessage-length=0 -fsigned-char -ffunction-sections -fdata-sections -DDEBUG=1 -I"../wch-ch56x-bsp/rvmsis" -I"../wch-ch56x-bsp/drv" -I"../wch-ch56x-bsp/board" -I"../wch-ch56x-bsp/usb/usb_devbulk" -I"./User" -std=gnu99 -MMD -MP -MT"build/core_riscv.o" -c -o "build/core_riscv.o" "../wch-ch56x-bsp/rvmsis/core_riscv.c"
In file included from /opt/riscv/sysroot/usr/include/features.h:527,
from /opt/riscv/sysroot/usr/include/bits/libc-header-start.h:33,
from /opt/riscv/sysroot/usr/include/stdint.h:26,
from /opt/riscv/lib/gcc/riscv64-unknown-linux-gnu/13.2.0/include/stdint.h:9,
from ../wch-ch56x-bsp/rvmsis/core_riscv.c:11:
/opt/riscv/sysroot/usr/include/gnu/stubs.h:8:11: fatal error: gnu/stubs-ilp32.h: No such file or directory
8 | # include <gnu/stubs-ilp32.h>
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:103: build/core_riscv.o] Error 1
trio@adapter-radio:~/risc-v/hydrausb3_fw_qmi/HydraUSB3_USB $

I assumed, I need something different:

make clean
./configure --prefix=/opt/riscv --with-arch=rv32gc --with-abi=ilp32
make linux

But after some time of compiling I got this (same!) error:

/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1/./gcc/xgcc -B/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1/./gcc/ -B/opt/riscv/riscv32-unknown-linux-gnu/bin/ -B/opt/riscv/riscv32-unknown-linux-gnu/lib/ -isystem /opt/riscv/riscv32-unknown-linux-gnu/include -isystem /opt/riscv/riscv32-unknown-linux-gnu/sys-include -O2 -mcmodel=medlow -O2 -O2 -mcmodel=medlow -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fPIC -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fPIC -I. -I. -I../.././gcc -I../../.././gcc/libgcc -I../../.././gcc/libgcc/. -I../../.././gcc/libgcc/../gcc -I../../.././gcc/libgcc/../include -DHAVE_CC_TLS -fvisibility=hidden -DHIDE_EXPORTS -c eh_dummy.c
-o eh_dummy.o;
objects=eh_dummy.o;
fi;
/opt/riscv/riscv32-unknown-linux-gnu/bin/ar rc libgcc.a $objects
/opt/riscv/riscv32-unknown-linux-gnu/bin/ranlib libgcc.a
/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1/./gcc/xgcc -B/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1/./gcc/ -B/opt/riscv/riscv32-unknown-linux-gnu/bin/ -B/opt/riscv/riscv32-unknown-linux-gnu/lib/ -isystem /opt/riscv/riscv32-unknown-linux-gnu/include -isystem /opt/riscv/riscv32-unknown-linux-gnu/sys-include -O2 -mcmodel=medlow -O2 -O2 -mcmodel=medlow -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fPIC -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fPIC -I. -I. -I../.././gcc -I../../.././gcc/libgcc -I../../.././gcc/libgcc/. -I../../.././gcc/libgcc/../gcc -I../../.././gcc/libgcc/../include -DHAVE_CC_TLS -o _gcov_merge_add.o -MT _gcov_merge_add.o -MD -MP -MF _gcov_merge_add.dep -DL_gcov_merge_add -c ../../.././gcc/libgcc/libgcov-merge.c
In file included from /opt/riscv/sysroot/usr/include/features.h:527,
from /opt/riscv/sysroot/usr/include/sys/mman.h:22,
from ../../.././gcc/libgcc/libgcov.h:49,
from ../../.././gcc/libgcc/libgcov-merge.c:26:
/opt/riscv/sysroot/usr/include/gnu/stubs.h:8:11: fatal error: gnu/stubs-ilp32.h: No such file or directory
8 | # include <gnu/stubs-ilp32.h>
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:924: _gcov_merge_add.o] Error 1
make[2]: Leaving directory '/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1/riscv32-unknown-linux-gnu/libgcc'
make[1]: *** [Makefile:12883: all-target-libgcc] Error 2
make[1]: Leaving directory '/home/trio/riscv-gnu-toolchain/build-gcc-linux-stage1'
make: *** [Makefile:434: stamps/build-gcc-linux-stage1] Error 2
trio@adapter-radio:~/riscv-gnu-toolchain $

[TommyMurphyTM1234]

This may be relevant:

• What is the correct way to generate 4 different architectures, rv32i, rv32im, rv32imv, all ilp32? #941 (comment)

[TommyMurphyTM1234]

Did that help at all @capiman?
Can you describe in detail all commands needed to reproduce the issue please?

[capiman]

I created an empty

gnu/stubs-ilp32.h

then it continued compiling, but next was missing

nano.specs

and

nosys.specs

I meanwhile put my own compiling on hold and use a pre-compiled version from

https://github.com/hydrausb3/riscv-none-elf-gcc-xpack

I am now compiling on Windows, scp created binary to RPI5 and flash CH569&test there.
(I don't need a multi-architecture-binary, just the one for CH569 (similar to https://github.com/hydrausb3/riscv-none-elf-gcc-xpack) would be enough.)

[TommyMurphyTM1234]

then it continued compiling, but next was missing

nano.specs

and

nosys.specs

Aren't those files part of the bare-metal/Newlib toolchain but you seem to be using the Linux/Glibc toolchain?
Are you sure that you're using the correct toolchain?
Is your target actually running some version of the Linux kernel/OS?
If not then you probably need the bare-metal toolchain.

I meanwhile put my own compiling on hold and use a pre-compiled version from

https://github.com/hydrausb3/riscv-none-elf-gcc-xpack

That's a bare-metal toolchain.
Your original post refers to the Linux/Glibc toolchain.
This is all a bit confusing.

I am now compiling on Windows, scp created binary to RPI5 and flash CH569&test there. (I don't need a multi-architecture-binary, just the one for CH569 (similar to https://github.com/hydrausb3/riscv-none-elf-gcc-xpack) would be enough.)

OK - I will close this issue so since you seem to have overcome or at least circumvented the original issue - which is probably an issue for the upstream GCC/Glibc in any case.

[TommyMurphyTM1234]

then it continued compiling, but next was missing
nano.specs
and
nosys.specs

Aren't those files part of the bare-metal/Newlib toolchain but you seem to be using the Linux/Glibc toolchain?

I confirmed that this is the case by looking at the contents of the latest CI builds of the Linux/Glibc and bare-metal/Newlib toolchains. Only the latter bundles those specs files.

I think that there may be some confusion here about the appropriate toolchain to use.

[capiman]

Many thanks for checking!

Have I just selected the wrong content of the parameters for the configure step?
Or is https://github.com/riscv-collab/riscv-gnu-toolchain the wrong toolchain at all for building bare-metal/Newlib?

[TommyMurphyTM1234]

Or is https://github.com/riscv-collab/riscv-gnu-toolchain the wrong toolchain at all for building bare-metal/Newlib?

This repo can build various bare-metal and Linux toolchains for RISC-V as illustrated by the CI/CT builds:

• https://github.com/riscv-collab/riscv-gnu-toolchain/releases/tag/2023.12.20

and as explained in the readme:

• https://github.com/riscv-collab/riscv-gnu-toolchain

Have I just selected the wrong content of the parameters for the configure step?

If you want to compile specifically for a bare-metal rv32imac_zicsr/ilp32 target then then the simplest thing would be to build a toolchain for that specific arch/abi:

./configure --prefix=... --with-arch=rv32imac_zicsr --with-abi=ilp32
make

and then compile your code as follows:

# Note - toolchain default arch/abi is rv32imac_zicsr/ilp32 so no need for -march=rv32imac -mabi=ilp32
riscv32-unknown-elf-gcc -c foobar.c
etc.

[capiman]

Many thanks! This looks very good.

I was able to compile gcc with the suggested commands on Raspberry Pi 5 64 bit bookworm.
I was also able to compile my own source code with it, no files were missing and which was also almost working:
I already knew that it will fail, because 1 patch is missing which is needed for proprietary handling of interrupts:
hydrausb3/riscv-none-elf-gcc-xpack@90893d7
(CH569 needs an additional keyword "WCH-Interrupt-fast": https://github.com/hydrausb3/wch-ch56x-bsp/blob/eb6318d0c61c05baa527f59c47e5b967e4e9b00e/usb/usb_devbulk/CH56x_usb20_devbulk.c#L546 )
I will try this patch tomorrow (already late here in Germany).
With this patch I can test if the generated binary really works on real hardware and the USB peripheral is working.

Is https://github.com/riscv-collab/riscv-gnu-toolchain the right place to commit such a patch,
or must it be committed higher (gcc itself, upstream) if it is finally working?

[cmuellner]

Support or __attribute__((interrupt("WCH-Interrupt-fast"))) has close to zero chance to get upstream. There are several different register prestacking behaviours in different uCs/SoCs and differentiating them by different names is not very maintenance friendly.

But there is an attempt to standardize a more generic model to tell the compiler about prestacked registers of an IRQ handler in riscv-non-isa/riscv-c-api-doc#53.

In this repo, we don't take any patches on top of upstream sources.

[TommyMurphyTM1234]

Support or __attribute__((interrupt("WCH-Interrupt-fast"))) has close to zero chance to get upstream.

@capiman perhaps you need to be using the WCH fork of the RISC-V GCC toolchain that supports propriety extensions of theirs such as this? If you can find/get their forked sources that is...

• https://stackoverflow.com/questions/76367328/where-is-the-source-code-from-the-wch-risc-v-toolchain
• https://www.reddit.com/r/RISCV/comments/126262j/notes_on_wch_fast_interrupts/

[capiman]

I added the above mentioned patch to local version of gcc.
Compiling of gcc was successful.
Compiling of my USB code was also successful. USB enumeration was also working,
so interrupt handling (including WCH-Interrupt-fast) seems to work.

[TommyMurphyTM1234]

I added the above mentioned patch to local version of gcc. Compiling of gcc was successful. Compiling of my USB code was also successful. USB enumeration was also working, so interrupt handling (including WCH-Interrupt-fast) seems to work.

Yes, applying the patch manually is one way to do this but

1. As @cmuellner says, getting this patch accepted in the upstream GCC is unlikely to happen and
2. WCH have their own custom RISC-V GCC toolchain which may contain more mods than just this one so that's the toolchain that might be best suited for use with their RISC-V target.

BTW, judging by commentary elsewhere, even their mod for WCH-Interrupt-fast may not be totally correct/safe, particularly in the presence of the use of floating point code/registers and/or nested interrupts.

[capiman]

Thanks for all (and @cmuellner) your feedback!

I will follow @cmuellner suggestion. If I can help somewhere, I will do.

For the time being, till a better way is available, the local version must be sufficient.
(knowing it could fail even in near future)