core dump x86/aarch64 with gcc 11 and -O3 optimization #70
Description
On Rocky9 (Redhat 9 likely same), with GCC 11.4.1, a core dump occurs for simple ./t_coffee if built from source.
When setting "-O0" as CFLAGS in the Makefile, or make CFLAGS="-O0" the crash disappears.
Stepping through line by line in GDB, we end up returning from a function to the wrong address - which is a sign of stack corruption.
Adding -g to help, the -O3 -Wno-write-strings are defaults in the makefile.
make clean ; make -j 32 CFLAGS="-O3 -g -Wno-write-strings"
We see:
*************************************************************************************************
* FULL TRACE BACK PID: 6077
6077 -- ERROR: COREDUMP: T-COFFEE Version_13.46.0.919e8c6b (2023-07-07 22:06:42 - Revision 29996c5 - Build 980)
*************************************************************************************************
and then:
gdb ./t_coffee
[...]
r
[...]
-max_n_proc D [0] 1
Program received signal SIGSEGV, Segmentation fault.
store_string_variable (name=<optimized out>, v=<optimized out>, mode=<optimized out>) at util_lib/util.c:6178
6178 name_array[a][0]='\0';
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.34-100.el9_4.4.x86_64
(gdb) bt
#0 store_string_variable (name=<optimized out>, v=<optimized out>, mode=<optimized out>) at util_lib/util.c:6178
#1 0x00007fffffffa000 in ?? ()
#2 0x00007fffffff9ff8 in ?? ()
#3 0x00007fffffff9fe0 in ?? ()
#4 0x00007fffffff9fc4 in ?? ()
#5 0x00000000007a7540 in ?? ()
#6 0x0000000000000000 in ?? ()
quit)
A debugging session is active.
Inferior 1 [process 6109] will be killed.
The stack trace is corrupted.
Amongst the warnings that relate to stacks during compilation:
util_lib/util.c: In function ‘int get_cl_param(int, char**, FILE**, const char*, int*, const char*, int, int, const char*, ...)’:
util_lib/util.c:7903:9: warning: second parameter of ‘va_start’ not last named argument [-Wvarargs]
7903 | va_start (ap, usage);
| ^~~~~~~~
I believe this should be usage_in, but it doesn't help or change this issue.
Compiling with "-O1" or "-O0" the problem disappears.