Error using pycparser with C code that calls OpenSSL

[vm2p]

Hi,

I'm trying to use pycparser with an application that uses OpenSSL. Concretely, I'm using the OpenSSL digest demo in https://github.com/openssl/openssl/blob/master/demos/digest/EVP_MD_xof.c.

I'm pre-processing the file with the command gcc -I../../include -I../../../pycparser/utils/fake_libc_include -E EVP_MD_xof.c, where the first include refers to the OpenSSL libraries and the second include refers to pycparser itself. Then, when trying to parse the file using the parse_file command, I get the error

Traceback (most recent call last):
  File "/Users/vm2p/Documents/repositories/pycparser/examples/rewrite_ast.py", line 25, in <module>
    ast = parse_file("/Users/vm2p/Documents/repositories/openssl/demos/digest/test.c", use_cpp=True)
  File "/usr/local/lib/python3.9/site-packages/pycparser/__init__.py", line 90, in parse_file
    return parser.parse(text, filename)
  File "/usr/local/lib/python3.9/site-packages/pycparser/c_parser.py", line 147, in parse
    return self.cparser.parse(
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/usr/local/lib/python3.9/site-packages/pycparser/c_parser.py", line 1931, in p_error
    self._parse_error(
  File "/usr/local/lib/python3.9/site-packages/pycparser/plyparser.py", line 67, in _parse_error
    raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: ../../include/openssl/safestack.h:205:256: before: (

I opened the safestack.h file from the OpenSSL distribution and between lines 250 and 256 there are a bunch of macro definitions. I've attached such file to this issue. Is this something that is not supported by pycparser? Or am I doing something wrong? I also attached the EVP_MD_xof.c as test.c.

I also tried to pre-process the file using just gcc -I../../include -E EVP_MD_xof.c, but then I get

Traceback (most recent call last):
  File "/Users/vm2p/Documents/repositories/pycparser/examples/rewrite_ast.py", line 25, in <module>
    ast = parse_file("/Users/vm2p/Documents/repositories/openssl/demos/digest/test.c", use_cpp=True)
  File "/usr/local/lib/python3.9/site-packages/pycparser/__init__.py", line 90, in parse_file
    return parser.parse(text, filename)
  File "/usr/local/lib/python3.9/site-packages/pycparser/c_parser.py", line 147, in parse
    return self.cparser.parse(
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/local/lib/python3.9/site-packages/pycparser/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/usr/local/lib/python3.9/site-packages/pycparser/c_parser.py", line 1931, in p_error
    self._parse_error(
  File "/usr/local/lib/python3.9/site-packages/pycparser/plyparser.py", line 67, in _parse_error
    raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/i386/_types.h:98:27: before: __darwin_va_list

The resulting file is also attached as test2.c.

Thank you in advance!

Archive.zip
.

[eliben]

Please minimize the example to pinpoint which specific line of preprocessed C code pycparser chokes on.

[vm2p]

pycparser chokes on line 205 of the safestack.h file:

SKM_DEFINE_STACK_OF_INTERNAL(OPENSSL_STRING, char, char)

where

# define SKM_DEFINE_STACK_OF_INTERNAL(t1, t2, t3) \
    STACK_OF(t1); \
    typedef int (*sk_##t1##_compfunc)(const t3 * const *a, const t3 *const *b); \
    typedef void (*sk_##t1##_freefunc)(t3 *a); \
    typedef t3 * (*sk_##t1##_copyfunc)(const t3 *a); \
    static ossl_unused ossl_inline t2 *ossl_check_##t1##_type(t2 *ptr) \
    { \
        return ptr; \
    } \
    static ossl_unused ossl_inline const OPENSSL_STACK *ossl_check_const_##t1##_sk_type(const STACK_OF(t1) *sk) \
    { \
        return (const OPENSSL_STACK *)sk; \
    } \
    static ossl_unused ossl_inline OPENSSL_STACK *ossl_check_##t1##_sk_type(STACK_OF(t1) *sk) \
    { \
        return (OPENSSL_STACK *)sk; \
    } \
    static ossl_unused ossl_inline OPENSSL_sk_compfunc ossl_check_##t1##_compfunc_type(sk_##t1##_compfunc cmp) \
    { \
        return (OPENSSL_sk_compfunc)cmp; \
    } \
    static ossl_unused ossl_inline OPENSSL_sk_copyfunc ossl_check_##t1##_copyfunc_type(sk_##t1##_copyfunc cpy) \
    { \
        return (OPENSSL_sk_copyfunc)cpy; \
    } \
    static ossl_unused ossl_inline OPENSSL_sk_freefunc ossl_check_##t1##_freefunc_type(sk_##t1##_freefunc fr) \
    { \
        return (OPENSSL_sk_freefunc)fr; \
    }