Support handling ```#define``` guard

[andkae]

Hello Together,

at the moment i try to parse a file with an #define guard:

from dissect.cstruct import cstruct

cdef = """
/* Define Guard */
#ifndef __MYGUARD
#define __MYGUARD

typedef struct myStruct
{
    char        charVal[16];
}

#endif // __MYGUARD
"""
cs = cstruct()
cs.load(cdef)

This leads to the error unexpected token <Token.IDENTIFIER value='ifndef':

    cs.load(cdef)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/dissect/cstruct/cstruct.py", line 264, in load
    TokenParser(self, **kwargs).parse(definition)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/dissect/cstruct/parser.py", line 386, in parse
    raise ParserError(f"line {self._lineno(token)}: unexpected token {token!r}")
dissect.cstruct.exceptions.ParserError: line 3: unexpected token <Token.IDENTIFIER value='ifndef'>

How can i handle such a message?

Thanks for your support.

BR,
Andreas

[Schamper]

Hi Andreas,

dissect.cstruct only supports a subset of C language syntax, so unfortunately we do not (yet) support #ifdef/#ifndef/#endif. If you're up for it, we would welcome a PR for it. Otherwise, you'll have to remove those statements from your definition.

[andkae]

Thanks for clarification :).

welcome a PR for it

Let me check what i can do. But no promise.

Thanks.

BR,
Andreas

[andkae]

Hi @Schamper,
i dived a little in the code and i believe it needs a little disccussion the ideas.

1. One solution could be the filter out such constructs like the comments:

dissect.cstruct/dissect/cstruct/parser.py

Line 328 in 8bb71c9

def _remove_comments(string: str) -> str:

There could be a function realized like _remove_guard.

2. The other way would be to parse such structs, and add it to the token list:

https://github.com/fox-it/dissect.cstruct/blob/main/dissect/cstruct/parser.py#L373-L384

here you could add the token IFNDEF. At the moment i have for this way no idea in which way i should sort in the parsed data.

What do you think?

Thanks.

BR,
Andreas

[Schamper]

I think option 2 is the way to go, since we can't just blanket remove any conditional block, we need to evaluate #define's in order to know what is and isn't defined. Unfortunately the whole "parser" is fairly complicated, but I think something like the following might work:

• Add tokens for #ifdef, #ifndef and #endif
• If we see #ifdef or #ifndef in the parsing loop, do the following:
  • Extract the constant we need to evaluate
  • Evaluate if the constant is defined
  • Add the result to a stack/list, e.g. self._conditionals
• Adjust the aforementioned parsing loop so that it checks if we're currently in any "conditional body" (e.g. self._conditionals is not empty)
• If we are, evaluate the most recent result, and either disregard any token we encounter if we need to skip this "body", or handle it like normal
• If we encounter a #endif token, check if it's valid (i.e. there are conditionals to terminate, self._conditionals is not empty), and pop from the stack
• If the stack is empty, we just handle tokens like normal

I hope I didn't overlook anything, but I think this might be the most robust way to handle this. Let me know what you think!

[akaeba]

Hi @Schamper sounds valid. How should look then the result, for instance of the following code?

from dissect.cstruct import cstruct
cdef = """
#ifndef __MYGUARD
#define __MYGUARD
  //...
#endif // __MYGUARD
"""
cs = cstruct()
cs.load(cdef)
print(cs.conditionals)

In that way?
[__MYGUARD]

Another construct what encounters in my code looks like that:

/* C++ compatibility */
#ifdef __cplusplus
extern "C"
{
#endif // __cplusplus
 //....
#ifdef __cplusplus
}
#endif // __cplusplus

For the second construct would a list in a list perhaps the right datastruture?

Thanks for your comments.

BR,
Andreas