Improve clang AST parsing #226
Open
+1,738
−536
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
draft because of the numbered renames - need some help with these in case we want to merge this |
mappu
reviewed
May 26, 2025
To prepare genbindings for accepting arbitrary libraries from the command line, we must retain parse types not only from the header being processed but also its includes, to understand things like type hierarchies etc without depending on a particular module order. This is the first baby step that addresses clang AST filtering. The change has some collateral benefits: the improved accuracy of the AST parsing keeps more relevant information but at the same time reduces the memory footprint since filtering is done streaming instead of loading everything into memory first (ditto when writing the cache). A consequence is that a lot more clang processes can run in parallel without OOMing. * parse "file" correctly to discover type provenance - using the standard go json parser for this does not work since the depend on serialization order which go discards (see https://github.com/dtolnay/clang-ast?tab=readme-ov-file#source-locations) * stream clang output to JSON decoder and stream-write the cache to reduce memory footprint * with the newfound memory efficiency, bump up the number of parallel jobs and use threads for the parsing as well * fix missing classes such as QSysInfo using the corrected `file` source location field * don't try to re-run clang if it fails (OOM is unlikely anyway)
arnetheduck
force-pushed
the
miqt-ast-full
branch
from
d564893 to
6de4e76
Compare
arnetheduck
commented
May 28, 2025
| @@ -29,13 +29,11 @@ typedef struct QVariant QVariant; | |||
| #endif | |||
|
|
|||
| QQmlListReference* QQmlListReference_new(); | |||
| QQmlListReference* QQmlListReference_new2(QVariant* variant); | |||
There was a problem hiding this comment.
the interesting thing about these removed overloads is that 3 and 4 are the same .. 🤷
|
the qmllist changes look legit as far as I can tell, so undrafting |
mappu
reviewed
Jun 11, 2025
|
ping @mappu anything more needed here? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To prepare genbindings for accepting arbitrary libraries from the command line, we must retain parse types not only from the header being processed but also its includes, to understand things like type hierarchies etc without depending on a particular module order.
This is the first baby step that addresses clang AST filtering.
The change has some collateral benefits: the improved accuracy of the AST parsing keeps more relevant information but at the same time reduces the memory footprint since filtering is done streaming instead of loading everything into memory first (ditto when writing the cache).
A consequence is that a lot more clang processes can run in parallel without OOMing.
https://github.com/dtolnay/clang-ast?tab=readme-ov-file#source-locations)
filesource location field