Description
The current compiler is implemented as follows:
- [Python] Verilog and Perl style text preprocessing
- [C++] Antlr-based lexer and parser to create parse tree
- [C++/Python] Translate parse tree back to Python objects
- [Python] Compile RDL structural tree from parse tree
- [Python] Propagate parameters and elaborate expressions
- [Python] Validate design
- [Python] API to exporters
This approach has worked decently, but clearly shows its limitations in performance with larger designs - Bulk data operations on text and data structures in Python simply are never going to be fast.
Since the internals of the RDL compiler are generally opaque to users, I think it is time to consider rewriting this to be primarily implemented in a compiled language, with a thin layer that provides an API to Python.
Still TBD which compiled language I want to use (C++? Rust? Other?) - whichever minimizes maintenance pain without sacrificing performance.
High level goals:
- Make as much of the process implemented in a compiled language
- Preserve as much of the existing Python API as practical such that this change is totally transparent to the user
- Likely lean more into the Node overlay, and deprecate the ability/necessity to reach into "Node.inst", etc.
- Plan to also provide a "native" language API in the same compiled language so that users can interact with the compiler API performantly without jumping through Python
- Do not bother providing a pure-Python implementation anymore. Now that I'm comfortable with publishing ABI3 wheels, having a fall-back is not particularly interesting anymore.
Note to readers: This will be a significant effort that may take a while to implement. Low-value comments "reminding" me to improve this will be ignored/deleted.