Skip to content

Performance: Rewrite compiler implementation in compiled language #268

Open
@amykyta3

Description

@amykyta3

The current compiler is implemented as follows:

  1. [Python] Verilog and Perl style text preprocessing
  2. [C++] Antlr-based lexer and parser to create parse tree
  3. [C++/Python] Translate parse tree back to Python objects
  4. [Python] Compile RDL structural tree from parse tree
  5. [Python] Propagate parameters and elaborate expressions
  6. [Python] Validate design
  7. [Python] API to exporters

This approach has worked decently, but clearly shows its limitations in performance with larger designs - Bulk data operations on text and data structures in Python simply are never going to be fast.

Since the internals of the RDL compiler are generally opaque to users, I think it is time to consider rewriting this to be primarily implemented in a compiled language, with a thin layer that provides an API to Python.
Still TBD which compiled language I want to use (C++? Rust? Other?) - whichever minimizes maintenance pain without sacrificing performance.

High level goals:

  • Make as much of the process implemented in a compiled language
  • Preserve as much of the existing Python API as practical such that this change is totally transparent to the user
    • Likely lean more into the Node overlay, and deprecate the ability/necessity to reach into "Node.inst", etc.
  • Plan to also provide a "native" language API in the same compiled language so that users can interact with the compiler API performantly without jumping through Python
  • Do not bother providing a pure-Python implementation anymore. Now that I'm comfortable with publishing ABI3 wheels, having a fall-back is not particularly interesting anymore.

Note to readers: This will be a significant effort that may take a while to implement. Low-value comments "reminding" me to improve this will be ignored/deleted.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions