Skip to content

Implement Just-in-Time concrete syntax trees #816

@MicahGale

Description

@MicahGale

Is your feature request related to a problem? Please describe.

For improved performance we are looking into Just-in-time parsing #529. For a long time we have assumed that parsing fully concrete syntax trees is slower than if we parsed abstract syntax trees. The reason we need concrete syntax trees is for if the object is edited, and the input must be edited. (shameless plug). We also posit that under the vast majority of use cases that not all objects will be edited.

In the JIT parsing implementation there are events that could trigger a mass parsing that will be very expensive, such as accessing surface.cells which will require parsing every cell and checking if the surface in question is used in that cell's geometry. This can be mitigated by greping.

Describe the solution you'd like

Another option is that when an object is parsed it is parsed as an abstract syntax tree by default. Only when that syntax tree is edited will it be re-parsed as a concrete one. This re-parsing would also have to be triggered whenever .comments is accessed.

I imagine doing the following:

  1. triggered a concrete syntax tree when a ValueNode is changed and it's detected via the format mechanism. Probably the easiest way to do this might be to raise an exception that triggers this re-parse. How to handle this properly with recursive structures is tbd.
  2. Add a needs_concrete_tree decorator that triggers full parses like the needs_full_tree in Add support for Just-In-Time parsing #529.
  3. Add a mix-in to simplify the two parser divergence. I imagine one mixin would do the normal:
@_("NUMBER padding")
def number_phrase(...

The abstract one would just do:

@_("NUMBER")
def number_phrase(...

Care would have to be taken that these two mixins don't try to re-write the rules for the same functions inside of sly though, and may require a vendoring, or some meta-class nonsense. We might do some meta-programming to generate the permutations of all parsers.

Describe alternatives you've considered

Just give up on sly #432, and go to a high-performance parsing engine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestAn issue that improves the user interface.parsers are hardExamples of where MCNP syntax is complicated and should be simplified.performance 🐌Issues related to speed and memory

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions