Support IELR(1) Parser Generation #398

junk0612 · 2024-04-23T05:41:29Z

Support the generation of the IELR(1) parser described in this paper.
https://www.sciencedirect.com/science/article/pii/S0167642309001191

This PR will be ready for merging when the following is done.

Add tests
- Currently, I have only created a grammar file and verified its operation in my local environment.
Refactor
- The data structure is awful and needs to be improved because the priority was to achieve the expected results.

jasone · 2024-07-26T05:18:16Z

A couple months ago I poked around the internet looking for independent IELR(1) implementations, and besides lrama found only an inactive rust-based attempt. Recently I completed my own implementation and wrote a technical report with the hope that it might help others. IELR(1) turns out to be a real bear to figure out, even if it takes only a moderate amount of code. I hope you find the time and inspiration to complete this effort. Good luck!

junk0612 · 2024-11-14T05:41:30Z

On 7da9676 , Lrama can parse CRuby's parse.y at ruby/ruby@028958f .

These are the files of parsing results.
ruby path/to/id2token path/to/parse.y | bundle exec lrama -v - lalr
lalr.output

ruby path/to/id2token path/to/parse.y | bundle exec lrama -D lr.type=ielr -v - ielr
ielr.output

yui-knk · 2025-01-13T03:31:07Z

lib/lrama/states.rb

@@ -92,6 +92,20 @@ def compute
      report_duration(:compute_default_reduction) { compute_default_reduction }
    end

+    def compute_ielr


[note]: Ref "3.5.4. Algorithm" of the paper.

yui-knk · 2025-01-13T03:32:50Z

lib/lrama/states.rb

@@ -524,5 +541,52 @@ def compute_default_reduction
        end.first
      end
    end
+
+    def split_states
+      transition_queue = []


[nits]: It seems this variable is not used.

yui-knk · 2025-01-13T03:33:41Z

lib/lrama/states.rb

@@ -524,5 +541,52 @@ def compute_default_reduction
        end.first
      end
    end
+
+    def split_states


[note]: "Definition 3.45 (split_states)"

yui-knk · 2025-01-13T03:37:38Z

lib/lrama/state.rb

@@ -142,5 +160,274 @@ def rr_conflicts
        conflict.type == :reduce_reduce
      end
    end
+
+    def propagate_lookaheads(next_state)


[note]: Definition 3.40 (propagate_lookaheads)

yui-knk · 2025-01-13T03:41:49Z

lib/lrama/state.rb

+    def propagate_lookaheads(next_state)
+      next_state.kernels.map {|item|
+        lookahead_sets =
+          if item.position == 1


[note]: Corresponding to case 2 of Definition 3.40 because position is 0-origin in this code base.

yui-knk · 2025-01-13T03:44:35Z

lib/lrama/state.rb

+    def propagate_lookaheads(next_state)
+      next_state.kernels.map {|item|
+        lookahead_sets =
+          if item.position == 1


[nits]: I personally like to make the condition order to be aligned with the original paper for readability of the algorithm. For example

if item.position > 1 ... else ... end

yui-knk · 2025-01-13T03:50:11Z

lib/lrama/state.rb

+          if item.position == 1
+            goto_follow_set(item.lhs)
+          else
+            kernel = kernels.find {|k| k.predecessor_item_of?(item) }


[note]: C[k] is the kernel (1) whose LHS and RHS are same with the item's one and (2) whose position is item.position - 1. #predecessor_item_of? is a method for such calculation.

yui-knk · 2025-01-13T03:55:20Z

lib/lrama/state.rb

+            item_lookahead_set[kernel]
+          end
+
+        [item, lookahead_sets & next_state.lookahead_set_filters[item]]


[note]: This is same with the definition case 1 because case 1 requires tokens which are in both lookahead_set_filters of s' kernel and item_lookahead_sets of s kernel.

yui-knk · 2025-01-13T04:05:00Z

lib/lrama/states.rb

@@ -92,6 +92,20 @@ def compute
      report_duration(:compute_default_reduction) { compute_default_reduction }
    end

+    def compute_ielr
+      report_duration(:split_states) { split_states }
+      report_duration(:compute_direct_read_sets) { compute_direct_read_sets }


[note]: Compute LA set again, once states are splitted. See Phase 4 and Phase 5 on "3.1. Overview".

yui-knk · 2025-01-13T04:10:41Z

lib/lrama/state.rb

+      !@item_lookahead_set.nil?
+    end
+
+    def compatible_lookahead?(filtered_lookahead)


[note]: This method is same with "Definition 3.43 (is_compatible)".

[nits]: If so, I prefer to use is_compatible? as a method name for clarity.

junk0612 force-pushed the generate_ielr_parser branch from 2d975c5 to a49fe82 Compare June 25, 2024 17:19

junk0612 force-pushed the generate_ielr_parser branch from a49fe82 to 604737d Compare September 4, 2024 06:28

junk0612 force-pushed the generate_ielr_parser branch 7 times, most recently from b3078e1 to 0a300f7 Compare September 25, 2024 18:33

junk0612 marked this pull request as ready for review September 25, 2024 18:33

junk0612 requested review from ydah and yui-knk September 25, 2024 18:33

Set %define when parsing grammar files

438b88d

junk0612 force-pushed the generate_ielr_parser branch from 0a300f7 to 0ba3241 Compare October 11, 2024 09:42

junk0612 added 5 commits November 14, 2024 14:10

Parse --define options

79c69cd

Support IELR(1) parser generation

a786828

Add a integration test case

222b252

Optimize calculating predecessors

0a03198

Check existence of contributions

7da9676

junk0612 force-pushed the generate_ielr_parser branch from 9bf8584 to 7da9676 Compare November 14, 2024 05:10

yui-knk reviewed Jan 13, 2025

View reviewed changes

yui-knk mentioned this pull request Jan 13, 2025

v0.7.0 #506

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support IELR(1) Parser Generation #398

Support IELR(1) Parser Generation #398

junk0612 commented Apr 23, 2024 •

edited

Loading

jasone commented Jul 26, 2024

junk0612 commented Nov 14, 2024 •

edited

Loading

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025 •

edited

Loading

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025 •

edited

Loading

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025

yui-knk Jan 13, 2025 •

edited

Loading

yui-knk Jan 13, 2025

Support IELR(1) Parser Generation #398

Are you sure you want to change the base?

Support IELR(1) Parser Generation #398

Conversation

junk0612 commented Apr 23, 2024 • edited Loading

jasone commented Jul 26, 2024

junk0612 commented Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

yui-knk Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yui-knk Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yui-knk Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

junk0612 commented Apr 23, 2024 •

edited

Loading

junk0612 commented Nov 14, 2024 •

edited

Loading

yui-knk Jan 13, 2025 •

edited

Loading

yui-knk Jan 13, 2025 •

edited

Loading

yui-knk Jan 13, 2025 •

edited

Loading