[Feature Request] Allow predicate code blocks to modify the 'expected' field of the parserError{}

[wolfwfr]

First things first

First let me say thanks to all contributors for your great work.
Pigeon has been a great tool and many times have I found myself appreciating the crystal clear documentation!

However, I might have a request.

Request

I'm working on a project where I'm trying to use go constants for input matching in pigeon.
For example:

// definitions.go
package definitions

const (
  plus = "+"
  percent = "%"
)
func Operators() []string { return []string{plus, percent} }

// grammar.pigeon
...
op <- [^ \n\t\ra-zA-Z0-9]+
&{ return helpers.Contains(definitions.Operators(), string(c.text)), nil }
{ return string(c.text), nil }
...

I'm trying to avoid having to repeat all the go definitions in pigeon, so I match liberally on the regex and then narrow it down within the predicate code block.

This works like a charm for parsing valid inputs and correctly returns errors on invalid inputs.

However, the suggestions made in the expected field of the parserError are purely based on the regex and I cannot modify these suggestions through the predicate code block. Since the predicate block is more restrictive than the regex, I'd like to be able to return custom suggestions.

What I tried

I have tried to return errors from the predicate code block, but this does not have the same effect (when returning false, err, the resulting errors of Parse(...) still contains the regex suggestion in the expected block and the predicate block must return false to allow potential subsequent rules to try and match the input).

I have also tried using the labeled failures, but find them hard to work with in more complicated grammar scenarios with many rules.

Conclusion

Since the predicate and regex both affect the outcome of an attempted match, I thought it might be sensible to provide more control in the predicate code block such that it can return its own suggestions in the expected field of a parserError.

For example:

// grammar.pigeon
...
op <- [^ \n\t\ra-zA-Z0-9]+
&{ return helpers.Contains(definitions.Operators(), string(c.text)), definitions.Operators(), nil } // returning match-result (bool), expected ([]string), err (error)
{ return string(c.text), nil }

[breml]

Hi @wolfwfr

Thanks for your detailed issue report. My initial reflex was to suggest labeled failures for your case, but I understand, that you have already tried those and you found it hard to work with them. I guess you have already consulted the provided examples as well.

In regards to you proposal, the issue is, that this is not backwards compatible. In order to consider a change to the way predicate code blocks work, we would need to find an API, that is backwards compatible with existing grammars.

[wolfwfr]

Hi @breml,

Thank you for the reply.

I understand your point. Since this concerns mostly a quality of life issue and is, at this time, not blocking my project, I'll leave it here.

Perhaps my question can be considered if pigeon ever gets a v2 version.

Kind regards

[fy0]

my fork supports this:

// peg
{
//nolint:unreachable
package main

{{ if not .grammarOnly }}

type ParserCustomData struct {
}

const (
  plus = "+"
  percent = "%"
)
func Operators() []string { return []string{plus, percent} }

func ContainsInSlice(slice []string, item string) bool {
	return strings.Contains(strings.Join(slice, ""), item)
}
{{ end }}
}

op <- val:<[^ \n\t\ra-zA-Z0-9]+>
	&{ return ContainsInSlice(Operators(), val.(string)) }
	{ return string(c.text) }

generate:

.\pigeon.exe -nolint -optimize-parser -optimize-ref-expr-by-index -o predicate_code.peg.go predicate_code.peg

// main.go

package main

import "fmt"

func main() {
	p := newParser("", []byte("%"))
	fmt.Println(p.parse(nil))

	p = newParser("", []byte("-"))
	fmt.Println(p.parse(nil))
}

result:

% <nil>
<nil> 1:2 (1): no match found, expected: [^ \n\t\ra-zA-Z0-9]

https://github.com/fy0/pigeon