Syntax highlighting - eyes needed

[finanalyst]

@Raku/core
We have changed the syntax highlighter from one developed for the atom editor (atom-perl6-highlighter) to a Raku based one (Rainbow).

• colouration request. Rainbow offers more categories that can be colourised, eg three separate regex categories, separate delimiter categories.
• The colours chosen for the moment are the same as before, but the same colour is used for duplicate categories. Ideally each category should be a different colour (more generally styling).
• Although the styling of each category is done via CSS, categories can be styled using colour, font, font decoration etc. However, that would need a bit more refactoring of the SCSS, eg adding variables for font as well as colour.
• Please offer suggestions. (Values are needed for the light and dark themes separately)
• concrete suggestions can be made using a PR changing the following files:
  • <repo root>/Website/plugins/rainbow/scss/rainbow-light.scss
  • .../scss/rainbow-dark.scss
  • the relevant colour values for the highlighting are assigned to SCSS variables $code-..... and start at about line 15.

Known problems:

• when the RakuDoc source has =for code :allow<C> or other value in 'allow', the :allow option is not respected.
  • I think that only the Pod.rakudoc file still retains ':allow'
  • in the discussions about RakuDoc v2, the allow option is difficult to implement in combination with syntax highlighting, and needs to be handled separately. This will be developed in due course. The problem existed with the previous highlighter too.
• Rainbow does not handle metadata options in RakuDoc blocks correctly as yet. The priority for addressing this flaw is considered low at present.
  • It is not known yet whether this affects any code snippets in the main sources. Help to determine this would be appreciated.

[patrickbkr]

• in the discussions about RakuDoc v2, the allow option is difficult to implement in combination with syntax highlighting, and needs to be handled separately. This will be developed in due course. The problem existed with the previous highlighter too.

Actually I've already done something similar in Rainbow with quote adverbs already (see the Adverbs section here). This shouldn't be too hard to get right.
Can you provide a quick primer how this option is meant to work exactly? (Ideally create a ticket.)

[finanalyst]

Quick smartphone response to Patrick. ``` =for code :allow<B I> my $var = 'filename.conf'.B<IO>.I<slurp>(:close); my $strng = 'This is B<bold> but this is NOT U<underlined>'; Normal para in Rakudoc source. ```

…

On Mon, 11 Nov 2024, 13:10 Patrick Böker, ***@***.***> wrote: - in the discussions about RakuDoc v2, the allow option is difficult to implement in combination with syntax highlighting, and needs to be handled separately. This will be developed in due course. The problem existed with the previous highlighter too. Actually I've already done something similar in Rainbow with quote adverbs already (see the Adverbs section here <https://git.sr.ht/~patrickb/Rainbow/tree/main/item/t/test-data/string.rakumod>). This shouldn't be too hard to get right. Can you provide a quick primer how this option is meant to work exactly? (Ideally create a ticket.) — Reply to this email directly, view it on GitHub <#419 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACYZHGOIAJ63GZC7CBKDDT2ACUDJAVCNFSM6AAAAABRO7STZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRYGE2DGOJWG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[habere-et-dispertire]

The comment in the last example of the leftward feed operator ( docs.raku.org ) seems peculiar :

my @result
    <== sort()
    <== grep({ /<[PE]>/ })
    <== my @caps            # unlike ==>, there's no need for additional statement
    <== map({ .tc })
    <== <people of earth>;

[habere-et-dispertire]

The last example in Literal Strings: Q ( docs.raku.org ) seems unaware of a unicode quoting character｟ -- FULLWIDTH LEFT WHITE PARENTHESIS :

Q｢｢Delimiters can be repeated/nested if they are adjacent.｣｣
Q｟Quoting with fancy unicode pairs｠

[finanalyst]

@habere-et-dispertire Thanks for the issues.
@patrickbkr Why is a comment contents being treated as Raku code? I thought it was just a string.

[finanalyst]

@habere-et-dispertire Your second question may be due to interpolation??
@lizmat : When an open punctuation is repeated, does this trigger interpolation, in the same way as << triggers interpolation, while < does not?

[habere-et-dispertire]

@habere-et-dispertire Your second question may be due to interpolation??

If I understand correctly, then it is being interpolated when it shouldn't. Note that the character is not being repeated -- it is a single character :

say '｟'.chars
# 1

say '｟'.uniname.say
# FULLWIDTH LEFT WHITE PARENTHESIS

say '｟'.ord.base(16)
# FF5F

Rainbow appears aware of this to the degree that it includes it as an open-close-punct under hex value FF5F.

I'd hazard a guess that we need a case for capital Q ( git.sr.ht ) here :

given $<q-letter> {
    when "q" { $*quote-mode.q = True; }
    when "qq" | "qqw" | "qqww" | "qqx" {
        $*quote-mode.s = True;
        $*quote-mode.a = True;
        $*quote-mode.h = True;
        $*quote-mode.f = True;
        $*quote-mode.c = True;
        $*quote-mode.b = True;
    }
#    ...

[habere-et-dispertire]

At the end of the interpolation basics page, there is a final colon suggestive of a missing example to follow ?

[finanalyst]

@habere-et-dispertire @coke The : appears to be a mistake in the content. I think it is a .

[patrickbkr]

@finanalyst I've now released Rainbow 0.3.0. That improves the RakuDoc support tremendously, including =begin code :allow<B>. There are now also additional tokenizer subs to tokenize other languages (regex, string, rakudoc).

[timo]

Here, a screenshot from this page, near the end of the linked section: https://docs.raku.org/language/containers#Flattening,_items_and_containers

the highlighting of the decont operator used on $x looks odd, this one: $x<>

i think the two angle brackets should at least be the same color. not sure what exactly they should be highlighted as.

[patrickbkr]

On December 25, 2024 10:50:19 PM GMT+01:00, timo ***@***.***> wrote: Here, a screenshot from this page, near the end of the linked section: https://docs.raku.org/language/containers#Flattening,_items_and_containers  the highlighting of the decont operator used on $x looks odd, this one: `$x<>` i think the two angle brackets should at least be the same color. not sure what exactly they should be highlighted as.

That's a Rainbow issue. I'll take it. Cross posted here: <https://todo.sr.ht/~patrickb/Rainbow/4>