Doesn't encode empty arrays

[arp242]

For example:

puts PerfectTOML.generate({k: [1]})
# Output: k = [1]

Okay, so that's fine, but with an empty array there is no output, and the key is skipped:

puts PerfectTOML.generate({k: []})
# No output

---

This is a failing test case in toml-test; right now it uses toml-test 1.2 from 2022: https://github.com/mame/perfect_toml/blob/main/Rakefile#L11

Since then quite a few new tests have been added.

There are also some "invalid TOML" test cases it fails, but these are less serious. For the full output see: https://toml-lang.github.io/toml-test-matrix/output/ruby-perfect_toml.html

[mame]

@arp242 Thank you! I think I fixed the issue of empty arrays.

I'd like to support Time range check too, but it may take time because Ruby itself has some issues: https://bugs.ruby-lang.org/issues/21301

I have a question for you about carriage return in multi-line strings. The TOML 1.0.0 specification says in the context of multi-line basic strings:

Any Unicode character may be used except those that must be escaped: backslash and the control characters other than tab, line feed, and carriage return ( U+0000 to U+0008, U+000B, U+000C, U+000E to U+001F, U+007F).

To me this reads just "carriage return may be used". Is there a statement anywhere that says it must accept only if it comes up as CR+LF?

[arp242]

That was clarified at some point; in the latest version it says:

Any Unicode character may be used except those that must be escaped: backslash and the control characters other than tab, line feed, and carriage return (U+0000 to U+0008, U+000B, U+000C, U+000E to U+001F, U+007F). Carriage returns (U+000D) are only allowed as part of a newline sequence.

[mame]

@arp242 Thank you, but with respect, what exactly do you mean by "the latest version"? As far as I can see, https://toml.io/en/v1.0.0 is the latest.

I've reviewed the contents of that page carefully, and I still don't see how it could be interpreted as rejecting a CR not followed by LF. Even if that wasn't the author's intent, I believe the specification should be interpreted as written.

Wouldn't it make more sense to update toml-test's expected values instead?

[arp242]

Sorry, I meant the latest main branch at https://github.com/toml-lang/toml: the upcoming v1.1.0. Not the latest released version.

[mame]

Thank you. According to toml-test --help, it is for TOML 1.0 by default.

    -toml         Select TOML version to run tests for. Supported versions are
                  "1.0" and "1.1" (which isn't released yet and may change).
                  Defaults to 1.0.

If TOML 1.0.1 is released and that part is changed, then I can understand the presence of this test. Until then, I believe it would be better to either change the expected value or, at the very least, exclude the test from the default set.

[arp242]

Which tests are we talking about exactly? I don't really recall adding tests for CRLF in multiline strings off-hand? Actually, adding test for that is blocked on some improvements in toml-test itself: toml-lang/toml-test#166

---

However, realistically, people have been writing TOML files with CRLF for a long time, and CRLF was always explicitly allowed everywhere else ("Newline means LF (0x0A) or CRLF (0x0D 0x0A)").

It was awkwardly phrased in the specification for multiline strings. It also refers to "newline" (defined as LF or CRLF) in the section, and mentions that "TOML parsers should feel free to normalize newline to whatever makes sense for their platform". It seems to me the "common sense reading" is that the intent was always to allow CRLF there as well: "allow CRLF everywhere except multiline strings" doesn't make much sense, especially with the explicit reference to normalisation. The change was just to clarify that (i.e. fix awkward/conflicting wording) rather that than allowing something that wasn't previously allowed.

But like I said: I don't know which tests you're having trouble with? I try to be conservative in tests and always happy to modify things, but in general CRLF in multiline strings should probably be tested, even for 1.0? De-facto you will encounter these documents in the real world, and AFAIK it's not the case that implementations commonly fail on it.

[mame]

Sorry, I am talking about tests where a CR not followed by LF appears inside a multi-line string. Specifically, these two:

https://github.com/toml-lang/toml-test/blob/main/tests/invalid/control/multi-cr.toml
https://github.com/toml-lang/toml-test/blob/main/tests/invalid/control/rawstring-cr.toml

I believe these are valid TOML 1.0.0 strings.

The specification states, "Newline means LF (0x0A) or CRLF (0x0D 0x0A)". So a CR not followed by LF is considered as a simple Unicode character, not a newline, and it is not subject to newline-related rules.

[mame]

Sorry, the second link was incorrect. The correct one is:

https://github.com/toml-lang/toml-test/blob/main/tests/invalid/control/rawmulti-cr.toml

(I have no objection to rawstring-cr.toml being invalid.)