Skip to content

crl-release-25.2: metamorphic and crossversion debugging improvements #4834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

RaduBerinde
Copy link
Member

metamorphic: user-friendly diff output

The diff output in a meta test failure is huge; once the histories
diverge, it's expected that a fraction of remaining operations will
differ.

This change shows only the first chunk of differences; it also
switches to showing diffs line-by-line instead of using unified diff
which groups differences into chunks.

Sample output:

===== DIFF =====
_meta/250604-110807.3903755035503/{standard-000,standard-025}
iter11.Prev("sziamgyg@5") // [valid,"vlqfkriwzhr",<no point>,["vlqfkriwzhr","yydpkltsctb")=>{"@11"="g","@9"="wbcwhbfszyji"}*] <nil> #802
db1.Set("yujhfwdgxzzk@9", "hgptsktbwcnkw") // <nil> #803
snap8.Get("cemsyb") // [""] pebble: not found #804
iter11.Last() // [true,"vlqfkriwzhr",<no point>,["vlqfkriwzhr","yydpkltsctb")=>{"@11"="g","@9"="wbcwhbfszyji"}] <nil> #805
db1.Set("bhxdce@12", "kpwvjwmvcjwxjtj") // <nil> #806
iter10.SetBounds("", "odueuzsefsqf@12") // <nil> #807
iter10.SeekLT("odueuzsefsqf@12", "") // [true,"nsgx@3","gdkyjtqkatm",<no range>] <nil> #808
snap9 = db1.NewSnapshot("bnbadmt", "kybn", "lnrdbtk", "pklhwdj", "rvhbyjcl", "ukvtn", "vlqfkriwzhr", "yshz") #809

-snap9.Get("rvhbyjcl@10") // ["thiwmgnem"] <nil> #810
+snap9.Get("rvhbyjcl@10") // ["thiwngnem"] <nil> #810

iter11.SetBounds("ppfgkaakurwg@6", "qjpjfqfggpj@2") // <nil> #811
iter11.SeekGE("ppfgkaakurwg@6", "") // [true,"ppfgkaakurwg@6",<no point>,["ppfgkaakurwg@6","qjpjfqfggpj@2")=>{"@11"="g","@1"="rh"}*] <nil> #812
iter11.Next("") // [false] <nil> #813
iter10.Prev("") // [true,"nsgx@4","khicwpgiqbgzhsdm",<no range>] <nil> #814
snap9.Get("iojegwnjm") // [""] pebble: not found #815
iter11.SeekGE("qwjzbcz@6", "") // [false] <nil> #816
iter10.Last() // [true,"nsgx@3","gdkyjtqkatm",<no range>] <nil> #817

metamorphic: crossversion debugging improvements

  • We now save the initial state directory too, which is necessary for
    reproduction.
  • We reformat test logs to make them more readable, especially when
    the paths are long.

metamorphic: show only the last part of the history

We now show the path to the history file and show only the last 30
lines of it.

scripts: run-crossversion-meta: use go 1.22 to build 24.1 test

crl-release-24.1 does not build with go 1.23+ because of an older
version of the swiss package.

scripts: run-crossversion-meta: use artifacts subdir

Currently crossversion artifacts go directly inside the crossversion
directory. They end up being gitignored (because they contain .test)
but they are hard to clean up and can't easily be configured to be
ignored by IDEs.

We now put everything in an artifacts subdir (already gitignored).

The diff output in a meta test failure is huge; once the histories
diverge, it's expected that a fraction of remaining operations will
differ.

This change shows only the first chunk of differences; it also
switches to showing diffs line-by-line instead of using unified diff
which groups differences into chunks.

Sample output:
```
===== DIFF =====
_meta/250604-110807.3903755035503/{standard-000,standard-025}
iter11.Prev("sziamgyg@5") // [valid,"vlqfkriwzhr",<no point>,["vlqfkriwzhr","yydpkltsctb")=>{"@11"="g","@9"="wbcwhbfszyji"}*] <nil> cockroachdb#802
db1.Set("yujhfwdgxzzk@9", "hgptsktbwcnkw") // <nil> cockroachdb#803
snap8.Get("cemsyb") // [""] pebble: not found cockroachdb#804
iter11.Last() // [true,"vlqfkriwzhr",<no point>,["vlqfkriwzhr","yydpkltsctb")=>{"@11"="g","@9"="wbcwhbfszyji"}] <nil> cockroachdb#805
db1.Set("bhxdce@12", "kpwvjwmvcjwxjtj") // <nil> cockroachdb#806
iter10.SetBounds("", "odueuzsefsqf@12") // <nil> cockroachdb#807
iter10.SeekLT("odueuzsefsqf@12", "") // [true,"nsgx@3","gdkyjtqkatm",<no range>] <nil> cockroachdb#808
snap9 = db1.NewSnapshot("bnbadmt", "kybn", "lnrdbtk", "pklhwdj", "rvhbyjcl", "ukvtn", "vlqfkriwzhr", "yshz") cockroachdb#809

-snap9.Get("rvhbyjcl@10") // ["thiwmgnem"] <nil> cockroachdb#810
+snap9.Get("rvhbyjcl@10") // ["thiwngnem"] <nil> cockroachdb#810

iter11.SetBounds("ppfgkaakurwg@6", "qjpjfqfggpj@2") // <nil> cockroachdb#811
iter11.SeekGE("ppfgkaakurwg@6", "") // [true,"ppfgkaakurwg@6",<no point>,["ppfgkaakurwg@6","qjpjfqfggpj@2")=>{"@11"="g","@1"="rh"}*] <nil> cockroachdb#812
iter11.Next("") // [false] <nil> cockroachdb#813
iter10.Prev("") // [true,"nsgx@4","khicwpgiqbgzhsdm",<no range>] <nil> cockroachdb#814
snap9.Get("iojegwnjm") // [""] pebble: not found cockroachdb#815
iter11.SeekGE("qwjzbcz@6", "") // [false] <nil> cockroachdb#816
iter10.Last() // [true,"nsgx@3","gdkyjtqkatm",<no range>] <nil> cockroachdb#817
```
 - We now save the initial state directory too, which is necessary for
   reproduction.
 - We reformat test logs to make them more readable, especially when
   the paths are long.
We now show the path to the history file and show only the last 30
lines of it.
`crl-release-24.1` does not build with go 1.23+ because of an older
version of the `swiss` package.
Currently crossversion artifacts go directly inside the crossversion
directory. They end up being gitignored (because they contain .test)
but they are hard to clean up and can't easily be configured to be
ignored by IDEs.

We now put everything in an `artifacts` subdir (already gitignored).
@RaduBerinde RaduBerinde requested a review from jbowens June 5, 2025 23:39
@RaduBerinde RaduBerinde requested a review from a team as a code owner June 5, 2025 23:39
@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jun 5, 2025

This change is Reviewable

@RaduBerinde
Copy link
Member Author

TFTR!

@RaduBerinde RaduBerinde merged commit fbf90d2 into cockroachdb:crl-release-25.2 Jun 6, 2025
4 checks passed
@cockroachdb cockroachdb deleted a comment from cockroach-teamcity Jun 6, 2025
@RaduBerinde RaduBerinde deleted the meta-debugging-improvements-25.2 branch June 6, 2025 15:45
@RaduBerinde RaduBerinde changed the title crl-release-25.2: scripts: run-crossversion-meta: use artifacts subdir crl-release-25.2: metamorphic and crossversion debugging improvements Jun 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants