Segfault when dumping a particular json document

[ubergeek42]

We encountered an issue where the library segfaults when writing out a particular json document that contains deeply nested hashes. It seems to be quite an edge case, and I wasn't able to have much luck narrowing down the document to a minimal reproducer. But I can provide a sanitized document that reproduces the issue consistently on my machine (libc linux amd64). This issue happened in production when called via multijson (which is where the arguments come from for indent/space/etc) in an alpine(musl) ruby container. It does not seem to trigger unless require "oj_mimic_json" is also present. A colleague was unable to reproduce this specific crash on an M-series macbook. If you play with the length of data before the deep nested object you can either remove the crash or change the message that prints from realloc to malloc. Additionally changing the indent/space/object_nl/array_nl arguments to different length strings will "resolve" the issue. So I guess there must be some memory corruption happening.

If there's any other data I can provide please let me know.

Reproducer:

$ bundle exec ruby -e 'require "oj"; require "oj_mimic_json"; Oj.dump(Oj.load(File.read("oj-crash-sanitized.json")), indent: "  ", space: " ", object_nl: "\n", array_nl: "\n");'
realloc(): invalid next size
Aborted                    (core dumped) bundle exec ruby -e 'require "oj"; require "oj_mimic_json"; Oj.dump(Oj.load(File.read("oj-crash-sanitized.json")), indent: "  ", space: " ", object_nl: "\n", array_nl: "\n");'

Environment details:

$ bundle exec ruby -e "require 'oj'; print(Oj::VERSION)"
3.16.11
$ ruby -v
ruby 3.4.4 (2025-05-14 revision a38531fd3f) +PRISM [x86_64-linux]
$ cat Gemfile
source "https://rubygems.org"

gem "oj"
gem "oj_mimic_json"

Sanitized reproducer document `oj-crash-sanitized.json`

{
    "aaaaa": [
        {
            "aaaaaaaaaaaaaaaaaaa": true,
            "bbbbbbbbbbbbbbbbbbb": "aaaaaaaaaaaaaaaaaaaaaaa"
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaaaaa"
        },
        {
            "aaaaaa": "aaaaaaa",
            "bbbbbb": "aaaaa",
            "ccccccc": 60,
            "dddddddddd": true
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaa"
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaaaaa",
            "bbbbbb": "aaa",
            "ccccccc": 60,
            "dddddddddd": true
        },
        {
            "aaaaaaaaaaaaaaaaa": true,
            "bbbbbbbbbbbbbbbbbbb": "aaaaaaaaaaaaaaaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaaaa",
            "bbbbbb": "aaaaaaaaaaaaaaaaaa",
            "ccccccccc": "aaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": "aaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaa",
            "bbbbbbb": 5
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": "aaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": "aaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaaaaaaaaaaaaaaa": true,
            "bbbbbbbbbbbbbbbbbbb": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaaa"
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaa",
            "bbbbbbb": 3
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaa",
            "bbbbbbb": 5
        },
        {
            "aaaa": false,
            "bbbbbb": "aaaaa",
            "cccccc": "aaaa",
            "dddddd": "aaaaaaaaaaaaa",
            "eeeeeeeeeeee": 1
        },
        {
            "aaaaaa": "aaaa",
            "bbbbbbb": 10
        },
        {
            "aaaaaa": "aaaaaaa"
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaa",
            "cccccc": "aaaaaaaaaaaaaaaaaaa"
        },
        {
            "aaaaaaaaaaaaaaaaaaa": true,
            "bbbbbbbbbbbbbbbbbbb": "aaaaaaaaaaaaaaaaaaaaaaa"
        },
        {
            "aaaa": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaaaaa"
        },
        {
            "aaaaaa": "aaaa",
            "bbbbbbb": 10
        },
        {
            "aaaaaa": "aaaaaaa"
        },
        {
            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
            "bbbbbb": "aaaaaaaaaaa",
            "ccccccc": 60,
            "dddddddddd": false,
            "eeeeeeeeeeeeeeeeee": {
                "aaaaaaaaaaaaaaaaaa": {
                    "aaaaa": [
                        {
                            "aaaaaa": "aaaaaaa"
                        },
                        {
                            "aaaaaa": "aaaa",
                            "bbbbbbb": 600
                        },
                        {
                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                            "bbbbbb": "aaaaaaaaaaa",
                            "ccccccc": 60,
                            "dddddddddd": false,
                            "eeeeeeeeeeeeeeeeee": {
                                "aaaaaaaaaaaaaaaaaa": {
                                    "aaaaa": [
                                        {
                                            "aaaaaa": "aaaaaaa"
                                        },
                                        {
                                            "aaaaaa": "aaaa",
                                            "bbbbbbb": 600
                                        },
                                        {
                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                            "bbbbbb": "aaaaaaaaaaa",
                                            "ccccccc": 60,
                                            "dddddddddd": false,
                                            "eeeeeeeeeeeeeeeeee": {
                                                "aaaaaaaaaaaaaaaaaa": {
                                                    "aaaaa": [
                                                        {
                                                            "aaaaaa": "aaaaaaa"
                                                        },
                                                        {
                                                            "aaaaaa": "aaaa",
                                                            "bbbbbbb": 600
                                                        },
                                                        {
                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                            "bbbbbb": "aaaaaaaaaaa",
                                                            "ccccccc": 60,
                                                            "dddddddddd": false,
                                                            "eeeeeeeeeeeeeeeeee": {
                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                    "aaaaa": [
                                                                        {
                                                                            "aaaaaa": "aaaaaaa"
                                                                        },
                                                                        {
                                                                            "aaaaaa": "aaaa",
                                                                            "bbbbbbb": 600
                                                                        },
                                                                        {
                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                            "bbbbbb": "aaaaaaaaaaa",
                                                                            "ccccccc": 60,
                                                                            "dddddddddd": false,
                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                    "aaaaa": [
                                                                                        {
                                                                                            "aaaaaa": "aaaaaaa"
                                                                                        },
                                                                                        {
                                                                                            "aaaaaa": "aaaa",
                                                                                            "bbbbbbb": 600
                                                                                        },
                                                                                        {
                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                            "bbbbbb": "aaaaaaaaaaa",
                                                                                            "ccccccc": 60,
                                                                                            "dddddddddd": false,
                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                    "aaaaa": [
                                                                                                        {
                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                        },
                                                                                                        {
                                                                                                            "aaaaaa": "aaaa",
                                                                                                            "bbbbbbb": 600
                                                                                                        },
                                                                                                        {
                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                            "bbbbbb": "aaaaaaaaaaa",
                                                                                                            "ccccccc": 60,
                                                                                                            "dddddddddd": false,
                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                    "aaaaa": [
                                                                                                                        {
                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                        },
                                                                                                                        {
                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                            "bbbbbbb": 600
                                                                                                                        },
                                                                                                                        {
                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                            "bbbbbb": "aaaaaaaaaaa",
                                                                                                                            "ccccccc": 60,
                                                                                                                            "dddddddddd": false,
                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                    "aaaaa": [
                                                                                                                                        {
                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                        },
                                                                                                                                        {
                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                            "bbbbbbb": 600
                                                                                                                                        },
                                                                                                                                        {
                                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                                            "bbbbbb": "aaaaaaaaaaa",
                                                                                                                                            "ccccccc": 60,
                                                                                                                                            "dddddddddd": false,
                                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                                    "aaaaa": [
                                                                                                                                                        {
                                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                                        },
                                                                                                                                                        {
                                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                                            "bbbbbbb": 600
                                                                                                                                                        },
                                                                                                                                                        {
                                                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                                                            "bbbbbb": "aaaaaaaaaaaaaaa",
                                                                                                                                                            "ccccccc": 60,
                                                                                                                                                            "dddddddddd": false,
                                                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                                                    "aaaaa": [
                                                                                                                                                                        {
                                                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                                                        },
                                                                                                                                                                        {
                                                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                                                            "bbbbbbb": 600
                                                                                                                                                                        },
                                                                                                                                                                        {
                                                                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                                                                            "bbbbbb": "aaaaaaaaaaaaaaa",
                                                                                                                                                                            "ccccccc": 60,
                                                                                                                                                                            "dddddddddd": false,
                                                                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                                                                    "aaaaa": [
                                                                                                                                                                                        {
                                                                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                                                                        },
                                                                                                                                                                                        {
                                                                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                                                                            "bbbbbbb": 600
                                                                                                                                                                                        },
                                                                                                                                                                                        {
                                                                                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                                                                                            "bbbbbb": "aaaaaaaaaaaaaaa",
                                                                                                                                                                                            "ccccccc": 60,
                                                                                                                                                                                            "dddddddddd": false,
                                                                                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                                                                                    "aaaaa": [
                                                                                                                                                                                                        {
                                                                                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                                                                                        },
                                                                                                                                                                                                        {
                                                                                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                                                                                            "bbbbbbb": 600
                                                                                                                                                                                                        },
                                                                                                                                                                                                        {
                                                                                                                                                                                                            "aaaaaa": "aaaaaaaaaaaaaaaaaaa",
                                                                                                                                                                                                            "bbbbbb": "aaaaaaaaaaaaaaa",
                                                                                                                                                                                                            "ccccccc": 60,
                                                                                                                                                                                                            "dddddddddd": false,
                                                                                                                                                                                                            "eeeeeeeeeeeeeeeeee": {
                                                                                                                                                                                                                "aaaaaaaaaaaaaaaaaa": {
                                                                                                                                                                                                                    "aaaaa": [
                                                                                                                                                                                                                        {
                                                                                                                                                                                                                            "aaaaaa": "aaaaaaa"
                                                                                                                                                                                                                        },
                                                                                                                                                                                                                        {
                                                                                                                                                                                                                            "aaaaaa": "aaaa",
                                                                                                                                                                                                                            "bbbbbbb": 600
                                                                                                                                                                                                                        }
                                                                                                                                                                                                                    ]
                                                                                                                                                                                                                }
                                                                                                                                                                                                            }
                                                                                                                                                                                                        }
                                                                                                                                                                                                    ]
                                                                                                                                                                                                }
                                                                                                                                                                                            }
                                                                                                                                                                                        }
                                                                                                                                                                                    ]
                                                                                                                                                                                }
                                                                                                                                                                            }
                                                                                                                                                                        }
                                                                                                                                                                    ]
                                                                                                                                                                }
                                                                                                                                                            }
                                                                                                                                                        }
                                                                                                                                                    ]
                                                                                                                                                }
                                                                                                                                            }
                                                                                                                                        }
                                                                                                                                    ]
                                                                                                                                }
                                                                                                                            }
                                                                                                                        }
                                                                                                                    ]
                                                                                                                }
                                                                                                            }
                                                                                                        }
                                                                                                    ]
                                                                                                }
                                                                                            }
                                                                                        }
                                                                                    ]
                                                                                }
                                                                            }
                                                                        }
                                                                    ]
                                                                }
                                                            }
                                                        }
                                                    ]
                                                }
                                            }
                                        }
                                    ]
                                }
                            }
                        }
                    ]
                }
            }
        }
    ]
}

gdb (partial) backtrace after crash

(gdb) bt
#0  0x00007ffff749894c in ?? () from /usr/lib/libc.so.6
#1  0x00007ffff743e410 in raise () from /usr/lib/libc.so.6
#2  0x00007ffff742557a in abort () from /usr/lib/libc.so.6
#3  0x00007ffff7426613 in ?? () from /usr/lib/libc.so.6
#4  0x00007ffff74a2d65 in ?? () from /usr/lib/libc.so.6
#5  0x00007ffff74a72ac in ?? () from /usr/lib/libc.so.6
#6  0x00007ffff74a841c in realloc () from /usr/lib/libc.so.6
#7  0x00007ffff796b09b in rb_gc_impl_realloc (objspace_ptr=0x55555555ccf0, ptr=0x555555da8440, new_size=<optimized out>, old_size=0) at gc/default/default.c:8285
#8  0x00007ffff796dc14 in ruby_sized_xrealloc2_body (old_n=0, ptr=0x555555da8440, n=32320, size=1) at gc.c:4663
#9  ruby_sized_xrealloc2 (ptr=0x555555da8440, n=n@entry=32320, size=size@entry=1, old_n=old_n@entry=0) at gc.c:4656
#10 0x00007ffff796dc37 in ruby_xrealloc2 (ptr=<optimized out>, n=n@entry=32320, size=size@entry=1) at gc.c:4669
#11 0x00007fffdb9f784a in oj_grow_out (out=out@entry=0x7fffffffbaf0, len=len@entry=2) at dump.c:1291
#12 0x00007fffdb9fbb4d in assure_size (out=0x7fffffffbaf0, len=2) at /home/keith/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/oj-3.16.11/ext/oj/dump.h:67
#13 dump_hash (obj=140736881091480, depth=46, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:664
#14 0x00007fffdb9fdb68 in dump_array (a=140736878703440, depth=<optimized out>, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:167
#15 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736878703440, ov=140737488337648) at dump_compat.c:644
#16 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488320112, error=0) at hash.c:1302
#17 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488320112) at hash.c:944
#18 hash_foreach_call (arg=140737488320112) at hash.c:1437
#19 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488320112, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881091960) at eval.c:1074
#20 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881091960, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#21 0x00007fffdb9fba79 in dump_hash (obj=140736881091960, depth=44, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#22 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881091960, ov=140737488337648) at dump_compat.c:644
#23 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488320592, error=0) at hash.c:1302
#24 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488320592) at hash.c:944
#25 hash_foreach_call (arg=140737488320592) at hash.c:1437
#26 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488320592, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881092120) at eval.c:1074
#27 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881092120, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#28 0x00007fffdb9fba79 in dump_hash (obj=140736881092120, depth=43, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#29 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881092120, ov=140737488337648) at dump_compat.c:644
#30 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488321072, error=0) at hash.c:1302
#31 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488321072) at hash.c:944
#32 hash_foreach_call (arg=140737488321072) at hash.c:1437
#33 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488321072, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881092280) at eval.c:1074
#34 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881092280, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#35 0x00007fffdb9fba79 in dump_hash (obj=140736881092280, depth=42, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#36 0x00007fffdb9fdb68 in dump_array (a=140736878703720, depth=<optimized out>, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:167
#37 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736878703720, ov=140737488337648) at dump_compat.c:644
#38 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488321632, error=0) at hash.c:1302
#39 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488321632) at hash.c:944
#40 hash_foreach_call (arg=140737488321632) at hash.c:1437
#41 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488321632, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881092760) at eval.c:1074
#42 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881092760, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#43 0x00007fffdb9fba79 in dump_hash (obj=140736881092760, depth=40, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#44 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881092760, ov=140737488337648) at dump_compat.c:644
#45 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488322112, error=0) at hash.c:1302
#46 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488322112) at hash.c:944
#47 hash_foreach_call (arg=140737488322112) at hash.c:1437
#48 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488322112, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881092920) at eval.c:1074
--Type <RET> for more, q to quit, c to continue without paging--
#49 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881092920, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#50 0x00007fffdb9fba79 in dump_hash (obj=140736881092920, depth=39, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#51 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881092920, ov=140737488337648) at dump_compat.c:644
#52 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488322592, error=0) at hash.c:1302
#53 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488322592) at hash.c:944
#54 hash_foreach_call (arg=140737488322592) at hash.c:1437
#55 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488322592, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881093080) at eval.c:1074
#56 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881093080, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#57 0x00007fffdb9fba79 in dump_hash (obj=140736881093080, depth=38, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#58 0x00007fffdb9fdb68 in dump_array (a=140736878704040, depth=<optimized out>, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:167
#59 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736878704040, ov=140737488337648) at dump_compat.c:644
#60 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488323152, error=0) at hash.c:1302
#61 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488323152) at hash.c:944
#62 hash_foreach_call (arg=140737488323152) at hash.c:1437
#63 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488323152, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881093720) at eval.c:1074
#64 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881093720, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#65 0x00007fffdb9fba79 in dump_hash (obj=140736881093720, depth=36, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#66 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881093720, ov=140737488337648) at dump_compat.c:644
#67 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488323632, error=0) at hash.c:1302
#68 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488323632) at hash.c:944
#69 hash_foreach_call (arg=140737488323632) at hash.c:1437
#70 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488323632, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881093880) at eval.c:1074
#71 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881093880, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#72 0x00007fffdb9fba79 in dump_hash (obj=140736881093880, depth=35, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#73 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881093880, ov=140737488337648) at dump_compat.c:644
#74 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488324112, error=0) at hash.c:1302
#75 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488324112) at hash.c:944
#76 hash_foreach_call (arg=140737488324112) at hash.c:1437
#77 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488324112, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881094040) at eval.c:1074
#78 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881094040, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#79 0x00007fffdb9fba79 in dump_hash (obj=140736881094040, depth=34, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#80 0x00007fffdb9fdb68 in dump_array (a=140736878704320, depth=<optimized out>, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:167
#81 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736878704320, ov=140737488337648) at dump_compat.c:644
#82 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488324672, error=0) at hash.c:1302
#83 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488324672) at hash.c:944
#84 hash_foreach_call (arg=140737488324672) at hash.c:1437
#85 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488324672, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881094520) at eval.c:1074
#86 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881094520, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#87 0x00007fffdb9fba79 in dump_hash (obj=140736881094520, depth=32, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#88 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881094520, ov=140737488337648) at dump_compat.c:644
#89 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488325152, error=0) at hash.c:1302
#90 ar_foreach_check (func=<optimized out>, never=36, hash=<optimized out>, arg=140737488325152) at hash.c:944
#91 hash_foreach_call (arg=140737488325152) at hash.c:1437
#92 0x00007ffff79469d3 in rb_ensure (b_proc=b_proc@entry=0x7ffff79731d0 <hash_foreach_call>, data1=data1@entry=140737488325152, e_proc=e_proc@entry=0x7ffff7970160 <hash_foreach_ensure>, 
    data2=data2@entry=140736881094680) at eval.c:1074
#93 0x00007ffff7977b8a in rb_hash_foreach (hash=hash@entry=140736881094680, func=func@entry=0x7fffdb9fd3c0 <hash_cb>, farg=farg@entry=140737488337648) at hash.c:1465
#94 0x00007fffdb9fba79 in dump_hash (obj=140736881094680, depth=31, out=0x7fffffffbaf0, as_ok=<optimized out>) at dump_compat.c:670
#95 0x00007fffdb9fd5cd in hash_cb (key=<optimized out>, value=140736881094680, ov=140737488337648) at dump_compat.c:644
#96 0x00007ffff7973236 in hash_ar_foreach_iter (key=<optimized out>, value=<optimized out>, argp=140737488325632, error=0) at hash.c:1302
...

[ohler55]

That's going to take some experimenting to reproduce.

[ohler55]

Does the crash occur when running without bundle?

Does the crash occur with just the load or on the call to dump.

I tried on a virtual machine but may have to resurrect an older linux box to recreate.

[ubergeek42]

Some more details, please let me know if you need any further help reproducing it.

---

Looks like it reproduces fine without bundle. It crashes on the dump, not the load.

root@717a7386b488:/# irb
irb(main):001> require 'oj'
=> true
irb(main):002> require 'oj_mimic_json'
=> true
irb(main):003> doc = Oj.load(File.read("oj-crash-sanitized.json"))
=> 
{"aaaaa" =>
...
irb(main):004> Oj.dump(doc, indent: "  ", space: " ", object_nl: "\n", array_nl: "\n")
realloc(): invalid next size
Aborted (core dumped)

I can reproduce this on my arch linux system (with ruby installed via mise/asdf), on my M2 macbook via docker (running the amd64 image variant), and on my linux system via podman:

keith@Mac ~ % docker run --platform linux/amd64 -it -v $PWD/oj-crash-sanitized.json:/oj-crash-sanitized.json  ruby:3.4.4 bash
root@c1076ff941d4:/# gem install oj oj_mimic_json
Fetching oj-3.16.11.gem
Building native extensions. This could take a while...
Successfully installed oj-3.16.11
Fetching oj_mimic_json-1.0.1.gem
Successfully installed oj_mimic_json-1.0.1
2 gems installed

A new release of RubyGems is available: 3.6.7 → 3.7.2!
Run `gem update --system 3.7.2` to update your installation.

root@c1076ff941d4:/# ruby -e 'require "oj"; require "oj_mimic_json"; Oj.dump(Oj.load(File.read("oj-crash-sanitized.json")), indent: "  ", space: " ", object_nl: "\n", array_nl: "\n");'
realloc(): invalid next size
Aborted
root@c1076ff941d4:/# 

And on my linux system via podman (though docker probably works too):

[keith@kjpc ~]$ podman run -it -v $PWD/oj-crash-sanitized.json:/oj-crash-sanitized.json  ruby:3.4.4 bash
root@717a7386b488:/# gem install oj oj_mimic_json
Fetching oj-3.16.11.gem
Building native extensions. This could take a while...
Successfully installed oj-3.16.11
Fetching oj_mimic_json-1.0.1.gem
Successfully installed oj_mimic_json-1.0.1
2 gems installed

A new release of RubyGems is available: 3.6.7 → 3.7.2!
Run `gem update --system 3.7.2` to update your installation.

root@717a7386b488:/# ruby -e 'require "oj"; require "oj_mimic_json"; Oj.dump(Oj.load(File.read("oj-crash-sanitized.json")), indent: "  ", space: " ", object_nl: "\n", array_nl: "\n");'
realloc(): invalid next size
Aborted (core dumped)
root@717a7386b488:/# 

[ubergeek42]

I did also test quickly just now and it doesn't seem to be specific to that version of ruby, running the ruby:latest (i.e. ruby 3.4.6) docker image exhibits the same crash.

[ohler55]

Thanks, I'll get docker installed again and give it a try.

[ohler55]

I've been a bit overloaded recently. I had issues getting docker working with amd64 images on M series apples. If you have an image already set up would you be willing to share that?

[ubergeek42]

It turns out you can just use the aarch64 images and it replicates as well. I don't think there's any image I can build to make it easier, the official ruby docker image seems pretty sufficient; you just then need to install the oj and oj_mimic_json gems.

Example of running it against the aarch64 docker image of ruby:

keith@Mac ~ % docker run -it -v $PWD/oj-crash-sanitized.json:/oj-crash-sanitized.json ruby:3.4.4 bash
root@e92a718032cc:/# gem install oj oj_mimic_json
Fetching oj-3.16.11.gem
Building native extensions. This could take a while...
Successfully installed oj-3.16.11
Fetching oj_mimic_json-1.0.1.gem
Successfully installed oj_mimic_json-1.0.1
2 gems installed

root@e92a718032cc:/# ruby -e 'require "oj"; require "oj_mimic_json"; Oj.dump(Oj.load(File.read("oj-crash-sanitized.json")), indent: "  ", space: " ", object_nl: "\n", array_nl: "\n");'
realloc(): invalid next size
Aborted

root@e92a718032cc:/# uname -m
aarch64
``

[ohler55]

Thanks, I'll try it out this weekend.

[ohler55]

Please see if the fix-dump-overflow brnach fixes the issue for you. With your instructions I was able to get the docker container running and nail down the issue. Thanks you. Sorry it has taken so long.