Heap overflow (write) in `FSE_compress_usingCTable`

[hgarrereyn]

Hi, the following testcase constructs an input (of repeated zeros) and builds a CTable to compress it.

However, invoking FSE_compress_usingCTable results in a heap overflow (write).

As far as I can tell, there is no API misuse here, but please let me know if that is incorrect.

See the following test case:

testcase.cpp

#include <cstddef>
#include <cstdint>
#include <cstring>
#include <string>
#include <vector>
extern "C" {
#include "fse.h"
}

int main() {
    const unsigned tableLog = 12;            // picked from the crashing run
    const unsigned maxSymbolValue = 255;     // standard byte alphabet

    // Source: 1024 bytes of the same symbol (0x00)
    std::string src(1024, '\0');

    // Build frequency counts and normalized counts
    std::vector<unsigned> count(maxSymbolValue + 1, 0u);
    for (unsigned char ch : src) count[(unsigned)ch]++;
    std::vector<short> ncount(maxSymbolValue + 1);
    size_t nr = FSE_normalizeCount(ncount.data(), tableLog, count.data(), src.size(), maxSymbolValue);
    if (FSE_isError(nr)) return 0;

    // Build a matching CTable
    FSE_CTable* ct = FSE_createCTable(maxSymbolValue, tableLog);
    if (!ct) return 0;
    if (FSE_isError(FSE_buildCTable(ct, ncount.data(), maxSymbolValue, tableLog))) return 0;

    // Allocate output buffer using the official bound
    size_t bound = FSE_compressBound(src.size());
    std::string out; out.resize(bound);

    // This call overflows by 8 bytes in FSE_compress_usingCTable_generic (FSE_FLUSHBITS)
    (void)FSE_compress_usingCTable(out.data(), out.size(), src.data(), src.size(), ct);
    return 0;
}

crash report

{
  "Date": "2025-09-29T16:40:05.800506+00:00",
  "Uname": "Linux 53eddc8c2cc2 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux",
  "OS": "Ubuntu",
  "OSRelease": "22.04",
  "Architecture": "amd64",
  "ExecutablePath": "/tmp/tmptlotf_y_/reproducer",
  "ProcEnviron": [
    "PREFIX=/fuzz/install",
    "LIBAFL_EDGES_MAP_SIZE=800000",
    "PWD=/fuzz/workspace",
    "CXX=gf_libafl_cxx",
    "GRAPHFUZZ_USE_ASAN=1",
    "HOME=/root",
    "ASAN_OPTIONS=hard_rss_limit_mb=1024:detect_leaks=0",
    "TERM=xterm-256color",
    "SHLVL=1",
    "LD_LIBRARY_PATH=/fuzz/install/lib",
    "PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "CC=gf_libafl_cc",
    "DEBIAN_FRONTEND=noninteractive",
    "OLDPWD=/fuzz/src",
    "_=/usr/local/bin/agfi"
  ],
  "ProcCmdline": "/tmp/tmptlotf_y_/reproducer",
  "Stdin": "",
  "ProcStatus": [],
  "ProcMaps": [],
  "ProcFiles": [],
  "NetworkConnections": [],
  "CrashSeverity": {
    "Type": "EXPLOITABLE",
    "ShortDescription": "heap-buffer-overflow(write)",
    "Description": "Heap buffer overflow",
    "Explanation": "The target writes data past the end, or before the beginning, of the intended heap buffer."
  },
  "Stacktrace": [
    "    #0 0x55555566165a in FSE_compress_usingCTable_generic /fuzz/src/lib/fse_compress.c:605:9",
    "    #1 0x55555565bb3d in main /tmp/tmptlotf_y_/reproducer.cpp:34:11",
    "    #2 0x7ffff7a6dd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16",
    "    #3 0x7ffff7a6de3f in __libc_start_main csu/../csu/libc-start.c:392:3",
    "    #4 0x555555580314 in _start (/tmp/tmptlotf_y_/reproducer+0x2c314) (BuildId: a7c4d64b26afbf1b7a2755d8b385e74214d74113)"
  ],
  "Registers": {},
  "Disassembly": [],
  "Package": "",
  "PackageVersion": "",
  "PackageArchitecture": "",
  "PackageDescription": "",
  "AsanReport": [
    "==156==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x51b000000695 at pc 0x55555566165b bp 0x7fffffffe960 sp 0x7fffffffe958",
    "WRITE of size 8 at 0x51b000000695 thread T0",
    "    #0 0x55555566165a in FSE_compress_usingCTable_generic /fuzz/src/lib/fse_compress.c:605:9",
    "    #1 0x55555565bb3d in main /tmp/tmptlotf_y_/reproducer.cpp:34:11",
    "    #2 0x7ffff7a6dd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16",
    "    #3 0x7ffff7a6de3f in __libc_start_main csu/../csu/libc-start.c:392:3",
    "    #4 0x555555580314 in _start (/tmp/tmptlotf_y_/reproducer+0x2c314) (BuildId: a7c4d64b26afbf1b7a2755d8b385e74214d74113)",
    "",
    "0x51b000000695 is located 0 bytes after 1557-byte region [0x51b000000080,0x51b000000695)",
    "allocated by thread T0 here:",
    "    #0 0x55555565912d in operator new(unsigned long) (/tmp/tmptlotf_y_/reproducer+0x10512d) (BuildId: a7c4d64b26afbf1b7a2755d8b385e74214d74113)",
    "    #1 0x7ffff7ed40bd in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x14c0bd) (BuildId: e72c155b714bc42a767ec9c0dd94589110e5b42f)",
    "",
    "SUMMARY: AddressSanitizer: heap-buffer-overflow /fuzz/src/lib/fse_compress.c:605:9 in FSE_compress_usingCTable_generic",
    "Shadow bytes around the buggy address:",
    "  0x51b000000400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x51b000000480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x51b000000500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x51b000000580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x51b000000600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "=>0x51b000000680: 00 00[05]fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "  0x51b000000700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "  0x51b000000780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "  0x51b000000800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "  0x51b000000880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "  0x51b000000900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa",
    "Shadow byte legend (one shadow byte represents 8 application bytes):",
    "  Addressable:           00",
    "  Partially addressable: 01 02 03 04 05 06 07",
    "  Heap left redzone:       fa",
    "  Freed heap region:       fd",
    "  Stack left redzone:      f1",
    "  Stack mid redzone:       f2",
    "  Stack right redzone:     f3",
    "  Stack after return:      f5",
    "  Stack use after scope:   f8",
    "  Global redzone:          f9",
    "  Global init order:       f6",
    "  Poisoned by user:        f7",
    "  Container overflow:      fc",
    "  Array cookie:            ac",
    "  Intra object redzone:    bb",
    "  ASan internal:           fe",
    "  Left alloca redzone:     ca",
    "  Right alloca redzone:    cb",
    "==156==ABORTING"
  ],
  "MsanReport": [],
  "UbsanReport": [],
  "LuaReport": [],
  "PythonReport": [],
  "GoReport": [],
  "JavaReport": [],
  "RustReport": [],
  "JsReport": [],
  "CSharpReport": [],
  "CrashLine": "/fuzz/src/lib/fse_compress.c:605:9",
  "Source": [
    "    601                FSE_encodeSymbol(&bitC, &CState2, *--ip);",
    "    602                FSE_encodeSymbol(&bitC, &CState1, *--ip);",
    "    603            }",
    "    604    ",
    "--->605            FSE_FLUSHBITS(&bitC);",
    "    606        }",
    "    607    ",
    "    608        FSE_flushCState(&bitC, &CState2);",
    "    609        FSE_flushCState(&bitC, &CState1);",
    "    610        return BIT_closeCStream(&bitC);"
  ]
}