fix: add Sync() calls to persist filesystem changes to storage

[marcelfarres]

I was trying to create folders & files in the SD card and it was not working.

This changes seem to make it functional again.

I test it on a M4 Gran Central, and seems to have dir persistent and creation even after mounting/un-moutning the file system.

Not an expert in any extend on TinyGo, hope it helps/you can test it/review it.

• Add Sync() after Format, Unmount, Remove, Rename, Mkdir
• Add Sync() after File.Close, File.Sync, File.Truncate
• Sync on OpenFile with O_CREATE/O_TRUNC flags
• Fix cache invalidation in lfs_bd_flush
• Add TestDirectoryPersistence and NestedDirectories tests"

Should fix #9

[marcelfarres]

Fix filesystem persistence on SD cards

Problem

I was working with LittleFS on a Grand Central M4 with an SD card and noticed that directories and files I created weren't surviving a power cycle. I'd run mkdir, it would succeed, ls would show the directory. But after unplugging and reconnecting (or even just unmounting and remounting), everything was gone.

Investigation

I traced the data flow through the code:

1. When I call fs.Mkdir("test"), it goes to go_lfs.go:247
2. That calls the C library's lfs_mkdir() via CGo
3. LittleFS writes through the callbacks in go_lfs_callbacks.go to the SD card driver
4. The SD card driver buffers the writes... and that's where they stay

The problem: the SD card driver holds writes in a buffer for efficiency. Without explicitly calling Sync(), that buffer never gets flushed to the physical flash. On power loss, the buffered data is gone.

Looking at the original code, I noticed that mutating operations just returned after calling the C function—no sync. The block device interface in tinyfs.go defines a Syncer interface, but it wasn't being used after filesystem operations.

What I changed

littlefs/go_lfs.go

Added Sync() calls after every operation that modifies the filesystem. Example for Mkdir:

func (l *LFS) Mkdir(path string, _ os.FileMode) error {
    cs := (*C.char)(cstring(path))
    defer C.free(unsafe.Pointer(cs))
    if err := errval(C.lfs_mkdir(l.lfs, cs)); err != nil {
        return err
    }
    if syncer, ok := l.dev.(tinyfs.Syncer); ok {
        return syncer.Sync()  // <-- This was missing!
    }
    return nil
}

Same pattern applied to:

• Format() - sync after formatting
• Unmount() - sync before closing
• Remove() - sync after deletion
• Rename() - sync after rename
• OpenFile() with O_CREATE/O_TRUNC - sync after creating
• File.Close() - sync after closing
• File.Sync() - sync to block device
• File.Truncate() - sync after truncation

littlefs/go_lfs_test.go

Added tests:

• TestDirectoryPersistence - creates a directory, unmounts, remounts, verifies it persists
• NestedDirectories - creates 3 levels of nested directories

Testing

Tested on actual hardware (Grand Central M4 + SD card):

• Formatted the card
• Created nested directories (level1/level2/level3)
• Created files
• Unmounted and remounted
• Everything persisted ✓

[deadprogram]

I do not think it makes sense to call Sync() on an Open().

[deadprogram]

Actually Sync() on a Sync()? Seems like a yes! 👍

[deadprogram]

Hello @marcelfarres thanks for the code submission. Please see my comments.

[deadprogram]

I probably did not notice the issue due to using flash pretty much all of the time.

[marcelfarres]

I am still investigating the fix, somehow I can not reproduce the error or the fix all the time?

Also it seems that using different -opt levels influence the results (with the unpatch branch opt 0 1 2 have the bug, but s and z pass the tests). Any idea or experience on that?

I want to be sure that the change does not add to the unreliable behavior.

It also seems that SPI freq plays a role too?

A bit puzzle.

[Copilot]

Pull request overview

This PR aims to improve reliability of LittleFS operations on TinyGo targets by adding a compiler miscompilation workaround in littlefs/lfs.c, adjusting Go↔C callback ABI types, and introducing both automated and hardware-driven regression coverage.

Changes:

• Add an optnone workaround to lfs_dir_fetchmatch in littlefs/lfs.c to mitigate an LLVM -O2 miscompilation.
• Update exported Go block-device callbacks to use uint32 for size (matching lfs_size_t) to avoid ABI/type mismatches.
• Add new directory-focused regression tests plus a hardware test harness and firmware example to reproduce/validate behavior on real boards.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/hw/main.go	Adds a hardware test runner that builds/flashes “fixed/unfixed” variants by patching lfs.c and classifying serial output.
tests/hw/go.mod	Introduces a standalone Go module for the hardware test harness.
tests/hw/go.sum	Adds dependency sums for the hardware test harness module.
tests/hw/.gitignore	Ignores build outputs for the hardware harness (builds/, *.exe).
littlefs/lfs.c	Adds an __attribute__((optnone)) workaround and explanation comment on lfs_dir_fetchmatch.
littlefs/go_lfs_test.go	Expands directory test coverage (nested dirs, persistence, dirs-with-files, deep nesting).
littlefs/go_lfs_callbacks.go	Changes callback size parameter type to uint32 to match lfs_size_t.
examples/sd_mkdir_test/main.go	Adds a large firmware example for interactive serial-driven FS testing on SD + LittleFS.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[marcelfarres]

Hey @deadprogram, this turned into a completely different fix than what I originally pushed.

Long story short: the mkdir bug has nothing to do with Sync(). It's LLVM generating bad code for one specific function (lfs_dir_fetchmatch) when you build at -opt=2. At -opt=z the bug never appears, which is why the Sync calls looked like they helped — they didn't, I just wasn't hitting the right opt level.

I spent a while trying to find a targeted workaround (volatile vars, memory barriers, noinline, etc.) but nothing short of disabling optimization on that one function works. So the fix is a single __attribute__((optnone)) on lfs_dir_fetchmatch.

While I was at it I also:

• Reverted all the Sync()/cache changes (they were a red herring)
• Fixed a type mismatch in the CGo callbacks (size was int but needs to be uint32 to match C — was crashing unit tests on 64-bit)
• Added some directory tests and an SD card example you can flash to verify

Tested on Grand Central M4 + SD at -opt=2: 50/50 mkdir+stat cycles pass, nested dirs work, files in subdirs work. Without the fix, everything fails.

PD: Sorry for the random co-pilot spam.