Skip to content

Commit beff129

Browse files
committed
atomicfs: add README
Adding a `README.md` with a high-level explanation (generated by GPT o4-mini-high).
1 parent 52551f8 commit beff129

File tree

1 file changed

+147
-0
lines changed

1 file changed

+147
-0
lines changed

vfs/atomicfs/README.md

+147
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
Here’s a walkthrough of how this “atomic marker” system works, what files it
2+
creates, and what to watch out for on Linux vs. Windows.
3+
4+
---
5+
6+
## 1. What is a “marker”?
7+
8+
A *marker* is simply a file whose name encodes a string value and a
9+
monotonically increasing counter. By always creating a *new* file for the next
10+
value (and then deleting the old one), the code ensures that at any point
11+
on-disk there is exactly one visible marker file whose name tells you “the
12+
current value is X, at iteration N.”
13+
14+
Filenames have the pattern:
15+
16+
```
17+
marker.<markerName>.<iteration>.<value>
18+
```
19+
20+
- `<markerName>` is an arbitrary identifier you chose.
21+
- `<iteration>` is a zero-padded, 6-digit decimal.
22+
- `<value>` is the new state you’re storing.
23+
24+
For example, if your marker is named `foo`, you might see files like:
25+
26+
```text
27+
marker.foo.000001.alpha
28+
marker.foo.000002.beta
29+
```
30+
31+
---
32+
33+
## 2. High-level API
34+
35+
### ReadMarker(fs, dir, markerName)
36+
- Lists all files in `dir`.
37+
- Picks out those beginning with `marker.` and matching your `markerName`.
38+
- Returns the highest-iteration file’s `value` (e.g. `"beta"`).
39+
40+
### LocateMarker(fs, dir, markerName)
41+
- Same as `ReadMarker`, but also opens `dir` for later syncing.
42+
- Returns a `*Marker` object plus the current value.
43+
44+
---
45+
46+
## 3. Core logic: parsing and scanning
47+
48+
```go
49+
func scanForMarker(fs vfs.FS, ls []string, markerName string) (scannedState, error)
50+
```
51+
- Iterates over filenames in `ls`.
52+
- Calls `parseMarkerFilename`, which:
53+
1. Strips the `marker.` prefix.
54+
2. Splits off the name, the iteration (parsed via `strconv.ParseUint`), and the value.
55+
- Keeps track of:
56+
- **state.filename**: the newest marker file seen so far.
57+
- **state.iter**: that file’s iteration number.
58+
- **state.value**: that file’s embedded value.
59+
- **state.obsolete**: a list of older marker files.
60+
61+
---
62+
63+
## 4. The `Marker` object
64+
65+
Once you have a `*Marker`, you can do two things:
66+
67+
### Move(newValue string)
68+
1. Increment the internal `iter` counter.
69+
2. Build a filename `marker.<name>.<iter>.<newValue>`.
70+
3. Create that file via `fs.Create(...)`.
71+
4. Close it, then delete the old `marker.*` file.
72+
5. Call `dirFD.Sync()` to flush the directory metadata.
73+
74+
If deletion of the old file fails, it’s remembered in an `obsoleteFiles` slice
75+
for later cleanup.
76+
77+
### RemoveObsolete()
78+
Loops over any filenames in `obsoleteFiles` and tries to remove them. Stops at
79+
first error (leaving the remaining entries in `obsoleteFiles`).
80+
81+
---
82+
83+
## 5. Examples of files created
84+
85+
Imagine we start with an empty directory and markerName = `"task"`:
86+
87+
```shell
88+
$ ls
89+
# (empty)
90+
91+
// First LocateMarker → no existing files, iter=0, value=""
92+
marker, _, err := LocateMarker(fs, "/path", "task")
93+
94+
// Move to "started":
95+
marker.Move("started")
96+
$ ls /path
97+
marker.task.000001.started
98+
99+
// Move to "halfway":
100+
marker.Move("halfway")
101+
$ ls /path
102+
marker.task.000002.halfway
103+
104+
// If RemoveObsolete hasn’t run yet, you might still see the old:
105+
marker.RemoveObsolete()
106+
$ ls /path
107+
marker.task.000002.halfway
108+
```
109+
110+
And if your value strings contain dots or special chars, they simply go into the
111+
filename (so avoid `/`).
112+
113+
---
114+
115+
## 6. Linux vs. Windows quirks
116+
117+
1. **Path separators & case-sensitivity**
118+
- On Linux, files named `marker.Task.000001.foo` and `marker.task.000001.foo` are
119+
distinct.
120+
- On Windows, filenames are case-insensitive, so your marker names should avoid
121+
differing only by letter case.
122+
123+
2. **Directory sync (`dirFD.Sync()`)**
124+
- Linux: opening a directory and `fsync`-ing its file descriptor reliably flushes the new/removed filenames to disk.
125+
- Windows: the equivalent “flush directory metadata” is more limited. In some cases you may see the new file appear immediately but metadata not fully durable until later; and deleting a file that’s still in use can fail.
126+
127+
3. **Deleting open files**
128+
- Linux lets you `unlink` (remove) a file even while a process still has it open; the file truly goes away only after all handles close.
129+
- Windows generally refuses to delete an open file handle, so if another process still has the old marker open, the `Remove` call will error. That error is caught and the old filename pushed into `obsoleteFiles`.
130+
131+
4. **Maximum filename length**
132+
- Windows MAX_PATH (260 chars) may be hit if your `<value>` is long; Linux allows up to around 255 bytes per component. Keep your values short.
133+
134+
---
135+
136+
### Summary
137+
138+
- **Naming scheme**: `marker.<name>.<6-digit-iter>.<value>`
139+
- **Read/Locate**: scan the directory, pick the highest iter.
140+
- **Move**: create new marker file, delete old, sync.
141+
- **Cleanup**: `RemoveObsolete()` for any leftovers.
142+
- **Linux vs. Windows**: watch case sensitivity, directory‐sync semantics, and
143+
open‐file deletion behavior.
144+
145+
This pattern guarantees that at most one marker file is “current,” and that any
146+
failure mid‐move leaves you either at the old value or the new one—never a
147+
corrupted state.

0 commit comments

Comments
 (0)