|
| 1 | +Here’s a walkthrough of how this “atomic marker” system works, what files it |
| 2 | +creates, and what to watch out for on Linux vs. Windows. |
| 3 | + |
| 4 | +--- |
| 5 | + |
| 6 | +## 1. What is a “marker”? |
| 7 | + |
| 8 | +A *marker* is simply a file whose name encodes a string value and a |
| 9 | +monotonically increasing counter. By always creating a *new* file for the next |
| 10 | +value (and then deleting the old one), the code ensures that at any point |
| 11 | +on-disk there is exactly one visible marker file whose name tells you “the |
| 12 | +current value is X, at iteration N.” |
| 13 | + |
| 14 | +Filenames have the pattern: |
| 15 | + |
| 16 | +``` |
| 17 | +marker.<markerName>.<iteration>.<value> |
| 18 | +``` |
| 19 | + |
| 20 | +- `<markerName>` is an arbitrary identifier you chose. |
| 21 | +- `<iteration>` is a zero-padded, 6-digit decimal. |
| 22 | +- `<value>` is the new state you’re storing. |
| 23 | + |
| 24 | +For example, if your marker is named `foo`, you might see files like: |
| 25 | + |
| 26 | +```text |
| 27 | +marker.foo.000001.alpha |
| 28 | +marker.foo.000002.beta |
| 29 | +``` |
| 30 | + |
| 31 | +--- |
| 32 | + |
| 33 | +## 2. High-level API |
| 34 | + |
| 35 | +### ReadMarker(fs, dir, markerName) |
| 36 | +- Lists all files in `dir`. |
| 37 | +- Picks out those beginning with `marker.` and matching your `markerName`. |
| 38 | +- Returns the highest-iteration file’s `value` (e.g. `"beta"`). |
| 39 | + |
| 40 | +### LocateMarker(fs, dir, markerName) |
| 41 | +- Same as `ReadMarker`, but also opens `dir` for later syncing. |
| 42 | +- Returns a `*Marker` object plus the current value. |
| 43 | + |
| 44 | +--- |
| 45 | + |
| 46 | +## 3. Core logic: parsing and scanning |
| 47 | + |
| 48 | +```go |
| 49 | +func scanForMarker(fs vfs.FS, ls []string, markerName string) (scannedState, error) |
| 50 | +``` |
| 51 | +- Iterates over filenames in `ls`. |
| 52 | +- Calls `parseMarkerFilename`, which: |
| 53 | + 1. Strips the `marker.` prefix. |
| 54 | + 2. Splits off the name, the iteration (parsed via `strconv.ParseUint`), and the value. |
| 55 | +- Keeps track of: |
| 56 | + - **state.filename**: the newest marker file seen so far. |
| 57 | + - **state.iter**: that file’s iteration number. |
| 58 | + - **state.value**: that file’s embedded value. |
| 59 | + - **state.obsolete**: a list of older marker files. |
| 60 | + |
| 61 | +--- |
| 62 | + |
| 63 | +## 4. The `Marker` object |
| 64 | + |
| 65 | +Once you have a `*Marker`, you can do two things: |
| 66 | + |
| 67 | +### Move(newValue string) |
| 68 | +1. Increment the internal `iter` counter. |
| 69 | +2. Build a filename `marker.<name>.<iter>.<newValue>`. |
| 70 | +3. Create that file via `fs.Create(...)`. |
| 71 | +4. Close it, then delete the old `marker.*` file. |
| 72 | +5. Call `dirFD.Sync()` to flush the directory metadata. |
| 73 | + |
| 74 | +If deletion of the old file fails, it’s remembered in an `obsoleteFiles` slice |
| 75 | +for later cleanup. |
| 76 | + |
| 77 | +### RemoveObsolete() |
| 78 | +Loops over any filenames in `obsoleteFiles` and tries to remove them. Stops at |
| 79 | +first error (leaving the remaining entries in `obsoleteFiles`). |
| 80 | + |
| 81 | +--- |
| 82 | + |
| 83 | +## 5. Examples of files created |
| 84 | + |
| 85 | +Imagine we start with an empty directory and markerName = `"task"`: |
| 86 | + |
| 87 | +```shell |
| 88 | +$ ls |
| 89 | +# (empty) |
| 90 | + |
| 91 | +// First LocateMarker → no existing files, iter=0, value="" |
| 92 | +marker, _, err := LocateMarker(fs, "/path", "task") |
| 93 | + |
| 94 | +// Move to "started": |
| 95 | +marker.Move("started") |
| 96 | +$ ls /path |
| 97 | +marker.task.000001.started |
| 98 | + |
| 99 | +// Move to "halfway": |
| 100 | +marker.Move("halfway") |
| 101 | +$ ls /path |
| 102 | +marker.task.000002.halfway |
| 103 | + |
| 104 | +// If RemoveObsolete hasn’t run yet, you might still see the old: |
| 105 | +marker.RemoveObsolete() |
| 106 | +$ ls /path |
| 107 | +marker.task.000002.halfway |
| 108 | +``` |
| 109 | + |
| 110 | +And if your value strings contain dots or special chars, they simply go into the |
| 111 | +filename (so avoid `/`). |
| 112 | + |
| 113 | +--- |
| 114 | + |
| 115 | +## 6. Linux vs. Windows quirks |
| 116 | + |
| 117 | +1. **Path separators & case-sensitivity** |
| 118 | + - On Linux, files named `marker.Task.000001.foo` and `marker.task.000001.foo` are |
| 119 | + distinct. |
| 120 | + - On Windows, filenames are case-insensitive, so your marker names should avoid |
| 121 | + differing only by letter case. |
| 122 | + |
| 123 | +2. **Directory sync (`dirFD.Sync()`)** |
| 124 | + - Linux: opening a directory and `fsync`-ing its file descriptor reliably flushes the new/removed filenames to disk. |
| 125 | + - Windows: the equivalent “flush directory metadata” is more limited. In some cases you may see the new file appear immediately but metadata not fully durable until later; and deleting a file that’s still in use can fail. |
| 126 | + |
| 127 | +3. **Deleting open files** |
| 128 | + - Linux lets you `unlink` (remove) a file even while a process still has it open; the file truly goes away only after all handles close. |
| 129 | + - Windows generally refuses to delete an open file handle, so if another process still has the old marker open, the `Remove` call will error. That error is caught and the old filename pushed into `obsoleteFiles`. |
| 130 | + |
| 131 | +4. **Maximum filename length** |
| 132 | + - Windows MAX_PATH (260 chars) may be hit if your `<value>` is long; Linux allows up to around 255 bytes per component. Keep your values short. |
| 133 | + |
| 134 | +--- |
| 135 | + |
| 136 | +### Summary |
| 137 | + |
| 138 | +- **Naming scheme**: `marker.<name>.<6-digit-iter>.<value>` |
| 139 | +- **Read/Locate**: scan the directory, pick the highest iter. |
| 140 | +- **Move**: create new marker file, delete old, sync. |
| 141 | +- **Cleanup**: `RemoveObsolete()` for any leftovers. |
| 142 | +- **Linux vs. Windows**: watch case sensitivity, directory‐sync semantics, and |
| 143 | + open‐file deletion behavior. |
| 144 | + |
| 145 | +This pattern guarantees that at most one marker file is “current,” and that any |
| 146 | +failure mid‐move leaves you either at the old value or the new one—never a |
| 147 | +corrupted state. |
0 commit comments