util.c: add fsync_close() helper, use where appropriate #268
Conversation
|
So the failing tests seem to be due to a "wrong" expected disk usage. But I do wonder how an added fsync() can change that - it's not really clear where the expected numbers come from. And in fact, when I just tried building the hdimage test case with current master, looked at |
Using genimage directly on an eMMC on target, I'm seeing that the BLKRRPART ioctl fails with EBUSY, presumably because there's still lots of data in flight, and then subsequent parts of my bootstrap procedure fail because the expected partitions can't be found. Whenever we've written some data, we really should ensure that all data has hit the disk before proceeding, and close() itself is not synchronous. Add a fsync_close() helper that does exactly what it says on the tin. Use that wherever we close an fd that has been open for writing and actually written to (i.e., no point in doing that in the reload_partitions() function). Currently, the return value of these close() calls are ignored, so at least for now continue to do that, but at least we do see an error message in case something went wrong. A test case is updated to reflect the new, and more accurate, disk usage. It turns out that the lack of this fsync'ing has really created images which later (long after genimage is done and the test framework has accepted them) would increase in disk usage, i.e. once the kernel gets around to write out any buffered data. With current master, one can observe this: $ mkdir images input tmp $ dd if=/dev/zero of=input/part1.img bs=512 count=7 && dd if=/dev/zero of=input/part2.img bs=512 count=11 && touch input/part3.img $ ./genimage --outputpath=images --inputpath=input --rootpath=/no/such/dir --tmppath=tmp --config test/hdimage.config $ date ; du -B 1 images/test.hdimage-2 Fri Nov 1 08:20:39 PM CET 2024 61440 images/test.hdimage-2 # Let time pass... $ date ; du -B 1 images/test.hdimage-2 Fri Nov 1 08:21:10 PM CET 2024 65536 images/test.hdimage-2 Signed-off-by: Rasmus Villemoes <[email protected]>
|
I amended the commit to update the test case, and added a description of how to reproduce the current somewhat unexpected behaviour. |
Using genimage directly on an eMMC on target, I'm seeing that the BLKRRPART ioctl fails with EBUSY, presumably because there's still lots of data in flight, and then subsequent parts of my bootstrap procedure fail because the expected partitions can't be found.
Whenever we've written some data, we really should ensure that all data has hit the disk before proceeding, and close() itself is not synchronous.
Add a fsync_close() helper that does exactly what it says on the tin. Use that wherever we close an fd that has been open for writing and actually written to (i.e., no point in doing that in the reload_partitions() function).
Currently, the return value of these close() calls are ignored, so at least for now continue to do that, but at least we do see an error message in case something went wrong.