Do not delete files?

[dschwoerer]

In case reading of the input file fails (#1161), it would probably best to not write a BOUT.settings - as in this case the command line arguments are not read. I got lucky as BOUT.settings contains that the settings file is BOUT.settings, but the parser changes that to bout.settings, so BOUT.settings survived, but this is less then ideal that my settings file nearly got overwritten ...

From this experience, I have been wondering whether it would be worth to not overwrite text files in general?
I guess they should be sufficiently small that in most cases we don't care about the size. In that case the log and setting files etc should be safe from accidental deletion. Rather then overwriting move the old file to a numbered backup ...

[bendudson]

Agreed, moving to a backup would be nice

[dschwoerer]

I think backups should be kept indefinitely, it would be really annoying, if my long run, often restarted simulation would suddenly decide it can delete the log of how it was started.

I guess the test should be patched to clean-up.

I can add the changes to #1508

I hope to discuss this before implementation. If a more advanced system is required/request, I am happy for other solutions, but I hope to get this fixed soon, so I can work on other stuff, without losing my logs.

[bendudson]

Thanks @dschwoerer . Agree, backup of text files would be nice, and if this is done then an arbitrary limit would be surprising. I think either:

• We always save these text files with some kind of shared key e.g. hash or a timestamp which is broadcast from processor 0. This means we never have to move or overwrite files, but would change behaviour: BOUT++ would now save "BOUT.settings.190122:17:43:21" and "BOUT.log.0.190122:17:43:21" for example.
• A numbering system, moving previous files. Could this run into trouble if a user write-protects them?

[dschwoerer]

I have implemented the numbering system.
That does indeed changes the behaviour for write protected fails (chmod -w ; but user owned). Before that failed with could not open log file - with #1508 it moves the file to a backup location.

The unique file names could be combined with sym links, such that BOUT.log.0 links to the most recent one, that would be a (mostly) backwards-compatible change, with hardlinks backwards compatible would probably be even higher ...

[ZedThree]

It would definitely be a nice feature, but it should be a runtime option, with the current behaviour as default.

Another option could be append, similar to how the restart files clobber by default but optionally append.

You can of course always backup the log files manually at any rate! :)

[dschwoerer]

While it is certainly possible to backup, it is also possible to delete unneeded files.

I think it is MUCH easier to delete. Also the impact, of not doing it in time, is mostly rather negligible.

On the other hand, restoring overwritten files is often impossible.
Backing up by hand is error prone. Setting a flag for BOUT is also error prone. A typo and the data is gone.

[dschwoerer]

If I reopen #1508 and make sure the tests get cleaned - will this be merged or rejected?
I think it is a useful feature, but I don't want to work on things that will be rejected.

[ZedThree]

It is a useful feature, but it should be optional, whether it's on or off by default.

Also note that backing up the actual output files would be more useful, given they're the interesting bits, but I really don't think we should be automatically copying them and potentially filling up all available space.