Page MenuHomeVyOS Platform

Implement Atomic Write Operations for Config Files
Open, NormalPublicFEATURE REQUEST

Description

Currently, VyOS directly replaces files instead of using atomic write operations, which can lead to data integrity issues. To address this, the write_file function was changed to use atomic writes with os.replace.

An implementation example can be found here: https://gist.github.com/robertoberto/56005f84093d5d76571b349fb49789de.
The original code can be found here: https://github.com/vyos/vyos-1x/blob/current/python/vyos/utils/file.py.
It is also possible to create an open wrapper to "w" operations and keep open "a" to append operarions like this https://gist.github.com/robertoberto/2c36dd863dd753d5d61f738e7f81ae5e

Reasoning:
While writing important files like /config/config.boot, the server can be shut down, or the process can be killed, leading to potential data corruption.
Therefore, for atomic writes, the first step is to create a new file with all the bytes that were going to be written anyway (as the original open() with "w" writes the full file each time).

After writing to the temp file (it should be in the same folder because os.replace does not work atomically if the folder is not in the same filesystem), the old file can be safely replaced with the temp file, ensuring atomicity.

Additionally, Python's shutil.copystat() can be used to copy metadata such as owner, permissions, and extended attributes from the original file to the new temporary file.

Previous Solutions:
There were some Python modules that performed this task, such as atomicwrites and atomicfile, but these have been deprecated.

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Perfectly compatible
Issue type
Improvement (missing useful functionality)

Related Objects

Event Timeline

Care to elaborate some on that "data integrity issues" claim since you will have integrity changed by all sort of writes (atomic or not)?

can you elaborate more your inquire?

integrity improvements over non atomic writes I belive I explained already.

there are others data integrity issues that can happen even in atomic
writes.

can you ask clearer what you want to elaborate?