When booting a recent VyOS 1.5 rolling release from ISO on ARM64 with VMware Fusion, it will fail during VyOS config migration:
[ OK ] Finished Record Runlevel Change in UTMP. [ 6.115307] vyos-router[960]: Waiting for NICs to settle down: settled in Osec.. [ 7.618934] vyos-router[960]: Mounting VyOS Config.. done. [ 10.150214] vyos-router[960]: Starting VyOS router: migrate system configure failed! [ 10.1777351 vyos-config[963]: Configuration error
I found the issue by reloading the default config and performing commit:
vyos@vyos:~$ config
WARNING: You are currently configuring a live-ISO environment, changes will not persist until installed
[edit]
vyos@vyos# load /usr/share/vyos/config.boot.default
Loading configuration from '/usr/share/uyos/config.boot.default'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[ system console ]
Traceback (most recent call last):
File "/usr/libexec/vyos/services/vyos-configd", line 146, in run_script script.apply(c)
File "/usr/libexec/vyos//conf_mode/system_console.py", line 128, in apply restart_log in_consoles(prompt_user=False, quiet=False)
File "/usr/lib/python3/dist-packages/vyos/utils/serial.py", line 113, in restart_login_consoles cnd (f' systemctl restart {unit_name}' )
File "/usr/lib/python3/dist-packages/vyos/utils/process.py", line 189, in cmd raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: None systemctl restart serial-getty@ttyS0.service
returned:
exit code: 1
[[system console]] failed
Commit failed [edit]
vyos@vyos#As you can see it fails to start serial-getty@ttyS0.service.
A workaround to successfully load the config is to delete the console entry in the config and then commit:
delete system console commit
This error did not happen on VyOS 1.4 saggita, digging deeper I found it is caused by the following commits in systemd:
- https://github.com/systemd/systemd/commit/dc38f9addd04c34d1fd743efc407bdebb3573d05
- https://github.com/systemd/systemd/commit/9da582984b632a75f562a304ff96dd1043281430
This comment explains a bit why it fails on ARM64: https://github.com/systemd/systemd/issues/36497#issuecomment-2686348656
If I revert that systemd commit / udev rule, it can successfully boot the VyOS config as the serial-getty@ttyS0.service will successfully start.
While now we know what causes it, i'm not sure what is best way forward to fix this issue on arm64:
- revert the commit specifically on arm64 builds
- don't add serial console to default boot config on arm64
- any other option?
I opened this task for discussion and further feedback. I'm happy to test anything on arm64. I have also linked a possible related forum thread, although it's not arm64 related but to booting as a VM without a serial port.