When VPP crashes, vpp-failure-handler.service should restart it by resetting
the "vpp" section: delete it, commit, then set it again and commit.
This stopped working after the VPP CLI rewrite that moved VPP interfaces from
"vpp interfaces ..." to "interfaces vpp ..." (T8283, T8324, T8327, ...).
Before, the interfaces lived inside the "vpp" section, so the handler deleted
and restored them along with it. Now they are a separate section, and commit
refuses to delete "vpp" while any of them exist:
VPP cannot be removed while VPP interfaces exist. Remove all "interfaces vpp" first!
The handler fails and VPP stays down. The VPP Prometheus exporter check
is expected to block the handler in the same way.
How to reproduce:
set vpp settings interface eth1 set interfaces ethernet eth1 address 192.0.2.1/24 set interfaces vpp loopback vpplo1 address 10.255.0.1/32 commit
Kill vpp process:
sudo systemctl kill -s SIGSEGV vpp.service
Log:
Oct 28 11:12:28 vyos systemd[1]: Starting Restart VPP on failure...
Oct 28 11:12:28 vyos vyos-configd[769]: Received message: {"type": "init"}
Oct 28 11:12:29 vyos vyos-netlinkd[2739]: Config commit in progress, draining netlink events without acting
Oct 28 11:12:29 vyos vyos-configd[769]: config session pid is 15101
Oct 28 11:12:29 vyos vyos-configd[769]: config session sudo_user is root
Oct 28 11:12:29 vyos vyos-configd[769]: commit_scripts: ['vpp']
Oct 28 11:12:29 vyos vyos-configd[769]: Received message: {"type": "node", "last": true, "data": "/usr/libexec/vyos/conf_mode/vpp.py"}
Oct 28 11:12:29 vyos vyos-configd[769]: VPP cannot be removed while VPP interfaces exist. Remove all "interfaces
Oct 28 11:12:29 vyos vyos-configd[769]: vpp" first!
Oct 28 11:12:29 vyos vyos-configd[769]: [vpp] VPP cannot be removed while VPP interfaces exist. Remove all "interfaces
Oct 28 11:12:29 vyos vyos-configd[769]: vpp" first!
Oct 28 11:12:29 vyos vyos-configd[769]: scripts_called: ['vpp']
Oct 28 11:12:29 vyos vyos-configd[769]: Sending reply: ERROR_COMMIT with output
Oct 28 11:12:29 vyos systemd[1]: opt-vyatta-config-tmp-new_config_15101.mount: Deactivated successfully.
Oct 28 11:12:30 vyos python3[15101]: Traceback (most recent call last):
Oct 28 11:12:30 vyos python3[15101]: File "/usr/libexec/vyos/reset_section.py", line 119, in <module>
Oct 28 11:12:30 vyos python3[15101]: session.commit()
Oct 28 11:12:30 vyos python3[15101]: File "/usr/lib/python3/dist-packages/vyos/configsession.py", line 354, in commit
Oct 28 11:12:30 vyos python3[15101]: out = self.__run_command([COMMIT])
Oct 28 11:12:30 vyos python3[15101]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Oct 28 11:12:30 vyos python3[15101]: File "/usr/lib/python3/dist-packages/vyos/configsession.py", line 263, in __run_command
Oct 28 11:12:30 vyos python3[15101]: raise ConfigSessionError(output)
Oct 28 11:12:30 vyos python3[15101]: vyos.configsession.ConfigSessionError: [ vpp ]
Oct 28 11:12:30 vyos python3[15101]: VPP cannot be removed while VPP interfaces exist. Remove all "interfaces
Oct 28 11:12:30 vyos python3[15101]: vpp" first!
Oct 28 11:12:30 vyos python3[15101]: delete [ vpp ] failed
Oct 28 11:12:30 vyos python3[15101]: Commit failed
Oct 28 11:12:30 vyos systemd[1]: opt-vyatta-config-tmp-new_config_15101.mount: Deactivated successfully.
Oct 28 11:12:30 vyos vyos-netlinkd[2739]: Config commit finished, reconciling DHCP client state with operstate
Oct 28 11:12:30 vyos systemd[1]: vpp-failure-handler.service: Main process exited, code=exited, status=1/FAILURE
Oct 28 11:12:30 vyos systemd[1]: vpp-failure-handler.service: Failed with result 'exit-code'.
Oct 28 11:12:30 vyos systemd[1]: Failed to start Restart VPP on failure.Expected:
VPP is restarted and comes back with its full config, including
"interfaces vpp" objects and their LCP pairs.