I found this problem in 1.2.1. Problem did not occur in 1.1.8.
I'm sure it can happen in other situations, but i found it because the VIP address for VRRP is not within the same subnet as my interfaces.
vyos@FW# show interfaces ethernet eth0
address 169.254.123.9/30
duplex auto
hw-id 52:54:00:42:1a:6f
smp-affinity auto
speed auto
vyos@FW# show interfaces ethernet eth5
address 192.168.10.2/29
duplex auto
hw-id 52:54:00:33:0e:ca
smp-affinity auto
speed auto
vyos@FW# show high-availability vrrp group eth0-200
advertise-interval 1
authentication {
password apaswd
type ah
}
interface eth0
virtual-address 192.168.22.12/24
vrid 200
vyos@FW# show protocols static route 0.0.0.0/0
next-hop 192.168.22.1 {
}
vyos@FW# show protocols static route 192.168.0.0/16
next-hop 192.168.10.4 {
}When the firewall starts, all the interfaces come up and routes are applied. So 192.168.22.1 is pointed to 192.168.10.4.
Then VRRP starts and creates the interface 192.168.22.12/24, so logically the route to 192.168.22.1 is directly connected via eth0. But the route doesn't update to reflect that. Instead we get a recursive default route.
S> 0.0.0.0/0 [1/0] via 192.168.22.1 (recursive), 00:00:40 * via 192.168.10.4, eth5, 00:00:40 C>* 192.168.22.0/24 is directly connected, eth0, 00:00:32
If i remove the greater prefix and reboot it works fine (of course) and if i delete and re-add the default route after the firewall has booted it works fine. But that isn't much of a permanent fix.
vyos@FW# delete protocols static route 0.0.0.0/0
[edit]
vyos@FW# commit
[edit]
vyos@FW# set protocols static route 0.0.0.0/0 next-hop 192.168.22.1
[edit]
vyos@FW# compare
[edit protocols static]
+route 0.0.0.0/0 {
+ next-hop 192.168.22.1 {
+ }
+}
[edit]
vyos@FW# commit
vyos@FW:~$ show ip route
S>* 0.0.0.0/0 [1/0] via 192.168.22.1, eth0, 00:00:39Could potentially be a problem when VRRP transitions from one firewall to another too.
We need to either make sure keepalived starts before static routes are applied, or FRR needs to be able to see when static routes change and update as needed.