Details
In some cases zebra does not redistribute connected routes to other protocols. For example, if there is more than one address added to an interface at the same time.
A configuration example to reproduce the problem:
set high-availability vrrp group vrrp1 interface 'eth0' set high-availability vrrp group vrrp1 virtual-address 192.168.123.1/24 set high-availability vrrp group vrrp1 virtual-address 192.168.123.2/24 set high-availability vrrp group vrrp1 vrid '100' set interfaces ethernet eth0 address '192.168.122.5/24' set protocols bgp 65000 address-family ipv4-unicast redistribute connected
Routing tables will look like this:
[email protected]:~$ show ip route Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued, r - rejected, b - backup C>* 192.168.122.0/24 is directly connected, eth0, 00:02:12 C * 192.168.123.0/24 is directly connected, eth0, 00:00:45 C>* 192.168.123.0/24 is directly connected, eth0, 00:00:45 [email protected]:~$ show ip bgp BGP table version is 1, local router ID is 192.168.123.2, vrf id 0 Default local pref 100, local AS 65000 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 192.168.122.0/24 0.0.0.0 0 32768 ? Displayed 1 routes and 1 total paths
A faster way to reproduce
To reproduce this it is enough to add to config only:
set protocols bgp 65000 address-family ipv4-unicast redistribute connected
And then try these two commands:
sudo ip a add 192.168.123.1/24 dev eth0 && sudo ip a add 192.168.123.2/24 dev eth0 sudo ip a add 192.168.123.1/24 dev eth0 & sudo ip a add 192.168.123.2/24 dev eth0
The first command will work as expected because addresses are added one by one:
[email protected]:~$ sudo ip a add 192.168.123.1/24 dev eth0 && sudo ip a add 192.168.123.2/24 dev eth0 [email protected]:~$ show ip route Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued, r - rejected, b - backup C * 192.168.123.0/24 is directly connected, eth0, 00:00:02 C>* 192.168.123.0/24 is directly connected, eth0, 00:00:02 [email protected]:~$ show ip bgp BGP table version is 5, local router ID is 192.168.123.2, vrf id 0 Default local pref 100, local AS 65000 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 192.168.123.0/24 0.0.0.0 0 32768 ? Displayed 1 routes and 1 total paths [email protected]:~$ sudo ip a flush dev eth0
But the second one:
[email protected]:~$ sudo ip a add 192.168.123.1/24 dev eth0 & sudo ip a add 192.168.123.2/24 dev eth0 [1] 3312 [1]+ Done sudo ip a add 192.168.123.1/24 dev eth0 [email protected]:~$ show ip route Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued, r - rejected, b - backup C * 192.168.123.0/24 is directly connected, eth0, 00:00:03 C>* 192.168.123.0/24 is directly connected, eth0, 00:00:03 [email protected]:~$ show ip bgp No BGP prefixes displayed, 0 exist [email protected]:~$ sudo ip a flush dev eth0
FRR debug for the normal command is:
Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWADDR(20), len=80, seq=1665165771, pid=3362 Oct 07 18:02:50 vyos zebra[940]: netlink_interface_addr RTM_NEWADDR eth0 flags 0x80: Oct 07 18:02:50 vyos zebra[940]: IFA_LOCAL 192.168.123.1/24 Oct 07 18:02:50 vyos zebra[940]: IFA_ADDRESS 192.168.123.1/24 Oct 07 18:02:50 vyos zebra[940]: IFA_CACHEINFO pref -1, valid -1 Oct 07 18:02:50 vyos zebra[940]: MESSAGE: ZEBRA_INTERFACE_ADDRESS_ADD 192.168.123.1/24 on eth0(0) Oct 07 18:02:50 vyos zebra[940]: rib_add_multipath_nhe: (0:0):192.168.123.0/24: Inserting route rn 0x560dcb8dfd00, re 0x560dcb8dfc20 (connected) existing (nil) Oct 07 18:02:50 vyos zebra[940]: rib_add_multipath_nhe: (0:0):192.168.123.0/24 (MRIB): Inserting route rn 0x560dcb8de610, re 0x560dcb8dd6d0 (connected) existing (nil) Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:02:50 vyos zebra[940]: RTM_NEWROUTE ipv4 local proto kernel NS 0 Oct 07 18:02:50 vyos zebra[940]: Route rtm_type: local(2) intentionally ignoring Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:02:50 vyos zebra[940]: RTM_NEWROUTE ipv4 unicast proto kernel NS 0 Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:02:50 vyos zebra[940]: RTM_NEWROUTE ipv4 broadcast proto kernel NS 0 Oct 07 18:02:50 vyos zebra[940]: Route rtm_type: broadcast(3) intentionally ignoring Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:02:50 vyos zebra[940]: RTM_NEWROUTE ipv4 broadcast proto kernel NS 0 Oct 07 18:02:50 vyos zebra[940]: Route rtm_type: broadcast(3) intentionally ignoring Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24: Adding route rn 0x560dcb8dfd00, re 0x560dcb8dfc20 (connected) Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24: Adding route rn 0x560dcb8de610, re 0x560dcb8dd6d0 (connected) Oct 07 18:02:50 vyos zebra[940]: netlink_nexthop_msg_encode: ID (19): directly connected, eth0(2) vrf default(0) Oct 07 18:02:50 vyos zebra[940]: netlink_nexthop_msg_encode: RTM_NEWNEXTHOP, id=19 Oct 07 18:02:50 vyos zebra[940]: nl_batch_send: netlink-dp (NS 0), batch size=40, msg cnt=1 Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): no fib nhg Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): rib nhg matched, changed 'false' Oct 07 18:02:50 vyos zebra[940]: (0:254):192.168.123.0/24: Redist update re 0x560dcb8dfc20 (connected), old 0x0 (None) Oct 07 18:02:50 vyos zebra[940]: redistribute_update: client bgp 192.168.123.0/24(0:254), type=2, distance=0, metric=0 Oct 07 18:02:50 vyos zebra[940]: zsend_redistribute_route: ZEBRA_REDISTRIBUTE_ROUTE_ADD to client bgp: type connected, vrf_id 0, p 192.168.123.0/24 Oct 07 18:02:50 vyos zebra[940]: redistribute_update: client ldp 192.168.123.0/24(0:254), type=2, distance=0, metric=0 Oct 07 18:02:50 vyos zebra[940]: zsend_redistribute_route: ZEBRA_REDISTRIBUTE_ROUTE_ADD to client ldp: type connected, vrf_id 0, p 192.168.123.0/24 Oct 07 18:02:50 vyos zebra[940]: Not Notifying Owner: connected about prefix 192.168.123.0/24(254) 2 vrf: 0 Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWADDR(20), len=80, seq=1665165771, pid=3364 Oct 07 18:02:50 vyos zebra[940]: netlink_interface_addr RTM_NEWADDR eth0 flags 0x81: Oct 07 18:02:50 vyos zebra[940]: IFA_LOCAL 192.168.123.2/24 Oct 07 18:02:50 vyos zebra[940]: IFA_ADDRESS 192.168.123.2/24 Oct 07 18:02:50 vyos zebra[940]: IFA_CACHEINFO pref -1, valid -1 Oct 07 18:02:50 vyos zebra[940]: MESSAGE: ZEBRA_INTERFACE_ADDRESS_ADD 192.168.123.2/24 on eth0(0) Oct 07 18:02:50 vyos zebra[940]: rib_add_multipath_nhe: (0:254):192.168.123.0/24: Inserting route rn 0x560dcb8dfd00, re 0x560dcb8de6f0 (connected) existing (nil) Oct 07 18:02:50 vyos zebra[940]: rib_add_multipath_nhe: (0:254):192.168.123.0/24 (MRIB): Inserting route rn 0x560dcb8de610, re 0x560dcb8de100 (connected) existing (nil) Oct 07 18:02:50 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:02:50 vyos zebra[940]: RTM_NEWROUTE ipv4 local proto kernel NS 0 Oct 07 18:02:50 vyos zebra[940]: Route rtm_type: local(2) intentionally ignoring Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24: Adding route rn 0x560dcb8de610, re 0x560dcb8dd6d0 (connected) Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): no fib nhg Oct 07 18:02:50 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): rib nhg matched, changed 'false' Oct 07 18:02:50 vyos zebra[940]: Not Notifying Owner: connected about prefix 192.168.123.0/24(254) 2 vrf: 0
And in case of problem:
Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWADDR(20), len=80, seq=1665165832, pid=3404 Oct 07 18:03:51 vyos zebra[940]: netlink_interface_addr RTM_NEWADDR eth0 flags 0x80: Oct 07 18:03:51 vyos zebra[940]: IFA_LOCAL 192.168.123.2/24 Oct 07 18:03:51 vyos zebra[940]: IFA_ADDRESS 192.168.123.2/24 Oct 07 18:03:51 vyos zebra[940]: IFA_CACHEINFO pref -1, valid -1 Oct 07 18:03:51 vyos zebra[940]: MESSAGE: ZEBRA_INTERFACE_ADDRESS_ADD 192.168.123.2/24 on eth0(0) Oct 07 18:03:51 vyos zebra[940]: rib_add_multipath_nhe: (0:0):192.168.123.0/24: Inserting route rn 0x560dcb8dfd00, re 0x560dcb8de180 (connected) existing (nil) Oct 07 18:03:51 vyos zebra[940]: rib_add_multipath_nhe: (0:0):192.168.123.0/24 (MRIB): Inserting route rn 0x560dcb8de820, re 0x560dcb8de700 (connected) existing (nil) Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:03:51 vyos zebra[940]: RTM_NEWROUTE ipv4 local proto kernel NS 0 Oct 07 18:03:51 vyos zebra[940]: Route rtm_type: local(2) intentionally ignoring Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:03:51 vyos zebra[940]: RTM_NEWROUTE ipv4 unicast proto kernel NS 0 Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:03:51 vyos zebra[940]: RTM_NEWROUTE ipv4 broadcast proto kernel NS 0 Oct 07 18:03:51 vyos zebra[940]: Route rtm_type: broadcast(3) intentionally ignoring Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:03:51 vyos zebra[940]: RTM_NEWROUTE ipv4 broadcast proto kernel NS 0 Oct 07 18:03:51 vyos zebra[940]: Route rtm_type: broadcast(3) intentionally ignoring Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWADDR(20), len=80, seq=1665165832, pid=3405 Oct 07 18:03:51 vyos zebra[940]: netlink_interface_addr RTM_NEWADDR eth0 flags 0x81: Oct 07 18:03:51 vyos zebra[940]: IFA_LOCAL 192.168.123.1/24 Oct 07 18:03:51 vyos zebra[940]: IFA_ADDRESS 192.168.123.1/24 Oct 07 18:03:51 vyos zebra[940]: IFA_CACHEINFO pref -1, valid -1 Oct 07 18:03:51 vyos zebra[940]: MESSAGE: ZEBRA_INTERFACE_ADDRESS_ADD 192.168.123.1/24 on eth0(0) Oct 07 18:03:51 vyos zebra[940]: rib_add_multipath_nhe: (0:254):192.168.123.0/24: Inserting route rn 0x560dcb8dfd00, re 0x560dcb8dda60 (connected) existing (nil) Oct 07 18:03:51 vyos zebra[940]: rib_add_multipath_nhe: (0:254):192.168.123.0/24 (MRIB): Inserting route rn 0x560dcb8de820, re 0x560dcb8d8140 (connected) existing (nil) Oct 07 18:03:51 vyos zebra[940]: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=60, seq=0, pid=0 Oct 07 18:03:51 vyos zebra[940]: RTM_NEWROUTE ipv4 local proto kernel NS 0 Oct 07 18:03:51 vyos zebra[940]: Route rtm_type: local(2) intentionally ignoring Oct 07 18:03:51 vyos zebra[940]: default(0:254):192.168.123.0/24: Adding route rn 0x560dcb8dfd00, re 0x560dcb8de180 (connected) Oct 07 18:03:51 vyos zebra[940]: default(0:254):192.168.123.0/24: Adding route rn 0x560dcb8de820, re 0x560dcb8de700 (connected) Oct 07 18:03:51 vyos zebra[940]: netlink_nexthop_msg_encode: ID (22): directly connected, eth0(2) vrf default(0) Oct 07 18:03:51 vyos zebra[940]: netlink_nexthop_msg_encode: RTM_NEWNEXTHOP, id=22 Oct 07 18:03:51 vyos zebra[940]: nl_batch_send: netlink-dp (NS 0), batch size=40, msg cnt=1 Oct 07 18:03:51 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): no fib nhg Oct 07 18:03:51 vyos zebra[940]: default(0:254):192.168.123.0/24 update_from_ctx(): rib nhg matched, changed 'false' Oct 07 18:03:51 vyos zebra[940]: Not Notifying Owner: connected about prefix 192.168.123.0/24(254) 2 vrf: 0
As we can see, the redistribution event is missed in the second case:
Oct 07 18:02:50 vyos zebra[940]: (0:254):192.168.123.0/24: Redist update re 0x560dcb8dfc20 (connected), old 0x0 (None) Oct 07 18:02:50 vyos zebra[940]: redistribute_update: client bgp 192.168.123.0/24(0:254), type=2, distance=0, metric=0 Oct 07 18:02:50 vyos zebra[940]: zsend_redistribute_route: ZEBRA_REDISTRIBUTE_ROUTE_ADD to client bgp: type connected, vrf_id 0, p 192.168.123.0/24 Oct 07 18:02:50 vyos zebra[940]: redistribute_update: client ldp 192.168.123.0/24(0:254), type=2, distance=0, metric=0 Oct 07 18:02:50 vyos zebra[940]: zsend_redistribute_route: ZEBRA_REDISTRIBUTE_ROUTE_ADD to client ldp: type connected, vrf_id 0, p 192.168.123.0/24
Additional info
The bug cannot be reproduced in FRRouting 8.0+. We should consider FRR update or find and backport the change that fixes this to FRR 7.5.1, if this is possible.