When setting up IPv6 BGP peers using link-local IP addressing, often the peering would become established and routes exchanged, however the next-hop IP for those routes would be marked as "inaccessible", for example:
~$ show bgp ipv6 fd42:d42:d42:54::1
BGP routing table entry for fd42:d42:d42:54::/64, version 261950
Paths: (2 available, no best path)
Not advertised to any peer
4242421588 4242421888 4242423914
:: (inaccessible) from fe80::1588 (172.20.16.142)
(fe80::1588) (used)
Origin IGP, invalid, external
Large Community: 4242421888:100:41 4242421888:101:1276
Last update: Tue Jan 21 08:46:46 2025
4242421816 4242423914
:: (inaccessible) from fe80::102 (172.23.246.4)
(fe80::102) (used)
Origin IGP, invalid, external
Large Community: 4242421816:10000:52 4242421816:10001:344
Last update: Mon Jan 20 12:35:58 2025Usually this happens when I do a "reset bgp ipv6 x.x.x.x" to reset the peering. Prior to entering this command, the BGP routes would be OK. After resetting the peering, the next-hops would be marked as inaccessible.
I am able to ping the next-hop (fe80::1588) even when it is marked as inaccessible:
~$ ping fe80::1588%wg4242421588 PING fe80::1588%wg4242421588(fe80::1588%wg4242421588) 56 data bytes 64 bytes from fe80::1588%wg4242421588: icmp_seq=1 ttl=64 time=48.2 ms 64 bytes from fe80::1588%wg4242421588: icmp_seq=2 ttl=64 time=48.0 ms 64 bytes from fe80::1588%wg4242421588: icmp_seq=3 ttl=64 time=47.8 ms 64 bytes from fe80::1588%wg4242421588: icmp_seq=4 ttl=64 time=48.1 ms 64 bytes from fe80::1588%wg4242421588: icmp_seq=5 ttl=64 time=47.9 ms 64 bytes from fe80::1588%wg4242421588: icmp_seq=6 ttl=64 time=48.1 ms ^C --- fe80::1588%wg4242421588 ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5007ms rtt min/avg/max/mdev = 47.826/48.013/48.162/0.117 ms
If I reboot the device, it fixes the issue and the next-hops are marked as OK in the above output. If I do a tcpdump on the Wireguard interface, it fixes the issue as well. I have spoken to others running Vyos 1.5 using IPv6 LL's for BGP and they are reporting similar issues, and similar fixes (rebooting device or running tcpdump on interface)
In the BGP debug logs I see this:
Jan 19 16:05:46 dn42-hub-fw01 bgpd[1344]: [HKWM3-ZC5QP] fe80::1588 fd -1 went from Established to Clearing Jan 19 16:05:46 dn42-hub-fw01 bgpd[1344]: [ZWCSR-M7FG9] fe80::1588 [FSM] Clearing_Completed (Clearing->Idle), fd -1 Jan 19 16:05:46 dn42-hub-fw01 bgpd[1344]: [HKWM3-ZC5QP] fe80::1588 fd -1 went from Clearing to Idle Jan 19 16:05:48 dn42-hub-fw01 bgpd[1344]: [ZQTB5-H8522] fe80::1588 [FSM] Timer (start timer expire). Jan 19 16:05:48 dn42-hub-fw01 bgpd[1344]: [ZWCSR-M7FG9] fe80::1588 [FSM] BGP_Start (Idle->Connect), fd -1 Jan 19 16:05:48 dn42-hub-fw01 bgpd[1344]: [Z195V-FNKRK] fe80::1588 [Event] Connect start to fe80::1588 fd 25 Jan 19 16:05:48 dn42-hub-fw01 bgpd[1344]: [G0837-S7QES] fe80::1588 [FSM] Non blocking connect waiting result, fd 25 Jan 19 16:05:48 dn42-hub-fw01 bgpd[1344]: [P90Z5-FP0GJ][EC 33554461] fe80::1588: nexthop_set failed, local: [fe80::100]:37621 remote: (null)p update_if: wg4242421588 resetting connection - intf (Unknown)
It seems like the BGP connect check is failing for some reason. I tried using "disable-connect-check" on the BGP peer config but that has not helped.
Given this is fixed by a reboot or even tcpdump on the interface it doesn't seem like normal behaviour...
Here is my config:
wireguard wg4242421588 {
address 172.20.47.227/32
address fe80::100/64
peer sg-sin01 {
address x.x.x.x
allowed-ips 0.0.0.0/0
allowed-ips ::/0
persistent-keepalive 60
port 53403
public-key xxxxx
}
port 21588
private-key xxxxx
}neighbor fe80::1588 {
interface {
source-interface wg4242421588
v6only {
}
}
peer-group generic_ipv6
remote-as 4242421588
update-source wg4242421588
}
peer-group generic_ipv6 {
address-family {
ipv6-unicast {
soft-reconfiguration {
inbound
}
}
}
ebgp-multihop 20
}~$ show version Version: VyOS 1.5-rolling-202412100007 Release train: current Release flavor: generic Built by: autobuild@vyos.net Built on: Tue 10 Dec 2024 00:07 UTC Build UUID: 1609a9df-717f-4871-b4d3-301696b398d3 Build commit ID: 0ba21e93c8213e Architecture: x86_64 Boot via: installed image System type: KVM guest Secure Boot: n/a (BIOS) Hardware vendor: QEMU Hardware model: Standard PC (i440FX + PIIX, 1996) Hardware S/N: Hardware UUID: fcfe9107-a9a6-4a32-a16d-fdc872d30574 Copyright: VyOS maintainers and contributors