I have 3 FRR routers on 3 different shared segments in different VRFs. The VyOS router is the uplink for the other two FRR nodes, which happens to be ubuntu 24.04 servers. All of them have FRR 10+ installed:
LBS-RO1# show ver FRRouting 10.2.2 (LBS-RO1) on Linux(6.6.79-vyos). sauna-vm1# show ver FRRouting 10.3 (sauna-vm1) on Linux(6.8.0-56-generic). lbs-vm1# show ver FRRouting 10.3 (lbs-vm1) on Linux(6.8.0-56-generic).
VyOS version: VyOS 2025.03.28-0018-rolling
The topology looks like this:
- LBS-RO1 DR <-FULL-> SAUNA-VM1 DROTHER
- LBS-RO1 DR <-FULL-> LBS-VM1 DROTHER
- LBS-VM1 DROTHER <-2-WAY-> SAUNA-VM1 DROTHER
IPv6 addressing before change:
LBS-RO1: (VyOS.)
- VLAN 4001: FE80::1:1/64 Router ID: 10.1.255.1
- VLAN 4002: FE80::2:1/64 Router ID: 10.2.255.1
- VLAN 4003: FE80::3:1/64 Router ID: 10.3.255.1
LBS-VM1: (Ubuntu 24.04)
- VLAN 4001: FE80::31:1/64 Router ID: 10.1.0.1
- VLAN 4002: FE80::32:1/64 Router ID: 10.2.0.1
- VLAN 4003: FE80::33:1/64 Router ID: 10.3.0.1
SAUNA-VM1: (Ubuntu 24.04)
- VLAN 4001: FE80::41:1/64 Router ID: 10.10.0.1
- VLAN 4002: FE80::42:1/64 Router ID: 10.11.0.1
- VLAN 4003: FE80::43:1/64 Router ID: 10.9.0.1
As you can see, all neighbors are working fine at the moment:
vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf NMS neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.1.0.1 0 00:00:38 Full/DROther 00:03:16 eth1.4001[DR] 10.10.0.1 0 00:00:31 Full/DROther 00:03:10 eth1.4001[DR] vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf INT neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.2.0.1 0 00:00:34 Full/DROther 00:03:20 eth1.4002[DR] 10.11.0.1 0 00:00:30 Full/DROther 00:03:15 eth1.4002[DR] vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf PUB neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.3.0.1 0 00:00:30 Full/DROther 00:03:24 eth1.4003[DR] 10.9.0.1 0 00:00:39 Full/DROther 00:03:18 eth1.4003[DR]
I use to hardcode the IPv6 link-local addresses on my VLAN interfaces for easy identification. Today I decided that I'm going to change the link-local addresses on the uplink router (VyOS) for all VLAN interfaces to be FE80::1/64. This should not be any problem since they are only local to the link.
However, OSPFv3 neighborship doesn't form after change, or it forms only on one of the interfaces. The others get stuck in INIT state on one side.
This is what happens when I change the IPv6 address on the SAUNA-VM1 interfaces to all be FE80::4:1/64:
sauna-vm1# show ipv6 ospf6 neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.1.0.1 0 00:00:31 Twoway/DROther 00:06:08 br4001[DROther] 10.1.255.1 1 00:00:36 Init/DR 00:06:13 br4001[DROther] sauna-vm1# show ipv6 ospf6 vrf PUB neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.3.0.1 0 00:00:37 Twoway/DROther 00:06:22 br4003[DROther] 10.3.255.1 1 00:00:32 Init/DR 00:06:35 br4003[DROther] sauna-vm1# show ipv6 ospf6 vrf INT neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.2.0.1 0 00:00:38 Twoway/DROther 00:06:31 br4002[DROther] 10.2.255.1 1 00:00:32 Full/DR 00:06:40 br4002[DROther]
Explanation:
10.X.0.1 is the other Ubuntu server running FRR. 2-way is expected because both servers are configured with ospf6 priority 0.
10.X.255.1 is the VyOS router. Only one neighbor at random formed.
When debugging on the VyOS router, I can only see Hello messages from the FRR node that has different link-local addresses on each interface. The one with the same I can only see one of the interfaces:
LBS-RO1# debug ospf6 neighbor 2025-03-29 18:12:46.365 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.0.1%eth1.4002: *HelloReceived* 2025-03-29 18:12:46.365 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.0.1%eth1.4001: *HelloReceived* 2025-03-29 18:12:46.365 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.0.1%eth1.4003: *HelloReceived* 2025-03-29 18:12:47.901 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.11.0.1%eth1.4002: *HelloReceived* 2025-03-29 18:12:56.368 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.0.1%eth1.4003: *HelloReceived* 2025-03-29 18:12:56.368 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.0.1%eth1.4002: *HelloReceived* 2025-03-29 18:12:56.368 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.0.1%eth1.4001: *HelloReceived* 2025-03-29 18:12:57.902 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.11.0.1%eth1.4002: *HelloReceived* 2025-03-29 18:13:06.371 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.0.1%eth1.4002: *HelloReceived* 2025-03-29 18:13:06.371 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.0.1%eth1.4001: *HelloReceived* 2025-03-29 18:13:06.371 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.0.1%eth1.4003: *HelloReceived* 2025-03-29 18:13:07.904 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.11.0.1%eth1.4002: *HelloReceived*
When debugging on the the other Ubuntu server, I can see helloes from all expected neighbors and interfaces:
lbs-vm1# debug ospf6 neighbor 2025-03-29 17:19:53.403 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.10.0.1%br4001: *HelloReceived* 2025-03-29 17:19:58.462 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.11.0.1%br4002: *HelloReceived* 2025-03-29 17:19:59.997 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.9.0.1%br4003: *HelloReceived* 2025-03-29 17:20:01.676 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.255.1%br4002: *HelloReceived* 2025-03-29 17:20:01.676 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.255.1%br4003: *HelloReceived* 2025-03-29 17:20:01.676 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.255.1%br4001: *HelloReceived*
Debugging from the router with identical link-local addresses, all neighbors are visible as well:
sauna-vm1# debug ospf6 neighbor 2025-03-29 18:24:16.920 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.0.1%br4003: *HelloReceived* 2025-03-29 18:24:16.920 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.0.1%br4001: *HelloReceived* 2025-03-29 18:24:16.920 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.0.1%br4002: *HelloReceived* 2025-03-29 18:24:21.542 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.3.255.1%br4003: *HelloReceived* 2025-03-29 18:24:21.542 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.1.255.1%br4001: *HelloReceived* 2025-03-29 18:24:21.542 [DEBG] ospf6d: [TT486-9H7F8] Neighbor Event 10.2.255.1%br4002: *HelloReceived*
Now I have restored the configuration on the SAUNA-VM1 node and instead changed the LBS-RO1 interfaces to all be FE80::1/64:
del interfaces ethernet eth1 vif 4001 address 'FE80::1:1/64' del interfaces ethernet eth1 vif 4002 address 'FE80::2:1/64' del interfaces ethernet eth1 vif 4003 address 'FE80::3:1/64' set interfaces ethernet eth1 vif 4001 address 'FE80::1/64' set interfaces ethernet eth1 vif 4002 address 'FE80::1/64' set interfaces ethernet eth1 vif 4003 address 'FE80::1/64' commit vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf NMS neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.1.0.1 0 00:00:36 Init/DROther 00:00:03 eth1.4001[DR] 10.10.0.1 0 00:00:38 Init/DROther 00:00:01 eth1.4001[DR] vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf INT neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.2.0.1 0 00:00:34 Full/DROther 00:00:15 eth1.4002[DR] 10.11.0.1 0 00:00:30 Full/DROther 00:00:15 eth1.4002[DR] vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf PUB neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 10.3.0.1 0 00:00:39 Init/DROther 00:00:20 eth1.4003[DR] 10.9.0.1 0 00:00:36 Init/DROther 00:00:23 eth1.4003[DR]
In the vtysh terminal on LBS-RO1: I discovered these messages:
LBS-RO1# terminal monitor 2025-03-29 19:03:37.945 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 17: Operation not permitted (1) 2025-03-29 19:03:37.945 [ERR!] ospf6d: [YRWJS-7QHSG][EC 100663307] Could not send entire message 2025-03-29 19:03:38.391 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 15: Operation not permitted (1) 2025-03-29 19:03:38.391 [ERR!] ospf6d: [YRWJS-7QHSG][EC 100663307] Could not send entire message 2025-03-29 19:03:47.945 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 17: Operation not permitted (1) 2025-03-29 19:03:47.945 [ERR!] ospf6d: [YRWJS-7QHSG][EC 100663307] Could not send entire message 2025-03-29 19:03:48.391 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 15: Operation not permitted (1) 2025-03-29 19:03:48.391 [ERR!] ospf6d: [YRWJS-7QHSG][EC 100663307] Could not send entire message 2025-03-29 19:03:57.945 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 17: Operation not permitted (1) 2025-03-29 19:03:57.945 [ERR!] ospf6d: [YRWJS-7QHSG][EC 100663307] Could not send entire message 2025-03-29 19:03:58.391 [WARN] ospf6d: [QDEV2-4Y01P] sendmsg failed: source: fe80::1 Dest: ff02::5 ifindex: 15: Operation not permitted (1)
I think you got enough information now.