Page MenuHomeVyOS Platform

Default route is inactive if an interface has multiple ip addresses of the same subnet in 1.3.2 Equuleus
Closed, ResolvedPublicBUG

Description

I manage few instances of VyOS routers and I build iso images of 1.3 Equuleus myself.

Recently, I found that instances running my images of Equuleus built in March 2023 fail to upgrade with my images built starting from the end of April. So, I had to revert back to the previous image every time I tried to upgrade.

So, my latest working image was Equuleus 1.3-rolling-202303080842.
And all images built 2023-04-30 and later fail when I try to upgrade that working image. The latest image I tried was built today, May 27th.

I believe I found the reason of failure. Though I don't really know if it is a bug or an intended change. Anyway, since the new versions break working configurations I believe it is worth sharing.

After the upgrade my VyOS instances were offline. After investigation I found that the default route is inactive:

vyos@vyos# run show ip rou
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S   0.0.0.0/0 [1/0] via 1xx.xxx.xxx.57, eth0 inactive, weight 1, 00:00:12

Usually, default route becomes inactive when the interface is not directly connected, so the router doesn't know how to reach the next hop. Indeed, my eth0 ip address subnet was not in the route table at all, however it was expected to be there like this:

C>* 1xx.xxx.xxx.56/29 is directly connected, eth0, 01:07:36

After investigation I found that I must change the syntax of the ip addresses for the eth0 interface from (what I had in all my previous images):

vyos@vyos# show interfaces ethernet 
 ethernet eth0 {
     address 1xx.xxx.xxx.58/29
     address 1xx.xxx.xxx.59/29
     address 1xx.xxx.xxx.60/29
     address 1xx.xxx.xxx.61/29
     address 1xx.xxx.xxx.62/29

to:

vyos@vyos# show interfaces ethernet 
 ethernet eth0 {
     address 1xx.xxx.xxx.58/29
     address 1xx.xxx.xxx.59/32
     address 1xx.xxx.xxx.60/32
     address 1xx.xxx.xxx.61/32
     address 1xx.xxx.xxx.62/32

After this change in the configuration everything starts working as expected. To my understanding, the required change is not trivial. Probably some zebra defaults had changed in the latest builds of VyOS 1.3 and it affects the working configurations.

So, finally I have the following working configuration:

vyos@vyos# show interfaces ethernet 
 ethernet eth0 {
     address 1xx.xxx.xxx.58/29
     address 1xx.xxx.xxx.59/32
     address 1xx.xxx.xxx.60/32
     address 1xx.xxx.xxx.61/32
     address 1xx.xxx.xxx.62/32
...
vyos@vyos# show protocols 
 static {
     route 0.0.0.0/0 {
         next-hop 1xx.xxx.xxx.57 {
             distance 1
             next-hop-interface eth0
         }
     }
...
vyos@vyos:~$ show ip rou
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* 0.0.0.0/0 [1/0] via 1xx.xxx.xxx.57, eth0, weight 1, 01:07:34
C>* 1xx.xxx.xxx.56/29 is directly connected, eth0, 01:07:36
C>* 1xx.xxx.xxx.59/32 is directly connected, eth0, 01:07:36
C>* 1xx.xxx.xxx.60/32 is directly connected, eth0, 01:07:36
C>* 1xx.xxx.xxx.61/32 is directly connected, eth0, 01:07:36
C>* 1xx.xxx.xxx.62/32 is directly connected, eth0, 01:07:36

Can anybody comment on what has changed in recent builds of Equuleus and why (with regard to the described behavior)?
Thank you.

Details

Difficulty level
Unknown (require assessment)
Version
1.3.2
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

dutty created this object in space S1 VyOS Public.

Maybe the related bug described in T4737
Could you show a version of FRR?

show version all | match frr
Viacheslav changed the subtype of this task from "Task" to "Bug".

Maybe the related bug described in T4737
Could you show a version of FRR?

show version all | match frr

in the working image that I built in March (and where I have /29 working) it is like this:

vyos@vyos:~$ show version all | match frr
ii  frr                                  7.5.1-20220709-00-g886a63a59-0 amd64        FRRouting suite of internet protocols (BGP, OSPF, IS-IS, ...)
ii  frr-pythontools                      7.5.1-20220709-00-g886a63a59-0 all          FRRouting suite - Python tools
ii  frr-rpki-rtrlib                      7.5.1-20220709-00-g886a63a59-0 amd64        FRRouting suite - BGP RPKI support (rtrlib)
ii  frr-snmp                             7.5.1-20220709-00-g886a63a59-0 amd64        FRRouting suite - SNMP support

And in the latest image (built on May 27th, where /29 is not working, and I had to change to /32) it is:

vyos@vyos:~$ show version all | match frr
ii  frr                                  7.5.1-20230524-02-gce3a50302-0 amd64        FRRouting suite of internet protocols (BGP, OSPF, IS-IS, ...)
ii  frr-pythontools                      7.5.1-20230524-02-gce3a50302-0 all          FRRouting suite - Python tools
ii  frr-rpki-rtrlib                      7.5.1-20230524-02-gce3a50302-0 amd64        FRRouting suite - BGP RPKI support (rtrlib)
ii  frr-snmp                             7.5.1-20230524-02-gce3a50302-0 amd64        FRRouting suite - SNMP support

Hope this helps.

zsdc changed the task status from Open to Needs testing.May 29 2023, 10:44 AM
zsdc added a subscriber: zsdc.

@dutty can you try to build an image again and check now?

The problem was fixed in the https://github.com/vyos/vyos-build/pull/353, but it seems CI was not triggered to update packages in a repository.

@zsdc I built the image now, and it works as expected. The issue looks resolved. Thank you.

vyos@vyos:~$ show version 

Version:          VyOS 1.3-rolling-202305291054
Release train:    equuleus

Built by:         [email protected]
Built on:         Mon 29 May 2023 10:54 UTC
Build UUID:       b085beb3-7715-4ec5-95c8-...
Build commit ID:  bc64a3a72244b9

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  Red Hat
Hardware model:   KVM
Hardware S/N:     
Hardware UUID:    e777ae6f-11ec-4af7-92ea-...

Copyright:        VyOS maintainers and contributors
Viacheslav assigned this task to zsdc.
Viacheslav moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus (1.3.3) board.