Page MenuHomeVyOS Platform
Create Task
Maniphest T7089

Static Route not working when PPPoE configured
Needs testing, HighPublicBUG
Actions

Description

PR: https://github.com/vyos/vyos-1x/pull/4318

Test system image: 1.5-rolling-202501250006

brief:

  1. no pppoe: static route works
  2. add pppoe interface
  3. change static route: not working

I use 192.168.51.0/24 for test.

vyos@vyos# show protocols static route 192.168.51.0/24 
 interface eth0 {
 }
 interface lo {
     distance 10
 }
[edit]
vyos@vyos# delete protocols static route 192.168.51.0/24 interface lo 
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# ip route show
default nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.12.0/24 dev eth0 proto kernel scope link src 192.168.12.70 
192.168.10.0/24 nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.51.0/24 nhid 7 dev eth0 proto static metric 20 
[edit]

# 192.168.51.0/24 dev eth0 appears above

vyos@vyos# delete protocols static route 192.168.51.0/24 interface eth0
[edit]
vyos@vyos# set protocols static route 192.168.51.0/24 interface lo
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# ip route show
default nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.12.0/24 dev eth0 proto kernel scope link src 192.168.12.70 
192.168.10.0/24 nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.51.0/24 nhid 13 dev lo proto static metric 20 
[edit]

# 192.168.51.0/24 changed to dev lo

vyos@vyos# show interfaces pppoe
Configuration under specified path is empty
[edit]
vyos@vyos# set interfaces pppoe pppoe0 authentication username test
[edit]
vyos@vyos# set interfaces pppoe pppoe0 authentication password test
[edit]
vyos@vyos# set interfaces pppoe pppoe0 source-interface eth1
[edit]
vyos@vyos# commit
[edit]

# pppoe added above, I tested on my vyos env (with a working pppoe config) and then using live image for minimum test env ( with random config). 


vyos@vyos# ip route show
default nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.12.0/24 dev eth0 proto kernel scope link src 192.168.12.70 
192.168.10.0/24 nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.51.0/24 nhid 13 dev lo proto static metric 20 
[edit]

# now try to change route

vyos@vyos# delete protocols static route 192.168.51.0/24 interface lo 
[edit]
vyos@vyos# set protocols static route 192.168.51.0/24 interface eth0
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# ip route show
default nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.12.0/24 dev eth0 proto kernel scope link src 192.168.12.70 
192.168.10.0/24 nhid 10 via 192.168.12.1 dev eth0 proto static metric 20 
192.168.51.0/24 nhid 13 dev lo proto static metric 20 
[edit]
vyos@vyos# 

# not changed

Details

Version
1.5-rolling-202501250006
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

sskaje created this task.Jan 26 2025, 6:07 AM
sskaje added a comment.Jan 26 2025, 6:10 AM

full config using 1.5-rolling-202501250006 live image

vyos@vyos# show 
 interfaces {
     ethernet eth0 {
         address dhcp
         hw-id bc:24:11:c5:28:9d
         offload {
             gro
             gso
             sg
             tso
         }
     }
     loopback lo {
     }
     pppoe pppoe0 {
         authentication {
             password test
             username test
         }
         source-interface eth1
     }
 }
 protocols {
     static {
         route 192.168.10.0/24 {
             next-hop 192.168.12.1 {
             }
         }
         route 192.168.51.0/24 {
             interface eth0 {
             }
         }
     }
 }
 service {
     ntp {
         allow-client {
             address 127.0.0.0/8
             address 169.254.0.0/16
             address 10.0.0.0/8
             address 172.16.0.0/12
             address 192.168.0.0/16
             address ::1/128
             address fe80::/10
             address fc00::/7
         }
         server time1.vyos.net {
         }
         server time2.vyos.net {
         }
         server time3.vyos.net {
         }
     }
     ssh {
         port 22
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vyos
     login {
         user vyos {
             authentication {
                 encrypted-password $6$QxPS.uk6mfo$9QBSo8u1FkH16gMyAVhus6fU3LOzvLR9Z9.82m3tiHFAxTtIkhaZSWssSgzt4v4dGAL8rhVQxTg0oAG9/q11h/
                 plaintext-password ""
             }
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility local7 {
                 level debug
             }
         }
     }
 }
[edit]
pasik subscribed.Jan 26 2025, 3:15 PM
sskaje added a comment.Jan 26 2025, 6:49 PM
vyos# exit
root@vyos:/home/vyos# cat /run/frr/config/vyos.frr.conf 
!
!
!
ip route 192.168.10.0/24 192.168.12.1
ip route 192.168.51.0/24 lo
ip route 192.168.52.0/24 192.168.12.1
ip  route 0.0.0.0/0 192.168.12.1 eth0 tag 210 210
ip  route 0.0.0.0/0 pppoe0 tag 210 1!
!
!
!
ip forwarding
!
!
!
!
ipv6 forwarding
!
!
!root@vyos:/home/vyos# 
exit
[edit]
vyos@vyos# set interfaces pppoe pppoe0 no-default-route 
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# sudo su
root@vyos:/home/vyos# cat /run/frr/config/vyos.frr.conf 
!
!
!
ip route 192.168.10.0/24 192.168.12.1
ip route 192.168.51.0/24 lo
ip route 192.168.52.0/24 192.168.12.1
ip  route 0.0.0.0/0 192.168.12.1 eth0 tag 210 210
!
!
!
!
ip forwarding
!
!
!
!
ipv6 forwarding
!
!
!root@vyos:/home/vyos#

frr debug msg

Jan 26 18:46:04 vyos vyos-configd[24976]: {'ip': {'afi': 'ip', 'arp': {'table_size': '8192'}},                                                                                
Jan 26 18:46:04 vyos vyos-configd[24976]:  'ipv6': {'afi': 'ipv6', 'neighbor': {'table_size': '8192'}},                                                                       
Jan 26 18:46:04 vyos vyos-configd[24976]:  'policy': {},                                                                                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]:  'static': {'dhcp': {'eth0': {'dhcp_options': {'default_route_distance': '210'}}},                                                  
Jan 26 18:46:04 vyos vyos-configd[24976]:             'pppoe': {'pppoe0': {'default_route_distance': '1'}},                                                                   
Jan 26 18:46:04 vyos vyos-configd[24976]:             'route': {'192.168.10.0/24': {'next_hop': {'192.168.12.1': {}}},                                                        
Jan 26 18:46:04 vyos vyos-configd[24976]:                       '192.168.51.0/24': {'interface': {'lo': {}}},                                                                 
Jan 26 18:46:04 vyos vyos-configd[24976]:                       '192.168.53.0/24': {'next_hop': {'192.168.12.1': {}}}}}}                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: FRR:        START CONFIGURATION RENDERING                                                                                           
Jan 26 18:46:04 vyos vyos-configd[24976]: !                                                                                                                                   
Jan 26 18:46:04 vyos vyos-configd[24976]: message repeated 2 times: [ !]                                                                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: ip route 192.168.10.0/24 192.168.12.1                                                  
Jan 26 18:46:04 vyos vyos-configd[24976]: ip route 192.168.51.0/24 lo                                                            
Jan 26 18:46:04 vyos vyos-configd[24976]: ip route 192.168.53.0/24 192.168.12.1                                                  
Jan 26 18:46:04 vyos vyos-configd[24976]: ip  route 0.0.0.0/0 192.168.10.1 eth0 tag 210 210                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: ip  route 0.0.0.0/0 pppoe0 tag 210 1!                                                  
Jan 26 18:46:04 vyos vyos-configd[24976]: !                                                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: message repeated 2 times: [ !]                                                         
Jan 26 18:46:04 vyos vyos-configd[24976]: ip forwarding                                                                          
Jan 26 18:46:04 vyos vyos-configd[24976]: !                                                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: message repeated 3 times: [ !]                                                         
Jan 26 18:46:04 vyos vyos-configd[24976]: ipv6 forwarding                                                                        
Jan 26 18:46:04 vyos vyos-configd[24976]: !                                                                                      
Jan 26 18:46:04 vyos vyos-configd[24976]: message repeated 2 times: [ !]                                                         
Jan 26 18:46:04 vyos vyos-configd[24976]: FRR:        RENDERING CONFIG COMPLETE                                                  
Jan 26 18:46:04 vyos vyos-configd[24976]: FRR: reloading configuration - tries: 1 | Python class ID: 140435475984528
Jan 26 18:46:15 vyos vyos-configd[24976]: Report time:      2025-01-26 18:46:15                                                                                               
Jan 26 18:46:15 vyos vyos-configd[24976]: Image version:    VyOS 1.5-rolling-202501250006                                                                                     
Jan 26 18:46:15 vyos vyos-configd[24976]: Release train:    current                                                                                                           
Jan 26 18:46:15 vyos vyos-configd[24976]: Built by:         autobuild@vyos.net                                                                                                
Jan 26 18:46:15 vyos vyos-configd[24976]: Built on:         Sat 25 Jan 2025 00:07 UTC                                                                                         
Jan 26 18:46:15 vyos vyos-configd[24976]: Build UUID:       75f98890-9b25-4959-8bde-3c1476c6303a                                                                              
Jan 26 18:46:15 vyos vyos-configd[24976]: Build commit ID:  16f9226b1f8696                                                                                                    
Jan 26 18:46:15 vyos vyos-configd[24976]: Architecture:     x86_64                                                                                                            
Jan 26 18:46:15 vyos vyos-configd[24976]: Boot via:         livecd                                                                                                            
Jan 26 18:46:15 vyos vyos-configd[24976]: System type:      KVM guest                                                                                                         
Jan 26 18:46:15 vyos vyos-configd[24976]: Hardware vendor:  QEMU                                                                                                              
Jan 26 18:46:15 vyos vyos-configd[24976]: Hardware model:   Standard PC (i440FX + PIIX, 1996)                                                                                 
Jan 26 18:46:15 vyos vyos-configd[24976]: Hardware S/N:                                                                          
Jan 26 18:46:15 vyos vyos-configd[24976]: Hardware UUID:    d88a0d09-7ea7-423a-bb0d-ce3772628693                                 
Jan 26 18:46:15 vyos vyos-configd[24976]: Traceback (most recent call last):                                                     
Jan 26 18:46:15 vyos vyos-configd[24976]:   File "/usr/libexec/vyos/services/vyos-configd", line 339, in <module>                
Jan 26 18:46:15 vyos vyos-configd[24976]:     frr.apply()                                                                        
Jan 26 18:46:15 vyos vyos-configd[24976]:   File "/usr/lib/python3/dist-packages/vyos/frrender.py", line 744, in apply           
Jan 26 18:46:15 vyos vyos-configd[24976]:     raise ConfigError(emsg)                                                            
Jan 26 18:46:15 vyos vyos-configd[24976]: vyos.base.ConfigError: 2025-01-26 18:46:12,993  INFO: Called via "Namespace(input=None,
Jan 26 18:46:15 vyos vyos-configd[24976]: reload=True, test=False, debug=True, log_level='info', stdout=True,                    
Jan 26 18:46:15 vyos vyos-configd[24976]: pathspace=None, filename='/run/frr/config/vyos.frr.conf',                              
Jan 26 18:46:15 vyos vyos-configd[24976]: overwrite=False, bindir='/usr/bin', confdir='/etc/frr',                                
Jan 26 18:46:15 vyos vyos-configd[24976]: rundir='/var/run/frr', vty_socket=None, daemon='', test_reset=False)"                  
Jan 26 18:46:15 vyos vyos-configd[24976]: 2025-01-26 18:46:12,993  INFO: Loading Config object from file                         
Jan 26 18:46:15 vyos vyos-configd[24976]: /run/frr/config/vyos.frr.conf 2025-01-26 18:46:12,999 #033[91m  ERROR#033[0m:          
Jan 26 18:46:15 vyos vyos-configd[24976]: vtysh failed to process new configuration: vtysh (mark file) exited with               
Jan 26 18:46:15 vyos Report time:      2025-01-26 18:46:15                                                                       
Jan 26 18:46:15 vyos vyos-configd[24976]: status 2: b'line 8: % Unknown command: ip  route 0.0.0.0/0 pppoe0 tag                  
Jan 26 18:46:15 vyos vyos-configd[24976]: 210 1!\n\n'                                                                            
Jan 26 18:46:15 vyos Image version:    VyOS 1.5-rolling-202501250006

seems the trailing '!' causes this issue

sskaje added a comment.EditedJan 26 2025, 7:07 PM

I tried two ways in staticd.frr.j2

{# IPv4 default routes from PPPoE interfaces #}
{% if pppoe is vyos_defined %}
{%     for interface, interface_config in pppoe.items() if interface_config.no_default_route is not vyos_defined %}
{{ ip_prefix }} route 0.0.0.0/0 {{ interface }} tag 210 {{ interface_config.default_route_distance if interface_config.default_route_distance is vyos_defined }}
{%-    endfor %}
{% endif %}
{# IPv6 routing #}
{% if route6 is vyos_defined %}
{%     for prefix, prefix_config in route6.items() %}
{{ static_routes(ipv6_prefix, prefix, prefix_config) }}
{# j2lint: disable=jinja-statements-delimeter #}
{%-    endfor %}
{% endif %}
{% if vrf is vyos_defined %}
exit-vrf
{% endif %}
!
{# Policy route tables #}

first try:

{%- endfor %} to {% endfor %}

second try:

! to \n!, like

{% endif %}

!
{# Policy route tables #}

works in minimum config and my working env.

but, I see this block is surrounded by vrf and exit-vrf, not sure if my changes is a good choice

sskaje added a comment.Jan 26 2025, 7:12 PM

As I tested, the endfor changes is the better way, because two pppoe interfaces would cause

ip  route 0.0.0.0/0 pppoe0 tag 210 11ip  route 0.0.0.0/0 pppoe1 tag 210 14!
sskaje updated the task description. (Show Details)Jan 27 2025, 12:03 PM
dmbaturin triaged this task as High priority.Jan 27 2025, 12:11 PM
dmbaturin added a project: VyOS 1.5 Circinus.
dmbaturin changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.
Viacheslav mentioned this in T7097: vtysh failed to process new configuration.Jan 27 2025, 8:59 PM
sskaje mentioned this in rVYOSONEX4dd4ffd24b42: T7089: Fix static route when using PPPoE default route.Jan 28 2025, 10:05 AM
sskaje mentioned this in rVYOSONEX1b831d438d0e: T7089: Fix Indentation.
Restricted Repository Identity mentioned this in rVYOSONEXd0760a58ed87: Merge pull request #4318 from sskaje/T7089.Jan 28 2025, 10:05 AM
Viacheslav changed the task status from Open to Needs testing.Jan 29 2025, 10:40 AM
Viacheslav assigned this task to sskaje.