Which proper format do we need?
For example 2 tunnels:
peer-100.64.0.2-tunnel-0
peer-100.64.0.2-tunnel-1

PBR works if we change the static route to:

Maybe related task T3505 which deletes the default route for DHCP in the commit.

PR https://github.com/vyos/vyos-1x/pull/834

Note that adding autosudo=False to the call to cmd does not fix this issue because this parameter is never passed to popen: https://github.com/vyos/vyos-1x/blob/current/python/vyos/util.py#L143

Confirming that this bug is still present on 1.4-rolling-20210505111.

I have not built vyos since 2018, so my last production version was 201803051831.

@carl.byington is it working in 1.3?

19: l2tpeth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master foo state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 7e:0a:f4:5c:ef:8c brd ff:ff:ff:ff:ff:ff
    alias L2 VPN Tunnel

master foo <- foo is the VRF name

PR https://github.com/vyos/vyos-1x/pull/832

@Viacheslav it must be added in the proper order and contexts need to be entered - exited appropriately

It seems a bug with frr-reload

!
router bgp 64500 vrf foo
 no bgp ebgp-requires-policy
 no bgp network import-check
 exit
!
vrf foo
 ip protocol bgp route-map RMAP
 exit-vrf
!
route-map RMAP permit 10
 set tag 555
!
line vty
!

It is implemented in VyOS 1.4

Update I find an example

!
vrf foo
 ip protocol bgp route-map RMAP
 exit-vrf
!
router bgp 64500 vrf foo
 no bgp ebgp-requires-policy
 no bgp network import-check
!

@ernstjo
Is it supported by FRR?
Can you provide an example?

PR https://github.com/vyos/vyatta-cfg-quagga/pull/76

Problem is because of LSP MTU missmatch

Kernel pipeline merged

Maybe I found the issue:

@rob it fixed in the latest 1.3 with commit https://github.com/vyos/vyos-1x/commit/c7430fbb8738d76e63a6972b7399fa39572e2254
probably just not hit at that time in 1.3-rc4

this error also occure in VyOS 1.3 RC4.

Implemented, wait for approval

Don't think this is related to the i40e driver. I think it's because vyos sees

vyos@vyos# set interfaces ethernet eth3 description Test
[edit]
vyos@vyos# commit
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3'
DEBUG/IFCONFIG returned (out):
[{"ifindex":5,"ifname":"eth3","flags":["BROADCAST","MULTICAST","UP","LOWER_UP"],"mtu":1500,"qdisc":"mq","operstate":"UP","linkmode":"DEFAULT ","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:ff:ff:ff:ff","promiscuity":0,"min_mtu ":68,"max_mtu":9702,"inet6_addr_gen_mode":"none","num_tx_queues":16,"num_rx_queues":16,"gso_max_size":65536,"gso_max_segs":65535,"phys_port_ id":"3cfdfeddbd89","vfinfo_list":[]}]
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3'
DEBUG/IFCONFIG returned (out):
[{"ifindex":5,"ifname":"eth3","flags":["BROADCAST","MULTICAST","UP","LOWER_UP"],"mtu":1500,"qdisc":"mq","operstate":"UP","linkmode":"DEFAULT ","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:ff:ff:ff:ff","promiscuity":0,"min_mtu ":68,"max_mtu":9702,"inet6_addr_gen_mode":"none","num_tx_queues":16,"num_rx_queues":16,"gso_max_size":65536,"gso_max_segs":65535,"phys_port_ id":"3cfdfeddbd89","vfinfo_list":[]}]
DEBUG/IFCONFIG cmd 'ethtool --show-pause eth3'
DEBUG/IFCONFIG returned (out):
Pause parameters for eth3:
Autonegotiate: off
RX: off
TX: off
DEBUG/IFCONFIG cmd 'ethtool --pause eth3 autoneg on tx on rx on'
could not set flowcontrol for eth3
DEBUG/IFCONFIG cmd 'ethtool -K eth3 gro off'
DEBUG/IFCONFIG cmd 'ethtool -K eth3 gso off'
DEBUG/IFCONFIG write '0' > '/sys/class/net/eth3/queues/rx-0/rps_cpus'
DEBUG/IFCONFIG cmd 'ethtool -K eth3 sg off'
DEBUG/IFCONFIG cmd 'ethtool -K eth3 tso off'
DEBUG/IFCONFIG cmd 'ethtool -K eth3 ufo off'
DEBUG/IFCONFIG cmd 'ethtool eth3'
DEBUG/IFCONFIG returned (out):
Settings for eth3:

Supported ports: [ FIBRE ]
Supported link modes:   1000baseX/Full
                        10000baseSR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes:  1000baseX/Full
                        10000baseSR/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
                       drv probe link
Link detected: yes

DEBUG/IFCONFIG cmd 'ethtool -s eth3 autoneg on'
{'description': 'Test',
'duplex': 'auto',
'ifname': 'eth3',
'ip': {'arp_cache_timeout': '30'},
'mtu': '1500',
'speed': 'auto',
'vif': {'80': {'address': ['1.3.3.7/29'],

'ip': {'arp_cache_timeout': '30'},
'mtu': '1500'}}}

DEBUG/IFCONFIG cmd 'ip link set dev eth3 alias "Test"'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3/link_filter'
DEBUG/IFCONFIG cmd 'sudo systemctl stop dhclient@eth3.service'
DEBUG/IFCONFIG cmd 'sudo systemctl stop dhcp6c@eth3.service'
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3'
DEBUG/IFCONFIG returned (out):
[{"ifindex":5,"ifname":"eth3","flags":["NO-CARRIER","BROADCAST","MULTICAST","UP"],"mtu":1500,"qdisc":"mq","operstate":"DOWN","linkmode":"DEF AULT","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:ff:ff:ff:ff","promiscuity":0,"min _mtu":68,"max_mtu":9702,"inet6_addr_gen_mode":"none","num_tx_queues":16,"num_rx_queues":16,"gso_max_size":65536,"gso_max_segs":65535,"phys_p ort_id":"3cfdfeddbd89","ifalias":"Test","vfinfo_list":[]}]
DEBUG/IFCONFIG cmd 'ip link set dev eth3 nomaster'
DEBUG/IFCONFIG write '30000' > '/proc/sys/net/ipv4/neigh/eth3/base_reachable_time_ms'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3/arp_filter'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/arp_accept'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/arp_announce'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/arp_ignore'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/proxy_arp'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/proxy_arp_pvlan'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3/forwarding'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3/rp_filter'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3/forwarding'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3/accept_ra'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv6/conf/eth3/autoconf'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3/dad_transmits'
DEBUG/IFCONFIG cmd 'ip link set dev eth3 mtu 1500'
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3'
DEBUG/IFCONFIG returned (out):
[{"ifindex":5,"ifname":"eth3","flags":["NO-CARRIER","BROADCAST","MULTICAST","UP"],"mtu":1500,"qdisc":"mq","operstate":"DOWN","linkmode":"DEF AULT","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:ff:ff:ff:ff","promiscuity":0,"min _mtu":68,"max_mtu":9702,"inet6_addr_gen_mode":"none","num_tx_queues":16,"num_rx_queues":16,"gso_max_size":65536,"gso_max_segs":65535,"phys_p ort_id":"3cfdfeddbd89","ifalias":"Test","vfinfo_list":[]}]
DEBUG/IFCONFIG cmd 'xdp_loader -d eth3 -U --auto-mode'
DEBUG/IFCONFIG returned (out):
INFO: xdp_link_detach() no curr XDP prog on ifindex:5
DEBUG/IFCONFIG cmd 'tc qdisc del dev eth3 handle ffff: ingress 2> /dev/null;tc qdisc del dev eth3 handle 1: root prio 2> /dev/null;set $?=0'
DEBUG/IFCONFIG cmd 'ip link set dev eth3 up'
{'address': ['1.3.3.7/29'],
'ifname': 'eth3.80',
'ip': {'arp_cache_timeout': '30'},
'mtu': '1500'}
DEBUG/IFCONFIG cmd 'ip link set dev eth3.80 alias ""'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3.80/link_filter'
DEBUG/IFCONFIG cmd 'sudo systemctl stop dhclient@eth3.80.service'
DEBUG/IFCONFIG cmd 'sudo systemctl stop dhcp6c@eth3.80.service'
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3.80'
DEBUG/IFCONFIG returned (out):
[{"ifindex":8,"link":"eth3","ifname":"eth3.80","flags":["NO-CARRIER","BROADCAST","MULTICAST","UP"],"mtu":1500,"qdisc":"noqueue","operstate": "LOWERLAYERDOWN","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:f f:ff:ff:ff","promiscuity":0,"min_mtu":0,"max_mtu":65535,"linkinfo":{"info_kind":"vlan","info_data":{"protocol":"802.1Q","id":80,"flags":["RE ORDER_HDR"]}},"inet6_addr_gen_mode":"none","num_tx_queues":1,"num_rx_queues":1,"gso_max_size":65536,"gso_max_segs":65535}]
DEBUG/IFCONFIG cmd 'ip link set dev eth3.80 nomaster'
DEBUG/IFCONFIG write '30000' > '/proc/sys/net/ipv4/neigh/eth3.80/base_reachable_time_ms'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3.80/arp_filter'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/arp_accept'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/arp_announce'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/arp_ignore'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/proxy_arp'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/proxy_arp_pvlan'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv4/conf/eth3.80/forwarding'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv4/conf/eth3.80/rp_filter'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3.80/forwarding'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3.80/accept_ra'
DEBUG/IFCONFIG write '0' > '/proc/sys/net/ipv6/conf/eth3.80/autoconf'
DEBUG/IFCONFIG write '1' > '/proc/sys/net/ipv6/conf/eth3.80/dad_transmits'
DEBUG/IFCONFIG cmd 'ip link set dev eth3.80 mtu 1500'
DEBUG/IFCONFIG cmd 'ip -json -detail link list dev eth3.80'
DEBUG/IFCONFIG returned (out):
[{"ifindex":8,"link":"eth3","ifname":"eth3.80","flags":["NO-CARRIER","BROADCAST","MULTICAST","UP"],"mtu":1500,"qdisc":"noqueue","operstate": "LOWERLAYERDOWN","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"ether","address":"3c:fd:fe:dd:bd:89","broadcast":"ff:ff:f f:ff:ff:ff","promiscuity":0,"min_mtu":0,"max_mtu":65535,"linkinfo":{"info_kind":"vlan","info_data":{"protocol":"802.1Q","id":80,"flags":["RE ORDER_HDR"]}},"inet6_addr_gen_mode":"none","num_tx_queues":1,"num_rx_queues":1,"gso_max_size":65536,"gso_max_segs":65535}]
DEBUG/IFCONFIG cmd 'xdp_loader -d eth3.80 -U --auto-mode'
DEBUG/IFCONFIG returned (out):
INFO: xdp_link_detach() no curr XDP prog on ifindex:8
DEBUG/IFCONFIG cmd 'ip link set dev eth3.80 up'

One solution for it:

vyos_bld@3c88687662fe:/vyos/work/T3516/vyatta-cfg-quagga$ git diff
diff --git a/templates/protocols/static/interface-route/node.tag/next-hop-interface/node.def b/templates/protocols/static/interface-route/node.tag/next-hop-interface/node.def
index f8bc9270..991a30ee 100644
--- a/templates/protocols/static/interface-route/node.tag/next-hop-interface/node.def
+++ b/templates/protocols/static/interface-route/node.tag/next-hop-interface/node.def
@@ -14,7 +14,10 @@ end:
       if ! ${vyatta_sbindir}/vyatta-next-hop-check $VAR(../@) ipv4 interface; then
         exit 1;
       fi
+      DIST=`cli-shell-api returnEffectiveValue protocols static interface-route $VAR(../@) next-hop-interface $VAR(@) distance`
+
       vtysh -c "configure terminal" \
+                   -c "no ip route $VAR(../@) $VAR(@) $DIST" \
                    -c "no ip route $VAR(../@) $VAR(@)"
     else
       if [[ -n "$VAR(./distance/@)" ]]; then
diff --git a/templates/protocols/static/route/node.tag/next-hop/node.def b/templates/protocols/static/route/node.tag/next-hop/node.def
index 0574781d..44a39bfb 100644
--- a/templates/protocols/static/route/node.tag/next-hop/node.def
+++ b/templates/protocols/static/route/node.tag/next-hop/node.def
@@ -16,10 +16,12 @@ end:
       if ! ${vyatta_sbindir}/vyatta-next-hop-check $VAR(../@) ipv4 address; then
         exit 1;
       fi
+      DIST=`cli-shell-api returnEffectiveValue protocols static route $VAR(../@) next-hop $VAR(@) distance`
       if ${vyatta_sbindir}/vyatta-gateway-static_route-check.pl \
           "$VAR(../@)" "$VAR(@)"
       then
         vtysh -c "configure terminal" \
+                     -c "no ip route $VAR(../@) $VAR(@) $DIST" \
                      -c "no ip route $VAR(../@) $VAR(@)"
       fi
     else