Page MenuHomeVyOS Platform
Create Task
Maniphest T3319

VXLAN uses ttl 1 (auto) by default
Closed, ResolvedPublicBUG
Actions

Description

In 1.3/1.4 multicast vxlan behavior was changed. That means that does not possible to route multicast packet with TTL 1 and vxlan multicast can work only in L2 segment.
In 1.2.x by default used TTL = 16

[email protected]:~$ sudo ip -s -d link show dev vxlan241
9: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 3e:1f:d6:b3:5e:13 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 ttl 16 ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx 
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_is
    RX: bytes  packets  errors  dropped overrun mcast   
    2250       28       0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    2898       35       0       0       0       0

Output from 1.3

[email protected]# sudo ip -s -d link show dev vxlan241
13: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ae:9f:50:04:37:1f brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 ttl auto ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx 
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8002 port_no 0x2 designated_port 32770 designated_cost 0 designated_bridge 8000.50:0:0:2s
    RX: bytes  packets  errors  dropped overrun mcast   
    470        6        0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    1378       21       0       0       0       0

Details

Difficulty level
Easy (less than an hour)
Version
1.3-beta-202102160443
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Dmitry created this task.Feb 16 2021, 11:32 PM
Dmitry added a comment.Feb 16 2021, 11:43 PM

As a solution proposed to use TTL by default equivalent 16, but also add the possibility to change it via VyOS CLI

set interfaces vxlan vxlan241 ttl 16
c-po added a subscriber: c-po.Mar 2 2021, 11:10 PM

Please retest with the latest 1.4 rolling version. TTL can now be set.

pasik added a subscriber: pasik.Mar 3 2021, 4:10 PM
Dmitry added a comment.Mar 4 2021, 7:00 PM

@c-po does not work on 1.4-rolling-202103040218

[email protected]# sudo ip -d link show dev vxlan241
7: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fe:08:e3:3c:d4:ab brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 tos inherit ttl auto ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx

On the middle router in traffic dump I see TTL=1

18:59:29.029090 IP (tos 0x0, ttl 1, id 24806, offset 0, flags [none], proto UDP (17), length 100)
    10.1.2.2.52948 > 239.0.0.241.8472: OTV, flags [I] (0x08), overlay 0, instance 241
c-po added a comment.EditedMar 4 2021, 8:39 PM

VyOS 1.2 has this hardcoded: /opt/vyatta/share/vyatta-cfg/templates/interfaces/vxlan/node.def: VXLAN_TTL="ttl 16"

Can you manually change the ttl to 16 by set interfaces vxlan vxlan0 parameters ip ttl 16 if this works we have to add a migrator for older configs then.

c-po changed the task status from Open to In progress.Mar 4 2021, 8:40 PM
c-po claimed this task.
Dmitry added a comment.Mar 5 2021, 6:40 AM

@c-po , yes now it works. Maybe we need to define ttl=16 as the default value?

Cheeze_It added a subscriber: Cheeze_It.EditedMar 5 2021, 6:45 AM

@Dmitry

I know my opinion is....really not that important but I would *highly* recommend going to maximum TTL of 255 or at minimum 127. TTL is a very hard thing to troubleshoot most of the time and therefore it's almost never worth going lower than maximum for IP TTL.

c-po added a comment.Mar 5 2021, 8:35 PM

I wonder why it is 1 by default now - we actually specify 0 which means "take it from the inbound packet". Nevertheless we need a migrator to mimic existing behavior vor users upgrading from 1.2 -> 1.3, so the migrator should simply insert ttl = 16 into the CLI.

Cheeze_It added a comment.Mar 5 2021, 8:47 PM

@c-po, using a value of "0" for use inbound packet would actually be the best behavior if we can specify/use that.

c-po added a comment.Mar 6 2021, 9:10 AM

@Cheeze_It that's the new "normal".

@Dmitry in your output above I see ttl auto for 1.3 which is ttl = 0 - is this a real bug or is it just out of curiosity?

Dmitry added a comment.Mar 6 2021, 6:37 PM

For VXLAN multicast this is a real bug.

c-po closed this task as Resolved.Mar 7 2021, 7:59 PM
SrividyaA set Issue type to Bug (incorrect behavior).Aug 31 2021, 12:19 PM
syncer moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.Nov 6 2021, 11:43 AM
syncer edited projects, added VyOS 1.3 Equuleus (1.3.0); removed VyOS 1.3 Equuleus.Aug 29 2022, 12:16 PM
syncer moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus (1.3.0) board.