Page MenuHomeVyOS Platform

kea: DHCPv4 ip allocation issue on vlan interfaces
Open, HighPublicBUG

Description

vyos 1.5.0 dhcp-server config:

set interfaces bonding bond0 address '172.16.1.1/24'
set interfaces bonding bond0 member interface 'eth1'
set interfaces bonding bond0 member interface 'eth2'
set interfaces bonding bond0 mode '802.3ad'
set interfaces bonding bond0 vif 3 address '172.16.3.1/24'
set interfaces bonding bond0 vif 4 address '172.16.4.1/24'
set interfaces bonding bond0 vif 7 address '172.16.7.1/24'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 option default-router '172.16.4.1'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 option name-server '1.1.1.1'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 range SERVERS start '172.16.4.20'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 range SERVERS stop '172.16.4.100'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 static-mapping Generator ip-address '172.16.4.28'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 static-mapping Generator mac 'dc:a6:32:11:95:d1'
set service dhcp-server shared-network-name ServerNet subnet 172.16.4.0/24 subnet-id '1'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 option default-router '172.16.3.1'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 option name-server '172.16.3.1'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 option name-server '1.1.1.1'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 range WIFI start '172.16.3.10'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 range WIFI stop '172.16.3.254'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 static-mapping printer ip-address '172.16.3.241'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 static-mapping printer mac 'C0:74:AD:F7:99:D8'
set service dhcp-server shared-network-name WIFI subnet 172.16.3.0/24 subnet-id '2'
set service dhcp-server shared-network-name Wired subnet 172.16.7.0/24 option default-router '172.16.7.1'
set service dhcp-server shared-network-name Wired subnet 172.16.7.0/24 option name-server '1.1.1.1'
set service dhcp-server shared-network-name Wired subnet 172.16.7.0/24 range Home start '172.16.7.10'
set service dhcp-server shared-network-name Wired subnet 172.16.7.0/24 range Home stop '172.16.7.200'
set service dhcp-server shared-network-name Wired subnet 172.16.7.0/24 subnet-id '3'
set service dhcp-server shared-network-name mgt subnet 172.16.1.0/24 option default-router '172.16.1.1'
set service dhcp-server shared-network-name mgt subnet 172.16.1.0/24 option name-server '1.1.1.1'
set service dhcp-server shared-network-name mgt subnet 172.16.1.0/24 range mgt start '172.16.1.50'
set service dhcp-server shared-network-name mgt subnet 172.16.1.0/24 range mgt stop '172.16.1.200'
set service dhcp-server shared-network-name mgt subnet 172.16.1.0/24 subnet-id '4'

switch:

set interfaces bonding bond0 mode '802.3ad'                                                                                              
set interfaces bonding bond0 member interface 'eth1'                                                                                   
set interfaces bonding bond0 member interface 'eth2'                                                                                                      
set interfaces bridge br0 member interface 'bond0'
set interfaces bridge br0 member interface 'eth0'                                                                                                                                                                                                                                                          
set interfaces bridge br0 enable-vlan                                                                                                    
set interfaces bridge br0 member interface bond0 allowed-vlan '1-7'
set interfaces bridge br0 member interface bond0 native-vlan '1'                                                                         
set interfaces bridge br0 member interface eth0 allowed-vlan '3'                                                                         
set interfaces bridge br0 member interface eth0 native-vlan '3'

client:

set interfaces ethernet eth0 address 'dhcp'

KEA sends two offers:

06:15:53.188005 IP 172.16.3.1.bootps > 172.16.3.10.bootpc   ← correct (VLAN 3)                                                           
06:15:53.189202 IP 172.16.1.1.bootps > 172.16.1.52.bootpc   ← wrong (mgt subnet)

One from the correct VLAN subnet and one from 172.16.1.0/24(bond0). Both are sent within 1ms

dhcp-server listening sockets:

vyos@vyos:~$ ss -f link -p
Netid  Recv-Q  Send-Q     Local Address:Port        Peer Address:Port  Process  
p_dgr  0       0                 [8193]:*                       *               
p_raw  0       0                      *:bond0                   *               
p_raw  0       0                      *:bond0.3                 *               
p_raw  0       0                      *:bond0.4                 *               
p_raw  0       0                      *:bond0.7                 *
  1. Kea with “interfaces”: [“*”] opens raw sockets on bond0 and bond0.X
  2. DHCP DISCOVER arrives on both (kernel delivers to parent first, then VLAN)
  3. Multi-threading processes both copies independently
  4. Two offers sent: one from 172.16.1.0/24 (bond0), one from correct VLAN subnet

Details

Version
1.5.0
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

I worked around this issue using a macvlan as per https://lists.isc.org/pipermail/kea-users/2020-February/002630.html

eth1 is my untagged LAN which also has a number of tagged VLANs (eth1.1, eth1.2 etc)

set interfaces pseudo-ethernet peth1 address 192.168.xxx.250/32 # assign a /32 in the same subnet as the eth1 LAN
set interfaces pseudo-ethernet peth1 source-interface eth1

delete service dhcp-server listen-interface eth1 # tell kea not to listen raw on eth1
set service dhcp-server listen-interface peth1 # listen on the macvlan interface instead
set service dhcp-server listen-interface eth1.2
set service dhcp-server listen-interface eth1.3

Hi Fransking, For some reason, Kea is just not playing nice with VIFs. Related: T8547. It’s quite striking that such a fundamental service as a DHCP server is not working correctly in this branch. The integration between the Kea engine and VyOS’s interface abstraction layer seems to have serious gaps, especially for unicast traffic.

Did anything land in 1.5.1 to help with this one?

Reproduced this on rolling with Kea 3.0.3-vyos.

I wrote the Kea config by hand so the templates were not part of the test. veth pair, 172.16.1.1/24 on the parent, 172.16.10.1/24 on VLAN 10. Kea listening with raw sockets on both, and each subnet had its own interface set. Client was udhcpc in a netns, discover sent on the VLAN.

Kea offered both 172.16.1.50 and 172.16.10.50 for that one discover. udhcpc took the VLAN address, but the untagged pool still handed one out. Setting interface on the subnet did not stop it. The tagged discover still shows up on the parent socket, and Kea answers that copy from the untagged subnet.

I then stopped listening on the parent and listened on a macvlan instead. Same subnets, same client. The VLAN discover only got 172.16.10.50. Kea logged the offer and the ack on the VLAN interface and nothing else. The extra offer is the parent socket.

That is the bug ISC fixed in 3.1.7 and shipped in 3.2.0 (GitLab #1117, #1738, #3792). In 3.2.0, pkt_filter_lpf.cc drops the frame on the parent when the VLAN tag is still present. I pulled the same file from the 3.0.4 tag and from the current v3_0 branch. Neither copy has that check. The 3.0.4 release notes do not mention those tickets. The changelog tags the change as a feature, and it is not on the 3.0 branch. libkea-dhcp.so.109 on the lab has no VLAN filter strings in it either.

The rolling isc-kea build in vyos-build does not have a patch for this. Our template listens on "*" with raw sockets, which is enough to hit the bug, but there is no config change that keeps DHCP working on the parent and on the VLANs at the same time. Leave the parent out of the listen list and the bad offer stops, and so does untagged DHCP. The macvlan workaround from the forum does work. I tried it. It needs a /32, arp ignore, and loose rp_filter, and I do not think we should generate that.

The fix is to ship Kea 3.2. I checked whether 3.2 would refuse the config we write today. It still accepts the old control-socket object and wraps it into the new list itself, so the templates do not have to change for the bump. I have not built 3.2 and rerun the discover against it.