Page MenuHomeVyOS Platform

Use Kea DHCP(v6) instead of ISC DHCP(v6)
Closed, ResolvedPublicFEATURE REQUEST

Description

Let's consider using ISC Kea instead of ISC DHCP

Perhaps we can get more complete support through migration

This is just a draft, if we can really improve the situation by replacing ISC, I don't see any reason why we shouldn't consider moving

There seems to be no officially packaged Kea on the existing Debian package system, which probably means that if we want to migrate to Kea, we'll need to maintain a package ourselves

https://packages.debian.org/sid/kea

This is a packaged code repository for isc-kea found upstream, but only 1.7.5-1, not the latest version, and may serve as a reference

https://salsa.debian.org/debian/isc-kea

Details

Difficulty level
Normal (likely a few hours)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
syncer raised the priority of this task from Wishlist to High.Oct 16 2022, 10:20 PM
syncer added a subscriber: syncer.

Basically,
all commercial hooks need to be implemented

njh added a subscriber: njh.

Add another feature that is improved if we're thinking of moving to KEA :

https://vyos.dev/T5044

sarthurdev changed the task status from Open to In progress.Mar 2 2023, 1:41 PM
sarthurdev claimed this task.
sarthurdev moved this task from Need Triage to In Progress on the VyOS 1.4 Sagitta board.

Have started work on migrating isc-dhcp v4/v6 server to Kea.

Migration might cause a fair amount of breakage. Notable issues so far:

  • global-parameters, shared-network-parameters and subnet-parameters nodes will be deleted on migration (likely to be the biggest issue)
  • ping-check no longer available

I was interested in why ping-check is no longer available.
There is a good article about it here:
https://kb.isc.org/docs/why-doesnt-kea-support-ping-check

@sdev just for clarification do you mean "deleted" as in only existing entries but new ones will work or completely deleted?
Im asking because I do use keas global, subnet, pool and class option-data support extensively outside of vyos.
If this would stay/become a part of vyos that would be great!

I'm using the *-parameters in isc-dhcp to manage ddns updates to an external nameserver. This way dns-entries supplied by dhcp-server will be kept in sync, even when using dhcp-failover.
I would love to still be able to keep this functionality with Kea in some way. Either by providing corresponding custom-parameters, or adding native support for this scenario.

Some brainstorming;

I haven't tried Kea out myself, merely skimmed the doc's.
However, I noticed the sync-leases & send-lease-updates-parameters in the high-availability-section being set to true by default (doc). Would that always trigger on-dhcp-event.sh on all instances? And thus, possibly keeping a ha-pair of vyos'es /etc/hosts-file in (better) sync?
Sure, if one dhcp-sever-instance goes down for a while, any leases handed by the remaining instance would be missing when the downed instance comes back online.
Perhaps it would be better if the on-dhcp-event.sh-script would keep the /etc/hosts-file up-to-date by reading the entire lease-file (csv?) on each execution. The lease-file would in turn be kept in sync on each instance by Kea?

A not too uncommon workaround for this is to disable the lease-file (if possible) and give out leases based on option82 information instead.

This way you dont have to sync anything between the VyOS-boxes except for the DHCP and Option82 configuration itself.

This will also bring semi-static address assignment in your network where whatever device is plugged into "SW2-INT4" (or however you choose to name vendorid and circuitid) always gets the same IP-address when asking for DHCP. This will also be compatible with static addressing within your network without having to change any ACL's to fullfill BCP38 (since you will always know what IP-address is expected to exist on "SW2-INT4"): http://www.bcp38.info/

Viacheslav changed the task status from In progress to Needs testing.Dec 10 2023, 12:23 AM

Hello, I'd like to provide my quick feedback after testing the version 1.5-rolling-202312130023

Here are the list of problem I found as of now.

  • dhcp server doesn't start automatically after reboot, and due to the next problem, I'm forced to use set service dhcp-server disable then delete service dhcp-server disable after each boot
  • op mode of restart dhcp server is not usable currently, using it will prompt so many errors
  • op mode of show dhcp server lease works, but, show dhcp server lease state all doesn't work <-- but this doesn't work for a long time now
  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all
  • configuration for dhcp reservation set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 static-mapping example mac-address 'xx:xx:xx:xx:xx:xx' with just this, you can't commit anymore. Originally, static-mapping only required mac address, and right now its validation only needs mac address, but in the code execution, it also assume that both mac-address and ip-address were set, so setting only mac address will cause exception

I appreciated the work being done here, and I just want to provide my experience. I don't know if I should open this so many tasks myself or not, so if I should be reporting that way, please let me know.

Thanks.

@Zen3515

  • dhcp server doesn't start automatically after reboot, and due to the next problem, I'm forced to use set service dhcp-server disable then delete service dhcp-server disable after each boot

Could not reproduce this:

Welcome to VyOS - vyos ttyS0
...
vyos@vyos:~$ ps aux | grep kea
_kea        1818  1.6  0.9  67384 20324 ?        Ssl  00:14   0:00 /usr/sbin/kea-dhcp4 -c /run/kea/kea-dhcp4.conf
  • op mode of restart dhcp server is not usable currently, using it will prompt so many errors

Also could not reproduce the issue:

vyos@vyos:~$ restart dhcp server
vyos@vyos:~$ sudo systemctl status kea-dhcp4-server
● kea-dhcp4-server.service - Kea IPv4 DHCP daemon
     Loaded: loaded (/lib/systemd/system/kea-dhcp4-server.service; disabled; pr>
    Drop-In: /etc/systemd/system/kea-dhcp4-server.service.d
             └─override.conf
     Active: active (running) since Sun 2023-12-17 00:15:48 UTC; 8s ago
  • op mode of show dhcp server lease works, but, show dhcp server lease state all doesn't work <-- but this doesn't work for a long time now

This was fixed in https://github.com/vyos/vyos-1x/pull/2623

  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all

Thanks, will investigate this.

  • configuration for dhcp reservation set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 static-mapping example mac-address 'xx:xx:xx:xx:xx:xx' with just this, you can't commit anymore. Originally, static-mapping only required mac address, and right now its validation only needs mac address, but in the code execution, it also assume that both mac-address and ip-address were set, so setting only mac address will cause exception

Confirmed this issue and will submit a fix shortly

Thanks

  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all

Thanks, will investigate this.

@sdev, this will require adjusting on-dhcp-event.sh. I have a hacky local version that writes to /etc/hosts that partially works -- the $domain part is not picked up (which I suspect could be related to how kea-dhcp4.conf is generating the FQDN).

Do you want me to raise a draft PR for you to review?

Update PR: https://github.com/vyos/vyos-1x/pull/2646

  • Move Kea socket permission change on-demand and speed up conf scripts
  • Fix issue with DHCP reservations when no ip-address value
  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all

Thanks, will investigate this.

@sdev, this will require adjusting on-dhcp-event.sh. I have a hacky local version that writes to /etc/hosts that partially works -- the $domain part is not picked up (which I suspect could be related to how kea-dhcp4.conf is generating the FQDN).

Do you want me to raise a draft PR for you to review?

By all means, thank you!

In T3316#167384, @sdev wrote:
  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all

Thanks, will investigate this.

@sdev, this will require adjusting on-dhcp-event.sh. I have a hacky local version that writes to /etc/hosts that partially works -- the $domain part is not picked up (which I suspect could be related to how kea-dhcp4.conf is generating the FQDN).

Do you want me to raise a draft PR for you to review?

By all means, thank you!

https://github.com/vyos/vyos-1x/pull/2647

This adds entries to /etc/hosts file like so:

# From 'system static-host-mapping' and DHCP server
# dhcp-server-10.144.40.8
10.144.40.8     host-3.host-3 
# dhcp-server-10.144.40.2
10.144.40.2     host-2.host-2

I would have expected the FQDN to be passed on via LEASE4_HOSTNAME but that isn't happening.

Indeed, the dhcp server wasn't working so I assume that kea doesn't start, my bad, it did start, but it failed to start, after failing for 5 attempts.

I don't know the exact cause, but I'll provide the interface and dhcp configuration below.
Maybe kea tartget in systemd start too soon ?

-- Boot b6cd28548c724a36a5d4a668c0105800 --
Dec 17 17:38:01 example-vyos-rt systemd[1]: Started kea-dhcp4-server.service - Kea IPv4 DHCP daemon.
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.765 INFO  [kea-dhcp4.dhcp4/4269.139786331605440] DHCP4_STARTING Kea DHCPv4 server version 2.2.0 (stable) starting
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.765 INFO  [kea-dhcp4.hosts/4269.139786331605440] HOSTS_BACKENDS_REGISTERED the following host backend types are available: mysql postgresql
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.765 INFO  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_CFGMGR_SOCKET_TYPE_SELECT using socket type raw
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.770 INFO  [kea-dhcp4.hooks/4269.139786331605440] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_run_script.so successfully closed
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.778 INFO  [kea-dhcp4.hooks/4269.139786331605440] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so successfully closed
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.778 INFO  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 192.168.1.0/24 with params: valid-lifetime=86400
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.779 INFO  [kea-dhcp4.commands/4269.139786331605440] COMMAND_ACCEPTOR_START Starting to accept connections via unix domain socket bound to /run/kea/dhcp4-ctrl-socket
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.779 INFO  [kea-dhcp4.run-script-hooks/4269.139786331605440] RUN_SCRIPT_LOAD Run Script hooks library has been loaded
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.779 INFO  [kea-dhcp4.hooks/4269.139786331605440] HOOKS_LIBRARY_LOADED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_run_script.so successfully loaded
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.780 INFO  [kea-dhcp4.lease-cmds-hooks/4269.139786331605440] LEASE_CMDS_INIT_OK loading Lease Commands hooks library successful
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.780 INFO  [kea-dhcp4.hooks/4269.139786331605440] HOOKS_LIBRARY_LOADED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so successfully loaded
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.780 INFO  [kea-dhcp4.dhcp4/4269.139786331605440] DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.780 INFO  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_MEMFILE_DB opening memory file lease database: name=/config/dhcp4.leases persist=true type=memfile universe=4
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.780 INFO  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /config/dhcp4.leases
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 INFO  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_MEMFILE_LFC_SETUP setting up the Lease File Cleanup interval to 3600 sec
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_MULTIPLE_RAW_SOCKETS_PER_IFACE current configuration will result in opening multiple broadcast capable sockets on some interfaces and some DHCP messages may be duplicated
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.784 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.794 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pppoe0 is down
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.794 WARN  [kea-dhcp4.dhcp4/4269.139786331605440] DHCP4_MULTI_THREADING_INFO enabled: no, number of threads: 0, queue size: 0
Dec 17 17:38:01 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:01.794 INFO  [kea-dhcp4.dhcp4/4269.139786331605440] DHCP4_STARTED Kea DHCPv4 server version 2.2.0 started
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:06 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:06.799 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Failed to open socket on interface pppoe0, reason: failed to bind fallback socket to address xxx.xxx.xxx.xxx, port 67, reason: Cannot assign requested address - is another DHCP server running?
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.805 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.805 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.805 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.805 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.805 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:11 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:11.806 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Failed to open socket on interface pppoe0, reason: failed to bind fallback socket to address xxx.xxx.xxx.xxx, port 67, reason: Cannot assign requested address - is another DHCP server running?
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:16 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:16.812 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Failed to open socket on interface pppoe0, reason: failed to bind fallback socket to address xxx.xxx.xxx.xxx, port 67, reason: Cannot assign requested address - is another DHCP server running?
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:21 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:21.819 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Failed to open socket on interface pppoe0, reason: failed to bind fallback socket to address xxx.xxx.xxx.xxx, port 67, reason: Cannot assign requested address - is another DHCP server running?
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.825 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth2 has no usable IPv4 addresses configured
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.825 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth0 has no usable IPv4 addresses configured
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.825 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface eth1 has no usable IPv4 addresses configured
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.825 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface pim6reg has no usable IPv4 addresses configured
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.825 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface br0 is not running
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.826 WARN  [kea-dhcp4.dhcpsrv/4269.139786331605440] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Failed to open socket on interface pppoe0, reason: failed to bind fallback socket to address xxx.xxx.xxx.xxx, port 67, reason: Cannot assign requested address - is another DHCP server running?
Dec 17 17:38:26 example-vyos-rt kea-dhcp4[4269]: 2023-12-17 17:38:26.826 INFO  [kea-dhcp4.dhcp4/4269.139786331605440] DHCP4_OPEN_SOCKETS_FAILED maximum number of open service sockets attempts: 5, has been exhausted without success
example-vyos-rt:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                                MAC                VRF        MTU  S/L    Description
-----------  ----------------------------------------  -----------------  -------  -----  -----  -----------------------------------
br0          192.168.1.1/24                            xx:xx:xx:xx:xx:xx  default   1500  u/u    Bridge Dual 10G
             fdef:abcd:efab:abab::1/64
eth0         -                                         xx:xx:xx:xx:xx:xx  br0       1500  u/u    LAN1 10GBase-T
eth1         -                                         xx:xx:xx:xx:xx:xx  br0       1500  u/u    LAN2 10GBase-T
eth2         -                                         xx:xx:xx:xx:xx:xx  default   1500  u/u    WAN 2.5GBase-T
lo           127.0.0.1/8                               00:00:00:00:00:00  default  65536  u/u
             ::1/128
pppoe0       xxx.xxx.xxx.xxx/32                        n/a                default   1492  u/u    FTTX
             xxxx:xxxx:xxxx:xx:xxxx:xxxx:xxxx:xxxx/64
wg0          xxx.xxx.xxx.xxx/24                        n/a                default   1420  u/u    Wireguard VPN with user in zone LAN
set service dhcp-server dynamic-dns-update
set service dhcp-server hostfile-update
set service dhcp-server shared-network-name LAN authoritative
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 default-router '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 domain-name 'home'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 name-server '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 start '192.168.1.120'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 stop '192.168.1.230'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 static-mapping ...
...

I also notice that ddclient has a similar problem.

ddclient[1710]: FAILED:   Unable to obtain information for 'pppoe0' -- missing ip or ifconfig command

This might be a problem that's specific to me. I don't know why every systemd start too early.

set service dhcp-server dynamic-dns-update
set service dhcp-server hostfile-update
set service dhcp-server shared-network-name LAN authoritative
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 default-router '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 domain-name 'home'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 name-server '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 start '192.168.1.120'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 stop '192.168.1.230'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 static-mapping ...
...

I also notice that ddclient has a similar problem.

ddclient[1710]: FAILED:   Unable to obtain information for 'pppoe0' -- missing ip or ifconfig command

This might be a problem that's specific to me. I don't know why every systemd start too early.

This was fixed recently for dynamic interfaces like pppoe. @Zen3515 can you please try with 1.5-rolling-202312171632?

/config/dhcp4.leases ownership seem to change when migrating to latest 1.5-rolling-202312171632

vyos@vyos:~$ journalctl -u kea-dhcp4-server.service -b
Dec 17 18:14:04 vyos systemd[1]: Started kea-dhcp4-server.service - Kea IPv4 DHCP daemon.
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.076 INFO  [kea-dhcp4.dhcp4/2734.140502928017856] DHCP4_STARTING Kea DHCPv4 server version 2.2.0 (stable) starting
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.076 INFO  [kea-dhcp4.hosts/2734.140502928017856] HOSTS_BACKENDS_REGISTERED the following host backend types are available: mysql postgresql
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.077 INFO  [kea-dhcp4.dhcpsrv/2734.140502928017856] DHCPSRV_CFGMGR_SOCKET_TYPE_SELECT using socket type raw
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.087 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_run_script.so successfully closed
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.111 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so successfully closed
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.111 INFO  [kea-dhcp4.dhcpsrv/2734.140502928017856] DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 192.168.1.0/24 with params: valid-lifetime=86400
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.111 INFO  [kea-dhcp4.commands/2734.140502928017856] COMMAND_ACCEPTOR_START Starting to accept connections via unix domain socket bound to /run/kea/dhcp4-ctrl-socket
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.117 INFO  [kea-dhcp4.run-script-hooks/2734.140502928017856] RUN_SCRIPT_LOAD Run Script hooks library has been loaded
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.117 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_LOADED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_run_script.so successfully loaded
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.lease-cmds-hooks/2734.140502928017856] LEASE_CMDS_INIT_OK loading Lease Commands hooks library successful
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_LOADED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so successfully loaded
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.dhcp4/2734.140502928017856] DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.dhcpsrv/2734.140502928017856] DHCPSRV_MEMFILE_DB opening memory file lease database: name=/config/dhcp4.leases persist=true type=memfile universe=4
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.dhcpsrv/2734.140502928017856] DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /config/dhcp4.leases
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 ERROR [kea-dhcp4.dhcp4/2734.140502928017856] DHCP4_CONFIG_LOAD_FAIL configuration error using file: /run/kea/kea-dhcp4.conf, reason: Unable to open database: unable to open '/config/dhcp4.leases'
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 ERROR [kea-dhcp4.dhcp4/2734.140502928017856] DHCP4_INIT_FAIL failed to initialize Kea server: configuration error using file '/run/kea/kea-dhcp4.conf': Unable to open database: unable to open '/config/dhcp4.leases'
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.lease-cmds-hooks/2734.140502928017856] LEASE_CMDS_DEINIT_OK unloading Lease Commands hooks library successful
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.run-script-hooks/2734.140502928017856] RUN_SCRIPT_UNLOAD Run Script hooks library has been unloaded
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so successfully closed
Dec 17 18:14:05 vyos kea-dhcp4[2734]: 2023-12-17 18:14:05.118 INFO  [kea-dhcp4.hooks/2734.140502928017856] HOOKS_LIBRARY_CLOSED hooks library /usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_run_script.so successfully closed
Dec 17 18:14:05 vyos systemd[1]: kea-dhcp4-server.service: Main process exited, code=exited, status=1/FAILURE
Dec 17 18:14:05 vyos systemd[1]: kea-dhcp4-server.service: Failed with result 'exit-code'.

the build i'm migrating from is 1.5-rolling-202312101419:

old 1.5-rolling-202312101419
vyos@vyos:/config$ ls -l | grep dhcp4
-rwxr-xr-x 1 _kea vyattacfg 38301 Dec 17 18:26 dhcp4.leases

looks like in 1.5-rolling-202312171632 the owner changed to root

new 1.5-rolling-202312171632
vyos@vyos:/config$ ls -l | grep dhcp4
-rwxr-xr-x 1 root vyattacfg 38223 Dec 17 17:58 dhcp4.leases

migrating from 1.5-rolling-202312171632 to 1.5-rolling-202312181606
this issue happened again.

So, I tested the version 1.5-rolling-202312191154

Let's update on the last comment

  • dhcp server doesn't start automatically after reboot dhcp server start on reboot but failed, in journalctl, it says that it stop retrying after 5 tries, and due to the next problem, I'm forced to use set service dhcp-server disable then delete service dhcp-server disable after each boot
  • op mode of restart dhcp server is not usable currently, using it will prompt so many errors
  • op mode of show dhcp server lease works, but, show dhcp server lease state all doesn't work <-- but this doesn't work for a long time now
  • with set service dhcp-server hostfile-update the file /etc/hosts doesn't get update with any entry from dhcp at all
  • configuration for dhcp reservation set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 static-mapping example mac-address 'xx:xx:xx:xx:xx:xx' with just this, you can't commit anymore. Originally, static-mapping only required mac address, and right now its validation only needs mac address, but in the code execution, it also assume that both mac-address and ip-address were set, so setting only mac address will cause exception

And here's a new one

  • Just like @workingagain report, the ownership of /config/dhcp4.leases was changed upon upgrading vyos version. Current workaround is to change it back.
  • configuration for dhcp reservation with just mac address doesn't give configuration exception, but cause an error to kea directly as ERROR [kea-dhcp4.dhcp4/49022.140176366361024] DHCP4_PARSER_FAIL failed to create or run parser for configuration element shared-networks: specified reservation for DUID: hwtype=1 xx:xx:xx:xx:xx:xx must include at least one resource, i.e. hostname, IPv4 address, IPv6 address/prefix, options (/run/kea/kea-dhcp4.conf:59:5)

As for the testing for @indrajitr, I've not found the conclusive answer yet, It's mostly on my part, I'll update with you if it does work.

And here are my thoughts

  • dhcp server still failed to start after 5 , could we just increase more tries? or make this systemd run after the bridges to be configured? I think this issue might need its own taks, I'll open it soon.
  • dhcp reservation should be done using the configured host name, not just the mac address to prevent the exception stated above.
  • /etc/hosts gets update, but with the wrong name, it should be using the configured hostname let see the example below.

with config

set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 domain-name 'home'
set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 static-mapping static-mapping-name ip-address '192.168.XXX.XXX'
set service dhcp-server shared-network-name LAN subnet 192.168.XXX.0/24 static-mapping static-mapping-name mac-address 'xx:xx:xx:xx:xx:xx'

and the lease being this

vyos@example-vyos-rt:~$ show dhcp server leases state all 
IP Address       MAC address        State    Lease start          Lease expiration     Remaining                 Pool    Hostname                       Origin
---------------  -----------------  -------  -------------------  -------------------  ------------------------  ------  -----------------------------  --------
192.168.XXX.XXX  xx:xx:xx:xx:xx:xx  active   2023/12/19 06:54:03  2023/12/20 06:54:03  4:17:16                   LAN     example-lease-name             local

one should expect this in the /etc/hosts

# From 'system static-host-mapping' and DHCP server
# dhcp-server-192.168.XXX.XXX
192.168.XXX.XXX   static-mapping-name.home

but instead, it use the hosts from the lease

# From 'system static-host-mapping' and DHCP server
# dhcp-server-192.168.XXX.XXX
192.168.XXX.XXX   example-lease-name

configuration for dhcp reservation with just mac address doesn't give configuration exception, but cause an error to kea directly as ERROR [kea-dhcp4.dhcp4/49022.140176366361024] DHCP4_PARSER_FAIL failed to create or run parser for configuration element shared-networks: specified reservation for DUID: hwtype=1 xx:xx:xx:xx:xx:xx must include at least one resource, i.e. hostname, IPv4 address, IPv6 address/prefix, options (/run/kea/kea-dhcp4.conf:59:5)

If the IP address is not specified, an invalid reservation array will be created like:

"reservations": [{
    "hw-address": "xx:xx:xx:xx:xx:xx"
}]

This never happened before Kea migration because hostname was always added in. I assume https://github.com/vyos/vyos-1x/pull/2650 will indirectly solve for this.

and the lease being this

vyos@example-vyos-rt:~$ show dhcp server leases state all 
IP Address       MAC address        State    Lease start          Lease expiration     Remaining                 Pool    Hostname                       Origin
---------------  -----------------  -------  -------------------  -------------------  ------------------------  ------  -----------------------------  --------
192.168.XXX.XXX  xx:xx:xx:xx:xx:xx  active   2023/12/19 06:54:03  2023/12/20 06:54:03  4:17:16                   LAN     example-lease-name             local

one should expect this in the /etc/hosts

# From 'system static-host-mapping' and DHCP server
# dhcp-server-192.168.XXX.XXX
192.168.XXX.XXX   static-mapping-name.home

but instead, it use the hosts from the lease

# From 'system static-host-mapping' and DHCP server
# dhcp-server-192.168.XXX.XXX
192.168.XXX.XXX   example-lease-name

Looks like Kea run hook doesn't pass domain-name to our custom script anymore. The custom script was modified recently to align with Kea (see: https://github.com/vyos/vyos-1x/pull/2647) after which /etc/hosts is populated. But domain-name isn't available. This is an open issue.

Cannot pass smoketest

vyos@r4:~$ /usr/libexec/vyos/tests/smoke/cli/test_service_dhcp-server.py
test_dhcp_exclude_in_range (__main__.TestServiceDHCPServer.test_dhcp_exclude_in_range) ... ok
test_dhcp_exclude_not_in_range (__main__.TestServiceDHCPServer.test_dhcp_exclude_not_in_range) ... ok
test_dhcp_failover (__main__.TestServiceDHCPServer.test_dhcp_failover) ... FAIL
test_dhcp_multiple_pools (__main__.TestServiceDHCPServer.test_dhcp_multiple_pools) ... ok
test_dhcp_relay_server (__main__.TestServiceDHCPServer.test_dhcp_relay_server) ... ok
test_dhcp_single_pool_options (__main__.TestServiceDHCPServer.test_dhcp_single_pool_options) ... ok
test_dhcp_single_pool_range (__main__.TestServiceDHCPServer.test_dhcp_single_pool_range) ... ok
test_dhcp_single_pool_static_mapping (__main__.TestServiceDHCPServer.test_dhcp_single_pool_static_mapping) ... ok

======================================================================
FAIL: test_dhcp_failover (__main__.TestServiceDHCPServer.test_dhcp_failover)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/libexec/vyos/tests/smoke/cli/test_service_dhcp-server.py", line 623, in test_dhcp_failover
    self.assertTrue(process_named_running(CTRL_PROCESS_NAME))
AssertionError: None is not true

----------------------------------------------------------------------
Ran 8 tests in 34.208s

FAILED (failures=1)
vyos@r4:~$
set service dhcp-server dynamic-dns-update
set service dhcp-server hostfile-update
set service dhcp-server shared-network-name LAN authoritative
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 default-router '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 domain-name 'home'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 name-server '192.168.1.1'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 start '192.168.1.120'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 range 0 stop '192.168.1.230'
set service dhcp-server shared-network-name LAN subnet 192.168.1.0/24 static-mapping ...
...

I also notice that ddclient has a similar problem.

ddclient[1710]: FAILED:   Unable to obtain information for 'pppoe0' -- missing ip or ifconfig command

This might be a problem that's specific to me. I don't know why every systemd start too early.

This was fixed recently for dynamic interfaces like pppoe. @Zen3515 can you please try with 1.5-rolling-202312171632?

Can confirm that it's fix now, albeit that the first time it run it doesn't get the ip, need to wait 5 minutes later after boot.

after manual ownership change dhcp4.leases so the server can start causes this error:

ERROR [kea-dhcp4.dhcpsrv/4006.140169685592512] DHCPSRV_MEMFILE_LFC_LEASE_FILE_RENAME_FAIL failed to rename the current lease file /config/dhcp4.leases to /config/dhcp4.leases.1, reason: Permission denied

after manual ownership change dhcp4.leases so the server can start causes this error:

ERROR [kea-dhcp4.dhcpsrv/4006.140169685592512] DHCPSRV_MEMFILE_LFC_LEASE_FILE_RENAME_FAIL failed to rename the current lease file /config/dhcp4.leases to /config/dhcp4.leases.1, reason: Permission denied

This happens because of kea-lfc. Kea spawns this every hour which is responsible for cleaning up the lease database and since kea-lfc doesn't have write access to /config, the process fails and the error is logged. I have a local fix of this where I have moved the lease file to /config/kea.

PR: https://vyos.dev/T3316, this is WIP and will probably need a migration.

The issue with the missing domain name in /etc/hosts with hostfile-update, as mentioned above, seems to trigger another problem. The hostname requested by the client seems to be added to /etc/hosts verbatim and some clients (eg. some Windows machines and printers) request a fully qualified name with a trailing dot. Since pdns-recursor unconditionally appends a dot, there are now two trailing dots, causing pdns-recursor to crash if it restarts.

# cat /etc/hosts
<...>
# dhcp-server-1.2.3.4
1.2.3.4      foobar.
Jan 07 19:25:12 vyos pdns-recursor[65948]: msg="Unable to load Lua script" error="recursor.vyos-hostsd.conf.lua:19: Caught exception: Found . in wrong position in DNSName: foobar\\.\\.
Jan 07 19:25:12 vyos pdns-recursor[65948]: stack traceback:
Jan 07 19:25:12 vyos pdns-recursor[65948]:         [C]: in function 'addNTA'
Jan 07 19:25:12 vyos pdns-recursor[65948]:         recursor.vyos-hostsd.conf.lua:19: in main chunk
Jan 07 19:25:12 vyos pdns-recursor[65948]:         [C]: in function 'dofile'
Jan 07 19:25:12 vyos pdns-recursor[65948]:         [string \"chunk\"]:8: in main chunk" subsystem="luaconfig" level="0" prio="Error" tid="0" ts="1704673512.482" file="recursor.conf.lua"
dmbaturin set Issue type to Unspecified (please specify).