Page MenuHomeVyOS Platform

ARP/NDP table-size isnt set properly
Closed, ResolvedPublicBUG

Description

Using the following config:

set system ip arp table-size '32768'
set system ipv6 neighbor table-size '32768'

The above is set in the system by (if I have traced this correctly):

IPv4:

/usr/libexec/vyos/conf_mode/system-ip.py

# Apply ARP threshold values                                                  
# table_size has a default value - thus the key always exists                 
size = int(dict_search('arp.table_size', opt))                                
# Amount upon reaching which the records begin to be cleared immediately      
sysctl_write('net.ipv4.neigh.default.gc_thresh3', size)                      
# Amount after which the records begin to be cleaned after 5 seconds         
sysctl_write('net.ipv4.neigh.default.gc_thresh2', size // 2)                 
# Minimum number of stored records is indicated which is not cleared         
sysctl_write('net.ipv4.neigh.default.gc_thresh1', size // 8)

IPv6:

/usr/libexec/vyos/conf_mode/system-ipv6.py

# Apply ND threshold values                                                  
# table_size has a default value - thus the key always exists                
size = int(dict_search('neighbor.table_size', opt))                     
# Amount upon reaching which the records begin to be cleared immediately
sysctl_write('net.ipv6.neigh.default.gc_thresh3', size)                 
# Amount after which the records begin to be cleaned after 5 seconds    
sysctl_write('net.ipv6.neigh.default.gc_thresh2', size // 2)            
# Minimum number of stored records is indicated which is not cleared    
sysctl_write('net.ipv6.neigh.default.gc_thresh1', size // 8)

However verifying if this is actually set returns this:

vyos@vyos:~$ sudo sysctl -a | grep gc_thresh3
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv6.neigh.default.gc_thresh3 = 8192

Meaning whatever I set through set system ip arp table-size and set system ipv6 neighbor table-size didnt have any effect (unless Im looking at the wrong place)?

Ref:

https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt

neigh/default/gc_thresh3 - INTEGER
	Maximum number of non-PERMANENT neighbor entries allowed.  Increase
	this when using large numbers of interfaces and when communicating
	with large numbers of directly-connected peers.
	Default: 1024

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.5-rolling-202309120021
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

I can confirm that setting these values AFTER boot (and doing commit) they will be properly set.

But when saved and rebooting the custom values are not being used (default of 8192 is used).

So the issue is how these settings are set during boot (or well lack thereof).

Turns out that the values who override the vyos-config values are set in /etc/sysctl.d/30-vyos-router.conf:

# Increase default garbage collection thresholds                          
net.ipv4.neigh.default.gc_thresh1 = 1024                                 
net.ipv4.neigh.default.gc_thresh2 = 4096                                 
net.ipv4.neigh.default.gc_thresh3 = 8192                                 
#                                                                         
net.ipv6.neigh.default.gc_thresh1 = 1024                                      
net.ipv6.neigh.default.gc_thresh2 = 4096                                 
net.ipv6.neigh.default.gc_thresh3 = 8192

By commenting these out and reboot the values set by vyos-config are properly set also after a boot.

Using VyOS 1.5-rolling-202309170024.

Deleted the current settings (commit and save):

delete system ip arp
delete system ipv6 neighbor
vyos@vyos:~$ sudo sysctl -a | grep default.gc_thresh
net.ipv4.neigh.default.gc_thresh1 = 1024
net.ipv4.neigh.default.gc_thresh2 = 4096
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv6.neigh.default.gc_thresh1 = 1024
net.ipv6.neigh.default.gc_thresh2 = 4096
net.ipv6.neigh.default.gc_thresh3 = 8192

Rebooted (with no settings set):

vyos@vyos:~$ sudo sysctl -a | grep default.gc_thresh
net.ipv4.neigh.default.gc_thresh1 = 1024
net.ipv4.neigh.default.gc_thresh2 = 4096
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv6.neigh.default.gc_thresh1 = 1024
net.ipv6.neigh.default.gc_thresh2 = 4096
net.ipv6.neigh.default.gc_thresh3 = 8192

Set the settings (commit and save):

set system ip arp table-size '32768'
set system ipv6 neighbor table-size '32768'
vyos@vyos:~$ sudo sysctl -a | grep default.gc_thresh
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 16384
net.ipv4.neigh.default.gc_thresh3 = 32768
net.ipv6.neigh.default.gc_thresh1 = 4096
net.ipv6.neigh.default.gc_thresh2 = 16384
net.ipv6.neigh.default.gc_thresh3 = 32768

Rebooted (with settings set):

vyos@vyos:~$ sudo sysctl -a | grep default.gc_thresh
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 16384
net.ipv4.neigh.default.gc_thresh3 = 32768
net.ipv6.neigh.default.gc_thresh1 = 4096
net.ipv6.neigh.default.gc_thresh2 = 16384
net.ipv6.neigh.default.gc_thresh3 = 32768

Works as expected.

Case can be set to resolved!