Integrate hsflowd for sflow accounting
https://github.com/sflow/host-sflow
https://blog.sflow.com/2023/03/vyos.html
Needs to check if can migrate all pmacct configuration for sflow
vyos@r14# set system flow-accounting sflow Possible completions: agent-address sFlow agent IPv4 address sampling-rate sFlow sampling-rate +> server sFlow destination server source-address Source IP address used to initiate connection
Or use it with the new CLI syntax as an alternative to flow-accounting
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | FEATURE REQUEST | Viacheslav | T5086 Integrate hsflowd for sflow accounting | ||
| Resolved | FEATURE REQUEST | Viacheslav | T5125 Add op-mode commands for hsflowd based sflow | ||
| Resolved | FEATURE REQUEST | c-po | T5968 hsflowd: add VRF support |
Jenkins job looks simple
git clone https://github.com/sflow/host-sflow make deb FEATURES="NFLOG PCAP TCP DBUS SYSTEMD"
I think NFLOG and TCP can be dropped for sure.
pcap is the best option to sample traffic and we use following configuration for many deployments:
sflow {
DNSSD=off
polling=30
agentIP=127.0.0.1
# Increase to 1000 if you have more then 1G of traffic
sampling=100
collector { ip = 127.0.0.1 udpport = 6343 }
pcap { dev=ens3 }
pcap { dev=ens4 }
}In pcap mode when sampling is set to value larger then 1 hsflowd uses kernel based sampling available on kernels starting from 3.16
According to our real field experience it's extremely efficient and introduces exceptionally small overhead on machine.
I have sample opcode described in my blog post in more details https://pavel.network/linux-and-bpf-random-opcode/
Finally, I think the only mode needed in VyOS case is a pcap.
PR https://github.com/vyos/vyos-build/pull/320
PR https://github.com/vyos/vyos-1x/pull/1891
set system sflow agent-address '192.168.122.14' set system sflow interface 'eth0' set system sflow interface 'eth1' set system sflow polling '30' set system sflow sampling-rate '100' set system sflow server 192.168.122.1 port '6343' set system sflow server 192.168.122.11 port '6343' set system sflow source-address '192.168.122.14'
check service
vyos@r14# sudo systemctl status hsflowd
● hsflowd.service - Host sFlow
Loaded: loaded (/lib/systemd/system/hsflowd.service; disabled; preset: enabled)
Drop-In: /run/systemd/system/hsflowd.service.d
└─override.conf
Active: active (running) since Tue 2023-03-14 20:58:28 EET; 3s ago
Main PID: 13144 (hsflowd)
Tasks: 2 (limit: 9400)
Memory: 716.0K
CPU: 10ms
CGroup: /system.slice/hsflowd.service
└─13144 /usr/sbin/hsflowd -m 4d6f4d291ae8446f8d2b3decd9da64c7 -d -f /run/sflow/hsflowd.conf
Mar 14 20:58:28 r14 systemd[1]: Started hsflowd.service - Host sFlow.
[edit]
vyos@r14#config
vyos@r14# cat /run/sflow/hsflowd.conf
# Genereated by /usr/libexec/vyos/conf_mode/system_sflow.py
# Parameters http://sflow.net/host-sflow-linux-config.php
sflow {
DNSSD=off
polling=30
# Increase to 1000 if you have more then 1G of traffic
sampling=100
agentIP=192.168.122.14
collector { ip = 192.168.122.1 udpport = 6343 }
collector { ip = 192.168.122.11 udpport = 6343 }
pcap { dev=eth0 }
pcap { dev=eth1 }
}As DROP_MONITOR merged we can extend configuration to
set system sflow drop-monitor-limit 50
hsflowd.conf
dropmon { limit=50 start=on sw=on hw=off }hsflowd will export the headers of dropped packets (along with the name of the function in the linux kernel where that skb was dropped) as part of the standard sFlow feed. This measurement complements the sFlow packet-sampling and counter-telemetry well because it provides visibility into the traffic that is not flowing. Very helpful for troubleshooting. The limit (a rate limit max of N drops per second that will be sent out in the sFlow datagrams) is the parameter that you would set in the CLI.