Page MenuHomeVyOS Platform
Create Task
Maniphest T7761

New NetFlow implementation based on ipt-netflow
Closed, ResolvedPublicBUG
Actions

Description

pmacct, unfortunately, has performance problems that make it very difficult to use in modern networks and those problems aren't easy to solve.

A number of VyOS users have already been using self-built ipt-netflow instead of the official implementation as a workaround for those performance problems. We can as well adopt that, since it's kernel-mode and fast, and is already verified to work properly.

My conclusion so far is that we can keep the existing CLI with minor modifications.

The following options need to be moved:

  • set system flow-accounting interface — move to set system flow-accounting netflow interface, so that we can move set system sflow back under set system flow-accounting sflow later.

The following options will need to be removed because they aren't applicable in ipt-netflow:

  • set system flow-accounting disable-imt — in-memory table plugin for pmacct, has no equivalent in ipt-netflow.
  • set system flow-accounting packet-length — how many bytes to capture for each packets, seems to have no equivalent in ipt-netflow.
  • set system flow-accounting syslog-facility — there's no logging facility setting in ipt-netflow.
  • set system flow-accounting netflow timeout <icmp|max-active-life|tcp-fin|tcp-generic|tcp-rst|udp> — there are no per-protocol timeout settings in ipt-netflow.

The following options can be kept with their current semantics:

  • set system flow-accounting netflow version — obviously, protocol=. All of 5, 9, and 10 (IPFIX) are supported in ipt-netflow.
  • set system flow-accounting buffer-size — sndbuf=.
  • set system flow-accounting netflow engine-id — engine_id=.
  • set system flow-accounting netflow max-flows — maxflows=.
  • set system flow-accounting enable-egress — just add to the OUTPUT chain or not.
  • set system flow-accounting netflow sampling-rate — can be done with sampler settings.

The following options can be expanded:

Collector settings

The old implementation has distinct options: set system flow-accounting netflow server and set system flow-accounting netflow source-address.

ipt-netflow supports multiple destinations and per-destination source addresses and source interfaces.

We could put it all under a tag node:

server <addr>
  port <num>
  source
    address <addr>
    interface <intf>

The following ipt-netflow options need a discussion:

  • natevents — ipt-netflow doesn't send NAT translation events by default. Should we enable it by default? Should there be an option to disable it?
  • aggregation — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?

Details

Version
1.5
Is it a breaking change?
Config syntax change (migratable)
Issue type
Performance optimization

Related Objects
Search...

Event Timeline

dmbaturin created this task.Aug 28 2025, 11:27 AM
dmbaturin triaged this task as High priority.
dmbaturin updated the task description. (Show Details)Aug 28 2025, 2:52 PM
dmbaturin updated the task description. (Show Details)
hedrok subscribed.Aug 28 2025, 7:36 PM

I suggest just removing buffer-size option - in data/templates/pmacct/uacctd.conf.j2 it is translated to pmacct option plugin_pipe_size - specific to pmacct option about pipe size between processes of pmacct.
ipt_NETFLOW's sndbuf is size of output socket buffer in bytes with system default as default - no relation to plugin_pipe_size pmacct option, I don't think there is need to publish it via vyos CLI.

I suggest removing whole timeout subtree from netflow:

  1. A lot of timeout options have no analogues in ipt_NETFLOW
  2. Even options that sound very similar have different semantics - in pmacct timeouts are about evicting flows and in ipt_NETFLOW they are about sending exports.

To make timeouts configurable I suggest to add two options: active-timeout and inactive-timeout directly under netflow node.

Apart from that I support CLI interface suggestion.

Fabse subscribed.Sep 1 2025, 10:02 AM

aggregation — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?

This feature is crucial for our use case. We are using a custom build for ipt_NETFLOW since 2024 in order to --enable-aggregation for being able to account all flows (sampling rate 1) while keeping flow cardinality manageable. --enable-direction would be great as well.

Viacheslav changed the task status from Open to In progress.Sep 1 2025, 4:24 PM
Viacheslav assigned this task to hedrok.
Viacheslav subscribed.

PR https://github.com/vyos/vyos-1x/pull/4688

hedrok added a comment.Sep 2 2025, 7:57 AM
In T7761#234119, @Fabse wrote:

aggregation — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?

This feature is crucial for our use case. We are using a custom build for ipt_NETFLOW since 2024 in order to --enable-aggregation for being able to account all flows (sampling rate 1) while keeping flow cardinality manageable. --enable-direction would be great as well.

Ok, I've enabled direction and added aggregation with CLI interface:

set system flow-accounting netflow aggregation netmask 10.0.0.0/8 to 8
set system flow-accounting netflow aggregation port 80-89 to 80
set system flow-accounting netflow aggregation port 3128 to 80

This sets ipt_NETFLOW aggregation parameter to:

vyos@vyos-router# cat /sys/module/ipt_NETFLOW/parameters/aggregation
10.0.0.0/8=8,80-89=80,2048=137,3128=80
dmbaturin merged a task: T6230: Implement kernel-mode flow collection and/or dataplane offload collection for high volume systems.Sep 23 2025, 2:33 PM
dmbaturin added subscribers: c-po, Maintainers.
Unknown Object (User) edited projects, added VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus.Oct 9 2025, 11:36 AM
Unknown Object (User) closed this task as Resolved.Oct 10 2025, 6:43 AM
Unknown Object (User) moved this task from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q3) board.
Unknown Object (User) moved this task from Need Triage to Completed on the VyOS Rolling board.