Change Details

pmacct, unfortunately, has performance problems that make it very difficult to use in modern networks and those problems aren't easy to solve. A number of VyOS users have already been using self-built [ipt-netflow](https://github.com/aabc/ipt-netflow) instead of the official implementation as a workaround for those performance problems. We can as well adopt that, since it's kernel-mode and fast, and is already verified to work properly. My conclusion so far is that we can keep the existing CLI with minor modifications. The following options need to be moved: * `set system flow-accounting interface` — move to `set system flow-accounting netflow interface`, so that we can move `set system sflow` back under `set system flow-accounting sflow` later. The following options will need to be removed because they aren't applicable in ipt-netflow: * `set system flow-accounting disable-imt` — in-memory table plugin for pmacct, has no equivalent in ipt-netflow. * `set system flow-accounting packet-length` — how many bytes to capture for each packets, seems to have no equivalent in ipt-netflow. * `set system flow-accounting syslog-facility` — there's no logging facility setting in ipt-netflow. * `set system flow-accounting netflow sampling-rate` — doesn't seem to have an equivalent. * `set system flow-accounting netflow timeout <icmp|max-active-life|tcp-fin|tcp-generic|tcp-rst|udp>` — there are no per-protocol timeout settings in ipt-netflow. The following options can be kept with their current semantics: * `set system flow-accounting netflow version` — obviously, `protocol=`. All of 5, 9, and 10 (IPFIX) are supported in ipt-netflow. * `set system flow-accounting buffer-size` — `sndbuf=`. * `set system flow-accounting netflow engine-id` — `engine_id=`. * `set system flow-accounting netflow max-flows` — `maxflows=`. The following options can be expanded: **Collector settings** The old implementation has distinct options: `set system flow-accounting netflow server` and `set system flow-accounting netflow source-address`. ipt-netflow supports multiple destinations and per-destination source addresses and source interfaces. We could put it all under a tag node: ``` server <addr> port <num> source address <addr> interface <intf> ``` The following VyOS CLI/pmacct options need to be discussed: * `set system flow-accounting enable-egress` — it's possible to translate it as is, but do we want it to be just boolean, or do we want to start implementing flexible traffic matching rules right away? The following ipt-netflow options need a discussion: * `natevents` — ipt-netflow doesn't send NAT translation events by default. Should we enable it by default? Should there be an option to disable it? * `aggregation` — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?

pmacct, unfortunately, has performance problems that make it very difficult to use in modern networks and those problems aren't easy to solve. A number of VyOS users have already been using self-built [ipt-netflow](https://github.com/aabc/ipt-netflow) instead of the official implementation as a workaround for those performance problems. We can as well adopt that, since it's kernel-mode and fast, and is already verified to work properly. My conclusion so far is that we can keep the existing CLI with minor modifications. The following options need to be moved: * `set system flow-accounting interface` — move to `set system flow-accounting netflow interface`, so that we can move `set system sflow` back under `set system flow-accounting sflow` later. The following options will need to be removed because they aren't applicable in ipt-netflow: * `set system flow-accounting disable-imt` — in-memory table plugin for pmacct, has no equivalent in ipt-netflow. * `set system flow-accounting packet-length` — how many bytes to capture for each packets, seems to have no equivalent in ipt-netflow. * `set system flow-accounting syslog-facility` — there's no logging facility setting in ipt-netflow. * `set system flow-accounting netflow sampling-rate` — doesn't seem to have an equivalent. * `set system flow-accounting netflow timeout <icmp|max-active-life|tcp-fin|tcp-generic|tcp-rst|udp>` — there are no per-protocol timeout settings in ipt-netflow. The following options can be kept with their current semantics: * `set system flow-accounting netflow version` — obviously, `protocol=`. All of 5, 9, and 10 (IPFIX) are supported in ipt-netflow. * `set system flow-accounting buffer-size` — `sndbuf=`. * `set system flow-accounting netflow engine-id` — `engine_id=`. * `set system flow-accounting netflow max-flows` — `maxflows=`. * `set system flow-accounting enable-egress` — just add to the OUTPUT chain or not. The following options can be expanded: **Collector settings** The old implementation has distinct options: `set system flow-accounting netflow server` and `set system flow-accounting netflow source-address`. ipt-netflow supports multiple destinations and per-destination source addresses and source interfaces. We could put it all under a tag node: ``` server <addr> port <num> source address <addr> interface <intf> ``` The following ipt-netflow options need a discussion: * `natevents` — ipt-netflow doesn't send NAT translation events by default. Should we enable it by default? Should there be an option to disable it? * `aggregation` — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?

pmacct, unfortunately, has performance problems that make it very difficult to use in modern networks and those problems aren't easy to solve. A number of VyOS users have already been using self-built [ipt-netflow](https://github.com/aabc/ipt-netflow) instead of the official implementation as a workaround for those performance problems. We can as well adopt that, since it's kernel-mode and fast, and is already verified to work properly. My conclusion so far is that we can keep the existing CLI with minor modifications. The following options need to be moved: * `set system flow-accounting interface` — move to `set system flow-accounting netflow interface`, so that we can move `set system sflow` back under `set system flow-accounting sflow` later. The following options will need to be removed because they aren't applicable in ipt-netflow: * `set system flow-accounting disable-imt` — in-memory table plugin for pmacct, has no equivalent in ipt-netflow. * `set system flow-accounting packet-length` — how many bytes to capture for each packets, seems to have no equivalent in ipt-netflow. * `set system flow-accounting syslog-facility` — there's no logging facility setting in ipt-netflow. * `set system flow-accounting netflow sampling-rate` — doesn't seem to have an equivalent. * `set system flow-accounting netflow timeout <icmp|max-active-life|tcp-fin|tcp-generic|tcp-rst|udp>` — there are no per-protocol timeout settings in ipt-netflow. The following options can be kept with their current semantics: * `set system flow-accounting netflow version` — obviously, `protocol=`. All of 5, 9, and 10 (IPFIX) are supported in ipt-netflow. * `set system flow-accounting buffer-size` — `sndbuf=`. * `set system flow-accounting netflow engine-id` — `engine_id=`. * `set system flow-accounting netflow max-flows` — `maxflows=`. * `set system flow-accounting enable-egress` — just add to the OUTPUT chain or not. The following options can be expanded: **Collector settings** The old implementation has distinct options: `set system flow-accounting netflow server` and `set system flow-accounting netflow source-address`. ipt-netflow supports multiple destinations and per-destination source addresses and source interfaces. We could put it all under a tag node: ``` server <addr> port <num> source address <addr> interface <intf> ``` The following VyOS CLI/pmacct options need to be discussed: * `set system flow-accounting enable-egress` — it's possible to translate it as is, but do we want it to be just boolean, or do we want to start implementing flexible traffic matching rules right away? The following ipt-netflow options need a discussion: * `natevents` — ipt-netflow doesn't send NAT translation events by default. Should we enable it by default? Should there be an option to disable it? * `aggregation` — ipt-netflow allows rewriting IP prefixes and ports. Do we want that?