Page MenuHomeVyOS Platform

Reorganize the flow accounting CLI
Open, HighPublic

Description

Our current flow accounting CLI is a mess. There are reasons for it and it's no one's fault, but we need to fix that nonetheless.

How it became messy: in the old days when the OS was just recently forked from Vyatta and @dmbaturin wasn't bald yet and didn't need glasses [1], there was system flow-accounting subtree handled by a single flow-accounting.pl script that generated NetFlow and sFlow configs for pmacct.

The system flow-accounting tree had subtrees netflow and sflow and a bunch of shared options like enable-eggress that would apply to pmacct as a whole.

Pmacct development had slowed down and it was no longer fast enough for modern networks, so people kept asking for better solutions.

The first thing we did was to add a different sFlow implementation based on hsflowd (T5086). We weren't yet sure if it was a good enough long-term solution for sFlow and it was pretty difficult to rework the script of system flow-accounting to handle sFlow separately, so we bit the bullet and placed it in a different subtree alongside the old implementation. People could use set system flow-accounting sflow if they wanted to use pmacct or use set system sflow if they wanted to give the new implementation a try.

Then we started reworking NetFlow to move it from pmacct to ipt-netflow. That allowed us to remove pmacct from the system completely.

Now system flow-accounting contains:

  • netflow for NetFlow options;
  • enable-eggress that was formerly shared between sFlow and NetFlow but now only applies to NetFlow;
  • vrf <name> option that is only used for verifying that the source address is assigned.

Unrelated to that, the reasons why it's system flow-accounting are purely historical. sFlow and NetFlow have nothing to do with typical system-wide settings placed in that subtree like system name-server, system host-name, or system option reboot-on-panic, and should have never been there.

What we should do:

In the CLI:

  1. Move "system flow-accounting" to "flow-accounting" (top-level node)
  2. Move "system flow-accounting enable-eggress" to "flow-accounting netflow enable-egress"
  3. Move "system flow-accounting vrf" to "flow-accounting netflow vrf" (if we need it at all)
  4. Move "system sflow" to "flow-accounting sflow"

In the code:

  1. Rename conf_mode/system_flow-accounting.py to flow-accounting_netflow.
  2. Rename system_sflow to flow-accounting_sflow.

And, obviously, update CLI versions and add migration script to move everything to the new places.

[1] Those were the days!

Details

Version
-
Is it a breaking change?
Config syntax change (migratable)
Issue type
Cosmetic issue (typos etc.)

Event Timeline

dmbaturin triaged this task as High priority.

vrf <name> option that is only used for verifying that the source address is assigned.

This was used in the past to start the old pmacct implementation in the appropriate VRF context.

set system sflow vrf also exists to mimic this behavior with the new hsflowd based implementation.

Unfortunately there is yet no implementation of VRF support in ipt-netflow - which makes this a non 1:1 replacement. For a medium skilled C developer it would not be that ward to add VRF support to ipt-netflow as we would only need to bind() the socket into the appropriate VRF.

Consolidating all implementation under one subsystem (flow-accounting) makes sense as it is somehow not related to system but also not related to service