Parsing a config with a large multi-value leaf, such as the network values of a firewall network-group, takes time proportional to the square of the number of values. On VyOS 1.5.1 one parse of a config with a 50,000-member network-group took 24–27 s, and a commit parses the config 8 times.
Root cause.
- The parser builds one leaf node per value line and then merges the same-named nodes (Vytree.merge_children, src/vytree.ml).
- The merge folded their data from the left, and the parser's merge_data appends the right node's values to the left node's, which copies the list built so far.
- Value k therefore copies the k−1 values before it: n values cost about n²/2 list cells, plus the garbage collection of those copies.
- The fixes of T9374 and T9375 (same-named tag nodes such as prefix-list rules) do not change this path.
Measurements. VyOS 1.5.1 VM (4 vCPU), config with a 10,001-rule firewall ruleset and a 50,000-member network-group. Both columns had the fixes of T9374, T9375, T9376, T9377 and T9379 installed; only the parser library differs between them. The commit row therefore includes the effect of those fixes in both columns.
| before | after | |
|---|---|---|
| one parse of the config (ConfigTree(), on the VM) | 24.2 s | 0.128 s |
| the 8 parses of one firewall rule commit | 204.1 s (24.2–27.2 s each) | 0.98 s (0.119–0.125 s each) |
| that commit, end to end ¹ | 516.1 s | 289.2 s |
| NIC naming at boot (vyos_net_name parses config.boot) | 24.5 s | 0.14 s |
¹ Before and after ran on two nodes of the same image, size and config. What remains of the commit is the legacy config store (whole-config reads through unionfs-fuse), which this fix does not touch.
One leaf, parse time alone (parser test harness, same OCaml 4.14.2 as the 1.5.1 build, T9374 and T9375 in both columns): 16,000 values 0.90 → 0.014 s, 32,000 values 6.3 → 0.030 s, 50,000 values 23.0 → 0.045 s, 64,000 values 47.8 → 0.062 s (rolling tree; on the 1.5.1 tree 64,000 values 48.4 → 0.059 s).
Fix direction. Collect the data of the same-named nodes and merge it once, from the right, so that each step copies one node's values only.