Page MenuHomeVyOS Platform

Add selectable conntrackd synchronization modes, transports, and CPU-cost tuning
Open, NormalPublicFEATURE REQUEST

Description

Summary

Expose conntrackd's synchronization mode, transport, and CPU-cost tuning parameters in service conntrack-sync. VyOS currently hard-codes a single combination.

Hard-coded today (data/templates/conntrackd/conntrackd.conf.j2, src/conf_mode/service_conntrack-sync.py):

  • Mode FTFW. Only PurgeTimeout, DisableExternalCache, StartupResync are settable; ResendQueueSize, ACKWindowSize, CommitTimeout are not.
  • Transport is IPv4 Multicast or IPv4 UDP, selected implicitly by whether interface <ifname> peer is set. No TCP. peer, listen-address and mcast-group are IPv4-only, so no IPv6 dedicated link.
  • Filter From Userspace. Kernel-space filtering not selectable; the State filter-set not exposed.
  • NetlinkOverrunResync off, NetlinkEventsReliable on. EventIterationLimit, PollSecs, TCPWindowTracking, Scheduler not exposed.

conntrackd supports modes FTFW / ALARM / NOTRACK, transports Multicast / UDP / TCP, and IPv6 addressing on UDP and TCP.

Use case

On high-churn stateful firewall / NAT deployments, conntrackd consumes enough CPU to degrade packet forwarding on the same box. FTFW is the most expensive mode: it maintains a resend queue and handles acknowledgements for every replicated state change. Operators today have two choices — accept the load, or disable conntrack-sync and lose stateful failover.

Two aggravating factors are visible in what VyOS generates:

  1. conntrackd defaults to SCHED_RR priority 99. VyOS neither overrides it nor exposes Scheduler, so under load a userspace daemon competes at RT priority with softirq processing on the same cores.
  2. NetlinkEventsReliable on with NetlinkOverrunResync off applies back-pressure in the kernel instead of dropping events, moving cost into kernel context.

Every upstream-documented lever for reducing conntrackd CPU — kernel-space event filtering, the TCP State filter-set, EventIterationLimit, PollSecs, scheduler policy — is unreachable from the CLI.

Cases the additions cover:

  • High-churn NAT/CGNAT with short-lived flows: NOTRACK (optionally with internal cache disabled) trades some flow loss on failover for much lower cost.
  • Long-lived sessions only: PollSecs may replace the event-driven model with periodic dumps; upstream states this only guarantees recovery of long-lifetime states.
  • Fast reconvergence after a node rejoins: ALARM, at a bandwidth cost.
  • Selective replication: TCP state filtering, done in kernel space, avoids copying discarded events to userspace at all.

Additional information

  • ALARM is not a low-resource mode. Upstream describes it as "spamming" and bandwidth-heavy; it converges fast, it does not reduce load. NOTRACK is the cheaper mode. This request is about giving operators the choice and exposing the knobs that actually govern CPU cost.
  • TCP transport is discouraged upstream: congestion control adds latency, and on message loss FIFO delivery pushes the backup out of sync. It only makes sense with NOTRACK. Implement it, but validate the mode/transport combination in verify().
  • Mode parameters are not interchangeable — DisableInternalCache is NOTRACK-only, RefreshTime/CacheTimeout are ALARM-only, ResendQueueSize/ACKWindowSize are FTFW-only. verify() must reject invalid combinations rather than emitting a config conntrackd refuses at startup.
  • DisableExternalCache is already exposed but its help text omits the trade-off: upstream notes it increases CPU on the backup while saving userspace memory. Docs fix alongside this work.

Details

Version
-
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Performance optimization