Page MenuHomeVyOS Platform

PPPoE ip-up hook races the commit that (re)starts pppd: reads pre-commit config and silently installs neither the default route nor adjust-mss rules
Open, LowPublic

Description

When a config commit re-enables a PPPoE interface (e.g. delete interfaces pppoe pppoe0 disable + commit), interfaces_pppoe.py's apply phase restarts ppp@pppoe0.service while the commit is still in flight. If the PPPoE session establishes quickly (a few seconds), pppd's ip-up hook (/etc/ppp/ip-up.d/99-vyos-pppoe-callback) runs before the commit completes. The hook reads the interface config via ConfigTreeQuery + get_interface_dict(), which at that moment can still return the pre-commit active config — in which the interface is still disabled. PPPoEIf.update() then configures nothing and the hook exits 0.

The result is a PPPoE interface that is UP with a valid address but:

  • no default route (with default-route auto, the route is only installed by this hook — same presentation as the long-fixed T2380), and
  • no adjust-mss nftables rules (the VYOS_TCP_MSS chains are left empty, because the preceding disable commit's PPPoEIf.remove() cleaned the interface's rules and the racing hook never re-adds them).

Everything reports success: the commit succeeds, pppd logs a normal connection, Script /etc/ppp/ip-up finished ... status = 0x0, and the committed config contains both default-route auto and the adjust-mss values. The failure is completely silent and config.boot inspection is misleading — only the running state (ip route, nft list chain ip raw VYOS_TCP_MSS) reveals it.

Steps to reproduce

On a router with a working PPPoE WAN (default-route auto, ip adjust-mss 1452), from a script or API client (anything that commits programmatically; the window is timing-dependent and easiest to hit when the session dials fast):

  1. set interfaces pppoe pppoe0 disable + commit.
  2. delete interfaces pppoe pppoe0 disable + commit.
  3. If the PPPoE session establishes before commit (2) finishes (observed: session up at T+7s, commit finished at T+15s), check ip route and nft list chain ip raw VYOS_TCP_MSS.

Expected: default route present; MSS clamp rules present.
Actual (when the race is hit): neither is installed; the ip-up hook exited 0.

Observed timeline from a production occurrence (journal timestamps):

20:11:15  script: delete .. disable ; commit (ConfigSession)
20:11:22  pppd: local IP address a.b.c.d
20:11:22  pppd: Script /etc/ppp/ip-up started
20:11:23  pppd: Script /etc/ppp/ip-up finished status = 0x0   <-- read stale config, did nothing
20:11:30  commit: Successful change to active configuration    <-- commit finishes AFTER the hook

After this: pppoe0 UP with address, show ip route has no 0.0.0.0/0, VYOS_TCP_MSS chains empty. In our case this took a site offline for ~1 hour (the same sequence 30 minutes earlier happened to win the race and came up fine — the outcome is nondeterministic).

Analysis

  • interfaces_pppoe.py apply(): on enable it (re)starts the systemd unit and relies on the ip-up callback for address-dependent state ("When interface comes 'live' a hook is called ... which triggers PPPoEIf.update()").
  • 99-vyos-pppoe-callback does conf = ConfigTreeQuery(); _, pppoe = get_interface_dict(conf.config, ['interfaces','pppoe'], ifname); PPPoEIf(interface).update(pppoe) — with no awareness that the commit that spawned this pppd may still be mid-flight, and no retry. When the read returns the pre-commit tree (interface disabled), update() is effectively a no-op, and the hook still exits 0.
  • Contrast with keepalived-fifo.py, which explicitly retries its config load while "a commit is in progress" — the pppoe callback has no such guard.

Suggested fixes (any of)

  1. Make the callback wait/retry while a commit is in progress (the keepalived-fifo pattern), or
  2. re-run PPPoEIf.update() from the commit's apply phase once the interface has an address (post-commit), or
  3. have the callback detect the "interface disabled in the tree I just read, yet pppd just dialed me" contradiction and re-read/log loudly instead of silently doing nothing.

A config-tracked protocols static route 0.0.0.0/0 interface pppoe0 is NOT a complete workaround: staticd keys static routes by (prefix, nexthop), so the hook's tag-210 route and the user's untagged route become one FRR object, and PPPoEIf._remove_routes()'s no ip route 0.0.0.0/0 <if> tag 210 on the next disable deletes it regardless of which writer asserted it (verified live).

Related existing tasks

  • T6639 (open): this report appears to be a concrete, user-visible instance of that task's item 1 — a Config/ConfigTreeQuery object instantiated outside the vyos-configd session "will miss the ambient config of vyos-configd, only picking up the effective config." The pppoe ip-up callback does exactly that, from a process (pppd's hook) that the in-flight commit itself spawned.
  • T2380 (closed, 1.3): same symptom via a since-fixed pon/poff→systemd mechanism — not a regression of that fix; the mechanism here is the config-read race.
  • T2502 (closed): IPv6 default-route-auto not installed — same route-installation surface.
  • T5934 (closed Invalid): the same callback never runs at all for IPv6-only sessions — adjacent gap in the same hook.
  • T6638 / T6642 (closed, 1.4.1/1.5): the QoS-on-pppoe variant of "config exists but interface state doesn't at apply time" — same commit-vs-interface-lifecycle family.

Workaround we deployed

Our automation now re-runs the callback's own logic (PPPoEIf.update() with a fresh config read) after its commit completes and the interface has an address — idempotent, restores both the default route and the MSS rules deterministically.

Details

Version
VyOS 1.5-rolling (1.5-rolling-cloudinit build; FRR 10.6.1, pppd 2.4.9)
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)