Page MenuHomeVyOS Platform

bgp: changing a peer-group's export route-map walks the whole table once per member
Open, NormalPublicBUG

Description

Replacing the export route-map of a BGP peer-group (set protocols bgp peer-group X address-family ipv4-unicast route-map export NEW while another one is set) makes bgpd walk its whole table once per member of the group, back to back on its main thread. While the walks run, bgpd answers neither vtysh nor watchfrr, so the commit waits, later commits wait, and with a large table watchfrr restarts bgpd.

Cause (FRR, 10.5.2 and master): when the route-map of a peer-group changes, bgpd moves the members to the new update group one at a time (peer_route_map_set() -> peer_on_policy_change() -> update_subgroup_split_peer()), and every member except the last gets a new subgroup of its own with a copy of the old adj-out. Each of those subgroups needs a refresh, so each one gets its own announce walk. Afterwards the subgroups merge back into one.

Measured on VyOS 1.5.1 (FRR 10.5.2), 4 vCPU VM, 2.24M paths, a peer-group of 150 eBGP members, export route-map replaced by another one:

  • 150 announce walks of 1.75 s each (263 s of bgpd CPU), +149 subgroups created and merged, 147 starvation warnings;
  • a show bgp command sent during it answered after 263 s;
  • with watchfrr's default 90 s timeout bgpd was restarted in 3 of 3 tries;
  • memory: each new subgroup holds a copy of the adj-out until the walks end (150 members x 65,536 exported prefixes: +1.7 GB bgpd RSS at the peak).
  • At 2.8M paths with an 80-member group and watchfrr's timeout raised to 600 s, bgpd was still restarted and the commit returned after 14 minutes.

Editing the attached route-map instead of replacing it (for example adding a set community rule) costs one walk per subgroup (3.5 s in the same setup), so it is the safe way to change export policy today.

Possible fix. An FRR patch: when a member is split off its subgroup because of a policy change, join a subgroup that an earlier member of the same change was already split into, instead of always creating a new one.

Workaround. Edit the route-map that is attached instead of attaching a different one; keep system frr watchfrr-timeout raised.

Details

Version
1.5.1, 1.5-rolling-20261010
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)