Page MenuHomeVyOS Platform
Create Task
Maniphest T7305

Container network loses VRF on container restart
Closed, ResolvedPublicBUG
Actions

Description

Container networks set via set container network <name> lose their VRF association when the last member container is restarted.

run add container image alpine:latest
set vrf name test table 999
set container network test prefix 192.168.0.0/24
set container network test vrf test
set container name test image alpine:latest
run restart container test

Workaround: retriggering something in the network config (e.g., add and remove no-name-server) brings it back.
Workaround 2: don't let the network go empty, e.g., by running a sleep container

Originally tried on 1.5-rolling-202411260813 but seems to still be an issue in current rolling 2025.04.01-0021-rolling.

Details

Version
2025.04.01
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

jmg.caguicla created this task.Apr 1 2025, 6:37 AM
jmg.caguicla updated the task description. (Show Details)
pasik subscribed.Apr 1 2025, 7:42 AM
jmg.caguicla updated the task description. (Show Details)Apr 1 2025, 8:54 AM
Viacheslav triaged this task as Normal priority.Apr 1 2025, 9:00 AM
jmg.caguicla updated the task description. (Show Details)Apr 1 2025, 9:14 AM
c-po claimed this task.Apr 26 2025, 6:35 AM
c-po changed the task status from Open to Confirmed.Nov 1 2025, 9:07 AM
c-po added a comment.EditedNov 1 2025, 9:09 AM

Some more background information:

set interfaces ethernet eth0 vif 10 address '172.16.33.201/24'
set interfaces ethernet eth0 vif 10 vrf 'MGMT'
set vrf name MGMT protocols static route 0.0.0.0/0 next-hop 172.16.33.254 interface 'eth0.10'
set vrf name MGMT table '100'

set container name test image 'alpine:latest'
set container name test network test mac '02:02:04:a9:ce:42'
set container network test prefix '192.168.0.0/24'
set container network test vrf 'MGMT'

Check VRF routing

vyos@vyos:~$ show ip route vrf MGMT
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF MGMT:
S>* 0.0.0.0/0 [1/0] via 172.16.33.254, eth0.10, weight 1, 00:28:19
C>* 172.16.33.0/24 is directly connected, eth0.10, weight 1, 00:28:21
L>* 172.16.33.201/32 is directly connected, eth0.10, weight 1, 00:28:21
C>* 192.168.0.0/24 is directly connected, pod-test, weight 1, 00:00:13
L>* 192.168.0.1/32 is directly connected, pod-test, weight 1, 00:00:13  <<-- this is the POD
vyos@vyos:~$ ping 192.168.0.1 vrf MGMT
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.022 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=0.036 ms

The issue is during container restart:

Nov 01 10:22:10 kernel: pod-test: port 1(veth0) entered disabled state
Nov 01 10:22:10 systemd[1]: vyos-container-test.service: Failed with result 'exit-code'.
Nov 01 10:22:10 systemd[1]: Stopped VyOS Container test.
Nov 01 10:22:10 systemd[1]: Starting VyOS Container test...
Nov 01 10:22:10 kernel: pod-test: port 1(veth0) entered blocking state
Nov 01 10:22:10 kernel: pod-test: port 1(veth0) entered disabled state
Nov 01 10:22:10 kernel: veth0: entered allmulticast mode
Nov 01 10:22:10 kernel: veth0: entered promiscuous mode
Nov 01 10:22:10 kernel: pod-test: port 1(veth0) entered blocking state
Nov 01 10:22:10 kernel: pod-test: port 1(veth0) entered forwarding state

So the interface is removed and re-attached to the Kernel.

Its also written in the comment at https://github.com/vyos/vyos-1x/blob/89e4e09ece8e54303f3d5e1f1ab4927e2351b24a/src/conf_mode/container.py#L676-L691

Networks are started only as soon as there is a consumer. If only a network is created in the first place, no need to assign it to a VRF as there's no consumer, yet.

So if there is only one POD or all PODs restarted - the VRF assignment needs to be retriggered - restored.

c-po added projects: VyOS 1.5 Circinus (1.5-stream-2025-Q4), VyOS 1.4 Sagitta (1.4.4).Nov 1 2025, 9:26 AM
c-po added a comment.Nov 2 2025, 8:37 AM

https://github.com/vyos/vyos-1x/pull/4828

c-po mentioned this in rVYOSONEXf61656f9a001: vyos.ifconfig: T7305: set_vrf() should call get_vrf() helper.Nov 4 2025, 3:53 PM
c-po mentioned this in rVYOSONEXae6d3437dc32: container: T7305: fix VRF loss when restarting pods.
Restricted Repository Identity mentioned this in rVYOSONEXcc60275ab0db: Merge pull request #4828 from c-po/restart-container-vrf.Nov 4 2025, 3:53 PM
c-po moved this task from Need Triage to Completed on the VyOS Rolling board.Nov 4 2025, 4:19 PM
c-po renamed this task from Container network loses VRF on container restart to Container network looses VRF on container restart.Nov 5 2025, 8:46 PM
c-po changed the task status from Confirmed to In progress.
c-po edited projects, added VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q4).Nov 6 2025, 3:28 PM
c-po moved this task from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q3) board.
c-po closed this task as Resolved.Nov 6 2025, 4:29 PM
c-po moved this task from Backlog to Finished on the VyOS 1.4 Sagitta (1.4.4) board.
dmbaturin removed a project: VyOS 1.4 Sagitta (1.4.4).Wed, Nov 19, 2:21 PM
dmbaturin renamed this task from Container network looses VRF on container restart to Container network loses VRF on container restart.Thu, Dec 4, 7:38 PM
dmbaturin changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.
dmbaturin edited projects, added VyOS 1.5 Circinus (2025.11), VyOS 1.4 Sagitta (1.4.4); removed VyOS 1.5 Circinus (1.5-stream-2025-Q3), VyOS Rolling.Mon, Dec 8, 4:43 PM