In my setup I have two ISPs. One is static and the other is dynamic. I was using VyOS_1.4-rolling-202101171022 for a while and things worked fine until one day this no longer worked. I installed VyOS_1.4-rolling-202106180936 and had the same issue. Then I tried VyOS_1.3-beta-202106180642 and it worked again. I upgraded to vyos-1.3.0-rc6-amd64 and it continued working. I compiled vyos-1.3.0-amd64.iso and installed it and it lo longer works. I rolled back to vyos-1.3.0-rc6-amd64 and it worked as expected. I never saw epa releases posted, so I'm not sure at what point after 1.3.0 rc6 this feature broke.
The issue when it doesn't work is that the table is not updated with the route for the dynamic connection. You can see what it should look like from the 'show ip route table 111' command below. When it's not working the table is blank. I have the relevant config below. I have removed IP specifics from the outputs.
[email protected]:~$ show ip route table 111 Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued, r - rejected, b - backup VRF default table 111: S>* 0.0.0.0/0 [1/0] via x.x.x.1, bond1.111, weight 1, 00:28:40 [email protected]:~$ [email protected]:~$ show conf commands | match 'network-group (FIOS|WOW)_01_INET_NETS network|policy route|table' set firewall group network-group FIOS_01_INET_NETS network '192.168.x.0/24' set firewall group network-group FIOS_01_INET_NETS network '10.x.x.0/24' set firewall group network-group FIOS_01_INET_NETS network '10.x.x.0/24' set firewall group network-group FIOS_01_INET_NETS network '10.x.x.0/24' set firewall group network-group FIOS_01_INET_NETS network '10.x.x.0/24' set firewall group network-group FIOS_01_INET_NETS network '192.168.x.0/24' set firewall group network-group WOW_01_INET_NETS network 'x.x.x.x/29' set firewall group network-group WOW_01_INET_NETS network 'x.x.x.x/29' set firewall group network-group WOW_01_INET_NETS network 'x.x.x.x/27' set firewall group network-group WOW_01_INET_NETS network 'x.x.x.x/27' set firewall group network-group WOW_01_INET_NETS network 'x.x.x.x/28' set interfaces bonding bond1 vif 211 policy route 'FIOS_01_INET' set interfaces bonding bond1 vif 221 policy route 'WOW_01_INET' set policy route FIOS_01_INET rule 1000 description 'Route traffic to ISP Modem (192.168.x.x) to ISP Interface' set policy route FIOS_01_INET rule 1000 destination address '192.168.x.0/24' set policy route FIOS_01_INET rule 1000 disable set policy route FIOS_01_INET rule 1000 set table 'main' set policy route FIOS_01_INET rule 1001 description 'Route traffic from the specified subnets through FIOS_01_INET' set policy route FIOS_01_INET rule 1001 set table '111' set policy route FIOS_01_INET rule 1001 source group network-group 'FIOS_01_INET_NETS' set policy route WOW_01_INET rule 1001 description 'Route traffic from the specified subnets through WOW_01_INET' set policy route WOW_01_INET rule 1001 set table '121' set policy route WOW_01_INET rule 1001 source group network-group 'WOW_01_INET_NETS' set protocols static table 111 route 0.0.0.0/0 dhcp-interface 'bond1.111' set protocols static table 121 route 0.0.0.0/0 next-hop x.x.x.x [email protected]:~$