After spending several hours debugging why ping and BGP would show up in TCP dumps when utilizing ZBF with VRFs i have found the root cause, documented below:
When utilizing the ZBF with VRF, issues occur with outgoing traffic - specifically with ping packets destined for interface on vyos itself.
eg a ping to a local interface assigned to a VRF.
the following commands result int he following outputs
set firewall zone LOCAL default-action 'drop'
set firewall zone LOCAL from MGMT firewall ipv6-name 'IPV6_MGMT_TO_LOCAL'
set firewall zone LOCAL from MGMT firewall name 'IPV4_MGMT_TO_LOCAL'
set firewall zone LOCAL local-zone
set firewall zone MGMT default-action 'reject'
set firewall zone MGMT from LOCAL firewall ipv6-name 'IPV6_LOCAL_TO_MGMT'
set firewall zone MGMT from LOCAL firewall name 'IPV4_LOCAL_TO_MGMT'
set firewall zone MGMT intra-zone-filtering action 'accept'
set firewall zone MGMT member interface 'bond1.4002'
set firewall zone MGMT member interface 'bond1.4003'
set firewall zone MGMT member interface 'dum0'
set firewall zone MGMT member interface 'eth1.20'
set firewall zone MGMT member vrf 'MGMT'
output in netfilter
vyos@fw1:~$ sudo nft list chain ip vyos_filter VZONE_LOCAL_OUT
table ip vyos_filter {
chain VZONE_LOCAL_OUT {
oifname "lo" counter packets 3060 bytes 208080 return
oifname "eth1.10" counter packets 3146 bytes 127689 jump NAME_IPV4_LOCAL_TO_HA-SYNC
oifname "eth1.10" counter packets 0 bytes 0 return
oifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 jump NAME_IPV4_LOCAL_TO_MGMT
oifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 return
oifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 jump NAME_IPV4_LOCAL_TO_MGMT
oifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 return
counter packets 1421 bytes 96804 drop comment "zone_LOCAL default-action drop"
}
}
vyos@fw1:~$ sudo nft list chain ip vyos_filter VZONE_LOCAL_IN
table ip vyos_filter {
chain VZONE_LOCAL_IN {
iifname "lo" counter packets 3064 bytes 208336 return
iifname "eth1.10" counter packets 86 bytes 5289 jump NAME_IPV4_HA-SYNC_TO_LOCAL
iifname "eth1.10" counter packets 0 bytes 0 return
iifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 jump NAME_IPV4_MGMT_TO_LOCAL
iifname { "dum0", "wg52000", "eth1.20", "bond0.4002", "bond0.4003" } counter packets 0 bytes 0 return
iifname "MGMT" counter packets 1029 bytes 32928 jump NAME_IPV4_MGMT_TO_LOCAL
iifname "MGMT" counter packets 0 bytes 0 return
counter packets 1545 bytes 49440 drop comment "zone_LOCAL default-action drop"
}
}
vyos@fw1:~$
as it is seen inf the "VZONE_{{ zone_name }}_OUT" part of the fil: https://github.com/vyos/vyos-1x/blob/current/data/templates/firewall/nftables-zone.j2
id adds a iifname "MGMT" corresponding to the zone
but in the "VZONE_{{ zone_name }}_IN" part, of the file it simply adds the interfaces instead of the VRF - this might be intentional, but it causes errors when utilizing BGP for neighbourships or expecting to make ICMP packet to verify interface connectivity as ICMP reply packets are bieng cault in the "default drop" as shown by the packet counters
I added the below entries in netfilter above the default drop, and it solved the issue
table ip vyos_filter {
oifname "MGMT" counter packets 0 bytes 0 jump NAME_IPV4_LOCAL_TO_MGMT
oifname "MGMT" counter packets 0 bytes 0 return
in my case, if one would like the existing behaviour, why not just use the interface member as this does excactly the same as the current VRF-member bahaviour. the behaviurs is also identical to the other parts, ant therefor it seems like a miss - pull request vill be created with a fix