Page MenuHomeVyOS Platform

The firewall does not filter incoming traffic on the interface with vrf.
Confirmed, NormalPublicBUG

Description

Incoming traffic is not filtered firewall on the eth0 interface if vrf is used. This example does not block ssh access to the router. Configuration:

set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall config-trap 'disable'
set firewall group address-group MGT-IP address '192.168.122.220'
set firewall ipv6-receive-redirects 'disable'
set firewall ipv6-src-route 'disable'
set firewall ip-src-route 'disable'
set firewall log-martians 'enable'
set firewall name MGT-LOCAL default-action 'drop'
set firewall name MGT-LOCAL description 'MGT-LOCAL'
set firewall name MGT-LOCAL rule 910 action 'accept'
set firewall name MGT-LOCAL rule 910 description 'Allow SSH Login'
set firewall name MGT-LOCAL rule 910 destination port '22'
set firewall name MGT-LOCAL rule 910 log 'enable'
set firewall name MGT-LOCAL rule 910 protocol 'tcp'
set firewall name MGT-LOCAL rule 910 source group address-group 'MGT-IP'
set firewall name MGT-LOCAL rule 910 state established 'enable'
set firewall name MGT-LOCAL rule 910 state new 'enable'
set firewall name MGT-LOCAL rule 910 state related 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall syn-cookies 'enable'
set firewall twa-hazards-protection 'disable'
set interfaces ethernet eth0 address '192.168.122.100/24'
set interfaces ethernet eth0 firewall local name 'MGT-LOCAL'
set interfaces ethernet eth0 vrf 'MGT'
set service ssh listen-address '192.168.122.100'
set service ssh port '22'
set service ssh vrf 'MGT'
set vrf name MGT table '200'

The problem is that vrf changes the name of the interface "eth0" to "MGT" and the rule does not filtering incoming packets:

trace id ce7870ee ip filter VYATTA_FW_LOCAL_HOOK packet: iif "MGT" ether saddr 0c:50:ab:54:00:00 ether daddr 0c:33:f8:61:00:00 ip saddr 192.168.122.220 ip daddr 192.168.122.100 ip dscp cs0 ip ecn not-ect ip ttl 128 ip id 10429 ip length 52 tcp sport 49263 tcp dport 22 tcp flags == syn tcp window 8192

If you change the configuration:

set interfaces ethernet eth0 firewall in name 'MGT-LOCAL'

The firewall rule does not filter incoming traffic. The input interface "eth0" still name "MGT":

trace id db39c263 ip filter VYATTA_FW_IN_HOOK packet: iif "MGT" oif "eth1" ether saddr 0c:50:ab:54:00:00 ether daddr 0c:33:f8:61:00:00 ip saddr 192.168.122.220 ip daddr 192.168.123.2 ip dscp cs0 ip ecn not-ect ip ttl 127 ip id 17604 ip length 52 tcp sport 49274 tcp dport 22 tcp flags == syn tcp window 8192

Details

Difficulty level
Hard (possibly days)
Version
VyOS 1.3.0-epa2 and VyOS 1.4-rolling-202110220645
Why the issue appeared?
Design mistake
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

zsdc changed the task status from Open to Confirmed.Oct 26 2021, 7:40 AM
zsdc triaged this task as Normal priority.
zsdc changed Difficulty level from Unknown (require assessment) to Hard (possibly days).
zsdc changed Why the issue appeared? from Will be filled on close to Design mistake.

Definitely miss behavior is generated by the new interface MGT that was created when assigning MGT vrf to eth0.

Important entries in iptables related to this config:

-A VYATTA_FW_LOCAL_HOOK -i eth0 -c 0 0 -j MGT-LOCAL
...
-A MGT-LOCAL -p tcp -m state --state NEW,RELATED,ESTABLISHED -m set --match-set MGT-IP src -m tcp --dport 22 -m comment --comment MGT-LOCAL-910 -c 0 0 -j LOG --log-prefix "[MGT-LOCAL-910-A] "
-A MGT-LOCAL -p tcp -m state --state NEW,RELATED,ESTABLISHED -m set --match-set MGT-IP src -m tcp --dport 22 -m comment --comment MGT-LOCAL-910 -c 0 0 -j RETURN
-A MGT-LOCAL -m comment --comment "MGT-LOCAL-10000 default-action drop" -c 0 0 -j DROP

So firewall will filter when traffic gets in through interface eth0, and as is incoming through MGT interface, no counters were increased.

Adding next rules:

vyos@vyos# sudo iptables -A VYATTA_FW_LOCAL_HOOK -i MGT -j MGT-LOCAL

And then, sshing from a forbidden host, counter are increased, and connection is denied:

-A VYATTA_FW_LOCAL_HOOK -i eth0 -c 0 0 -j MGT-LOCAL
-A VYATTA_FW_LOCAL_HOOK -i MGT -c 4 252 -j MGT-LOCAL
-A MGT-LOCAL -p tcp -m state --state NEW,RELATED,ESTABLISHED -m set --match-set MGT-IP src -m tcp --dport 22 -m comment --comment MGT-LOCAL-910 -c 0 0 -j LOG --log-prefix "[MGT-LOCAL-910-A] "
-A MGT-LOCAL -p tcp -m state --state NEW,RELATED,ESTABLISHED -m set --match-set MGT-IP src -m tcp --dport 22 -m comment --comment MGT-LOCAL-910 -c 0 0 -j RETURN
-A MGT-LOCAL -m comment --comment "MGT-LOCAL-10000 default-action drop" -c 4 252 -j DROP

PR https://github.com/vyos/vyos-1x/pull/1330

set firewall name FOO default-action 'accept'
set firewall name FOO description 'desc'
set firewall name FOO rule 10 action 'drop'
set firewall name FOO rule 10 source address '8.8.8.8'
set interfaces ethernet eth0 firewall local name 'FOO'
set interfaces ethernet eth0 vrf 'ONE'
set vrf name ONE table '150'

Check:

table ip filter {
	chain VYOS_FW_LOCAL {
		type filter hook input priority filter; policy accept;
		iifname "ONE" counter packets 63 bytes 6024 jump NAME_FOO
		jump VYOS_POST_FW
	}
...
	chain NAME_FOO {
		ip saddr 8.8.8.8 counter packets 79 bytes 6636 drop comment "FOO-10"
		counter packets 3 bytes 984 return comment "FOO default-action accept"
	}
}

There is an issue with vrf device for LOCAL direction
Imagine if you have 50 interfaces in one VRF and you want to drop all traffic from one interface for example - eth2 and don't touch other interfaces
You set firewall on eth2 Local - drop all traffic for device vrf and it will be affected to another 49 interfaces as iifname VRF_DEVICE the same

@Viacheslav I tested your fix in my environment. The inbound filtering worked as expected after the fix. However it did not work correctly for the case we where we want both inbound and outbound firewalls on a single vrf member interface (or any case that has more than 2 directions on the same interface).

table ip filter {
	chain VYOS_FW_LOCAL {
		type filter hook input priority filter; policy accept;
		oifname "ONE" counter packets 63 bytes 6024 jump NAME_FOO # <<< Problem here, oifname should be eth0, not vrf name
		iifname "ONE" counter packets 63 bytes 6024 jump NAME_FOO
		jump VYOS_POST_FW
	}
...
	chain NAME_FOO {
		ip saddr 8.8.8.8 counter packets 79 bytes 6636 drop comment "FOO-10"
		counter packets 3 bytes 984 return comment "FOO default-action accept"
	}
}

How to reproduce:

  1. Apply inbound firewall on vrf member interface
  2. Apply outbound firewall on the same vrf member interface
  3. Check nftables

You should apply your fix in apply() function instead of get_config() as it loops through all fw directions instead of just the first one.

@Viacheslav As for your other concern, you can filter the actual inbound interface (eth4 in this my case) in mangle-PREROUTING. Maybe you could try packet marking in mangle-PREROUTING, then filter them later in VYOS_FW_FORWARD/VYOS_FW_LOCAL in the filter table?
Something like this:

vyos@vyos:~$ conf
[edit]
vyos@vyos# show i ethernet eth4
 address 10.106.100.2/29
 description peer0
 }
 vrf vrf-peer0
[edit]
vyos@vyos# sudo nft -a list table mangle
table ip mangle { # handle 35
        chain VYOS_PBR_PREROUTING { # handle 1
                type filter hook prerouting priority mangle; policy accept;
                iifname "eth4" counter packets 465 bytes 39060 meta mark set 0x00001388 # handle 6
        }

        chain VYOS_PBR_POSTROUTING { # handle 2
                type filter hook postrouting priority mangle; policy accept;
        }
}
[edit]

vyos@vyos# sudo nft -a list table filter
table ip filter { # handle 28
        chain VYOS_FW_FORWARD { # handle 1
                type filter hook forward priority filter; policy accept;
                iifname "vrf-peer0" meta mark 0x00001388 counter packets 33 bytes 2772 accept # handle 269
                jump VYOS_POST_FW # handle 6
        }

        ...
}
[edit]

Just an idea, I'm not sure about VyOS' packet marking convention so I can't suggest how to implement this.

@Viacheslav I tested your fix in my environment. The inbound filtering worked as expected after the fix. However it did not work correctly for the case we where we want both inbound and outbound firewalls on a single vrf member interface (or any case that has more than 2 directions on the same interface).

table ip filter {
	chain VYOS_FW_LOCAL {
		type filter hook input priority filter; policy accept;
		oifname "ONE" counter packets 63 bytes 6024 jump NAME_FOO # <<< Problem here, oifname should be eth0, not vrf name
		iifname "ONE" counter packets 63 bytes 6024 jump NAME_FOO
		jump VYOS_POST_FW
	}
...
	chain NAME_FOO {
		ip saddr 8.8.8.8 counter packets 79 bytes 6636 drop comment "FOO-10"
		counter packets 3 bytes 984 return comment "FOO default-action accept"
	}
}

It shouldn't touch outbound direction, for outbound, it is expected ethX interface, as I remember this fix touched only traffic to the router itself and inbound (LOCAL/IN)
Outbound was worked as expected. But need retest

@zsdc what do you think about marking?

True, marking packets can help. I would only be very careful because we use marks a lot for PBR, LB, etc. Not sure if they can conflict with each other. Also, the performance is the question - better to check how marking each packet on an interface affects it.

Another related issue, by the way (I think that we should think about both of them as different sides of the same problem). If an interface is a bridge member firewall will also not work. Nothing will be filtered with this, for example:

set firewall name drop_all default-action 'drop'
set interfaces bridge br1 member interface eth1
set interfaces ethernet eth1 firewall in name 'drop_all'

The root cause is the same, but with one difference - an original/physical interface is not available at all, even on the prerouting hook. But, obviously, it is available in the bridge table.

Based on this, I would say that better to consider moving the in firewall from forward to prerouting hook.

Moving in from forwardto prerouting doesn't seem to be a good idea. Filtering in prerouting will also filter local traffic.
Also, as remarked in previous entry, I would try to avoid using marks in mangle, since it may lead to mayor problems/incompatibilities when PBR also present in configuration.

Instead of mangle prerouting, in certain way filtering can also be done in filter prerouting. But once again is prerouting, so there are several things that should be considered.

In case of filtering on a VRF, would it be an idea to use the MAC address instead of the interface name in the rule?

table ip filter {
	chain VYOS_FW_LOCAL {
		type filter hook input priority filter; policy accept;
		ether daddr 00:90:27:b4:40:59 counter packets 63 bytes 6024 jump NAME_FOO
		jump VYOS_POST_FW
	}
...
	chain NAME_FOO {
		ip saddr 8.8.8.8 counter packets 79 bytes 6636 drop comment "FOO-10"
		counter packets 3 bytes 984 return comment "FOO default-action accept"
	}
}

Didn't test it though... I think I read it somewhere but cannot find it anymore.