Page MenuHomeVyOS Platform

Anomalies in show firewall command after refactoring
Closed, ResolvedPublicBUG

Description

Attempting to setup a skeleton/template regarding zone-based firewalling with the new firewall frontend (since refactoring early august 2023).

For config scroll to the bottom.

Noticed these three anomalies (so far):

  1. show firewall group doesnt properly display members of each interface-group:
vyos@vyos:~$ show firewall group 
Firewall Groups

Name    Type             References    Members
------  ---------------  ------------  ---------
DMZ     interface_group  N/A           N/A
LAN     interface_group  N/A           N/A
MGMT    interface_group  N/A           N/A
WAN     interface_group  N/A           N/A
  1. show firewall statistics shows all counters for packets and bytes as 0 (se below for output).
  1. show firewall statistics displays source/destination for "IPv6 Firewall" as "0.0.0.0/0" where "::/0" or similar would be expected - that is it displays as a IPv4-address instead of a IPv6-address:
vyos@vyos:~$ show firewall statistics 
Rulesets Statistics

---------------------------------
IPv4 Firewall "forward filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
40               0        0  jump      0.0.0.0/0  0.0.0.0/0
50               0        0  jump      0.0.0.0/0  0.0.0.0/0
60               0        0  jump      0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv4 Firewall "input filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv4 Firewall "name V4_TO_DMZ"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv4 Firewall "name V4_TO_LAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv4 Firewall "name V4_TO_WAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv4 Firewall "output filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "forward filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
40               0        0  jump      0.0.0.0/0  0.0.0.0/0
50               0        0  jump      0.0.0.0/0  0.0.0.0/0
60               0        0  jump      0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "input filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_DMZ"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_LAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_WAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "output filter"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0
20               0        0  accept    0.0.0.0/0  0.0.0.0/0
30               0        0  accept    0.0.0.0/0  0.0.0.0/0
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

Using following config:

set firewall global-options all-ping 'enable'
set firewall global-options broadcast-ping 'disable'
set firewall global-options ip-src-route 'disable'
set firewall global-options ipv6-receive-redirects 'disable'
set firewall global-options ipv6-src-route 'disable'
set firewall global-options log-martians 'enable'
set firewall global-options receive-redirects 'disable'
set firewall global-options resolver-cache
set firewall global-options resolver-interval '60'
set firewall global-options send-redirects 'disable'
set firewall global-options source-validation 'strict'
set firewall global-options syn-cookies 'enable'
set firewall global-options twa-hazards-protection 'disable'
set firewall group interface-group DMZ interface 'eth2'
set firewall group interface-group LAN interface 'eth3'
set firewall group interface-group MGMT interface 'eth0'
set firewall group interface-group WAN interface 'eth1'
set firewall ipv4 forward filter default-action 'accept'
set firewall ipv4 forward filter rule 10 action 'drop'
set firewall ipv4 forward filter rule 10 state invalid 'enable'
set firewall ipv4 forward filter rule 20 action 'accept'
set firewall ipv4 forward filter rule 20 state established 'enable'
set firewall ipv4 forward filter rule 30 action 'accept'
set firewall ipv4 forward filter rule 30 state related 'enable'
set firewall ipv4 forward filter rule 40 action 'jump'
set firewall ipv4 forward filter rule 40 jump-target 'V4_TO_WAN'
set firewall ipv4 forward filter rule 40 outbound-interface interface-group 'WAN'
set firewall ipv4 forward filter rule 50 action 'jump'
set firewall ipv4 forward filter rule 50 jump-target 'V4_TO_DMZ'
set firewall ipv4 forward filter rule 50 outbound-interface interface-group 'DMZ'
set firewall ipv4 forward filter rule 60 action 'jump'
set firewall ipv4 forward filter rule 60 jump-target 'V4_TO_LAN'
set firewall ipv4 forward filter rule 60 outbound-interface interface-group 'LAN'
set firewall ipv4 input filter default-action 'accept'
set firewall ipv4 input filter rule 10 action 'drop'
set firewall ipv4 input filter rule 10 state invalid 'enable'
set firewall ipv4 input filter rule 20 action 'accept'
set firewall ipv4 input filter rule 20 state established 'enable'
set firewall ipv4 input filter rule 30 action 'accept'
set firewall ipv4 input filter rule 30 state related 'enable'
set firewall ipv4 name V4_TO_DMZ default-action 'return'
set firewall ipv4 name V4_TO_LAN default-action 'return'
set firewall ipv4 name V4_TO_WAN default-action 'return'
set firewall ipv4 output filter default-action 'accept'
set firewall ipv4 output filter rule 10 action 'drop'
set firewall ipv4 output filter rule 10 state invalid 'enable'
set firewall ipv4 output filter rule 20 action 'accept'
set firewall ipv4 output filter rule 20 state established 'enable'
set firewall ipv4 output filter rule 30 action 'accept'
set firewall ipv4 output filter rule 30 state related 'enable'
set firewall ipv6 forward filter default-action 'accept'
set firewall ipv6 forward filter rule 10 action 'drop'
set firewall ipv6 forward filter rule 10 state invalid 'enable'
set firewall ipv6 forward filter rule 20 action 'accept'
set firewall ipv6 forward filter rule 20 state established 'enable'
set firewall ipv6 forward filter rule 30 action 'accept'
set firewall ipv6 forward filter rule 30 state related 'enable'
set firewall ipv6 forward filter rule 40 action 'jump'
set firewall ipv6 forward filter rule 40 jump-target 'V6_TO_WAN'
set firewall ipv6 forward filter rule 40 outbound-interface interface-group 'WAN'
set firewall ipv6 forward filter rule 50 action 'jump'
set firewall ipv6 forward filter rule 50 jump-target 'V6_TO_DMZ'
set firewall ipv6 forward filter rule 50 outbound-interface interface-group 'DMZ'
set firewall ipv6 forward filter rule 60 action 'jump'
set firewall ipv6 forward filter rule 60 jump-target 'V6_TO_LAN'
set firewall ipv6 forward filter rule 60 outbound-interface interface-group 'LAN'
set firewall ipv6 input filter default-action 'accept'
set firewall ipv6 input filter rule 10 action 'drop'
set firewall ipv6 input filter rule 10 state invalid 'enable'
set firewall ipv6 input filter rule 20 action 'accept'
set firewall ipv6 input filter rule 20 state established 'enable'
set firewall ipv6 input filter rule 30 action 'accept'
set firewall ipv6 input filter rule 30 state related 'enable'
set firewall ipv6 name V6_TO_DMZ default-action 'return'
set firewall ipv6 name V6_TO_LAN default-action 'return'
set firewall ipv6 name V6_TO_WAN default-action 'return'
set firewall ipv6 output filter default-action 'accept'
set firewall ipv6 output filter rule 10 action 'drop'
set firewall ipv6 output filter rule 10 state invalid 'enable'
set firewall ipv6 output filter rule 20 action 'accept'
set firewall ipv6 output filter rule 20 state established 'enable'
set firewall ipv6 output filter rule 30 action 'accept'
set firewall ipv6 output filter rule 30 state related 'enable'

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202308260020
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Fixed on this op-mode commands were introduced on PR https://github.com/vyos/vyos-1x/pull/2186

Please check again on next rolling release.

n.fort changed the task status from Open to Needs testing.Aug 30 2023, 1:54 PM
  1. Error in show firewall group:
vyos@vyos:~$ show firewall group 
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/firewall.py", line 401, in <module>
    show_firewall_group(args.name)
  File "/usr/libexec/vyos/op_mode/firewall.py", line 305, in show_firewall_group
    references = find_references(group_type, group_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/libexec/vyos/op_mode/firewall.py", line 270, in find_references
    for rule_id, rule_conf in priority_conf['rule'].items():
                              ~~~~~~~~~~~~~^^^^^^^^
KeyError: 'rule'
  1. The IPv6-addresses which previously were being displayed as IPv4-addresses have been somewhat resolved, "default" still displays IPv4-address when IPv6 should be shown:
vyos@vyos:~$ show firewall statistics
Rulesets Statistics

...

---------------------------------
IPv6 Firewall "forward filter"

Rule       Packets    Bytes  Action    Source     Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  ---------  -------------  -------------------  --------------------
10               0        0  drop      ::/0       ::/0           any                  any
20               0        0  accept    ::/0       ::/0           any                  any
30               0        0  accept    ::/0       ::/0           any                  any
40               0        0  jump      ::/0       ::/0           any                  WAN
50               0        0  jump      ::/0       ::/0           any                  DMZ
60               0        0  jump      ::/0       ::/0           any                  LAN
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "input filter"

Rule       Packets    Bytes  Action    Source     Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  ---------  -------------  -------------------  --------------------
10               0        0  drop      ::/0       ::/0           any                  any
20               0        0  accept    ::/0       ::/0           any                  any
30               0        0  accept    ::/0       ::/0           any                  any
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_DMZ"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_LAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "name V6_TO_WAN"

Rule       Packets    Bytes  Action    Source     Destination
-------  ---------  -------  --------  ---------  -------------
default          0        0  return    0.0.0.0/0  0.0.0.0/0

---------------------------------
IPv6 Firewall "output filter"

Rule       Packets    Bytes  Action    Source     Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  ---------  -------------  -------------------  --------------------
10               0        0  drop      ::/0       ::/0           any                  any
20               0        0  accept    ::/0       ::/0           any                  any
30               0        0  accept    ::/0       ::/0           any                  any
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

However packets/bytes counters looks like it have been resolved:

vyos@vyos:~$ show firewall
Rulesets Information

...

---------------------------------
IPv4 Firewall "input filter"

Rule     Action    Protocol      Packets    Bytes  Conditions
-------  --------  ----------  ---------  -------  ----------------------------
10       drop      all                 0        0  ct state invalid
20       accept    all              1743   106937  ct state established  accept
30       accept    all                 6     1347  ct state related  accept
default  accept    all

...

---------------------------------
IPv4 Firewall "output filter"

Rule     Action    Protocol      Packets    Bytes  Conditions
-------  --------  ----------  ---------  -------  ----------------------------
10       drop      all                 0        0  ct state invalid
20       accept    all              3278   856246  ct state established  accept
30       accept    all                 0        0  ct state related  accept
default  accept    all
vyos@vyos:~$ show firewall statistics
Rulesets Statistics

...

---------------------------------
IPv4 Firewall "input filter"

Rule       Packets    Bytes  Action    Source     Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  ---------  -------------  -------------------  --------------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0      any                  any
20            1781   109701  accept    0.0.0.0/0  0.0.0.0/0      any                  any
30               6     1347  accept    0.0.0.0/0  0.0.0.0/0      any                  any
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

...

---------------------------------
IPv4 Firewall "output filter"

Rule       Packets    Bytes  Action    Source     Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  ---------  -------------  -------------------  --------------------
10               0        0  drop      0.0.0.0/0  0.0.0.0/0      any                  any
20            3338   872950  accept    0.0.0.0/0  0.0.0.0/0      any                  any
30               0        0  accept    0.0.0.0/0  0.0.0.0/0      any                  any
default          0        0  accept    0.0.0.0/0  0.0.0.0/0

There were 1-2 seconds between the commands being runned.

Apachez changed the task status from Needs testing to Open.Aug 31 2023, 10:47 AM

Resolved by: https://vyos.dev/T5564

  1. show firewall group works as expected:
vyos@vyos:~$ show firewall group 
Firewall Groups

Name        Type                References              Members
----------  ------------------  ----------------------  ---------------
DMZ         interface_group     ipv4-forward-filter-50  eth2
                                ipv6-forward-filter-50
LAN         interface_group     ipv4-forward-filter-60  eth3
                                ipv6-forward-filter-60
MGMT        interface_group     N/D                     eth0
WAN         interface_group     ipv4-forward-filter-40  eth1
                                ipv6-forward-filter-40
V6_DMZ      ipv6_network_group  N/D                     N/D
V6_LAN      ipv6_network_group  N/D                     N/D
V6_MGMT     ipv6_network_group  N/D                     N/D
V6_WAN      ipv6_network_group  N/D                     ::/0
V4_BOGONS   network_group       N/D                     0.0.0.0/8
                                                        10.0.0.0/8
                                                        100.64.0.0/10
                                                        127.0.0.0/8
                                                        169.254.0.0/16
                                                        172.16.0.0/12
                                                        192.0.0.0/24
                                                        192.0.2.0/24
                                                        192.168.0.0/16
                                                        198.18.0.0/15
                                                        198.51.100.0/24
                                                        203.0.113.0/24
                                                        224.0.0.0/4
                                                        240.0.0.0/4
V4_DMZ      network_group       N/D                     192.168.2.0/24
V4_LAN      network_group       N/D                     192.168.3.0/24
V4_MGMT     network_group       N/D                     192.168.56.0/24
V4_RFC1918  network_group       N/D                     10.0.0.0/8
                                                        172.16.0.0/12
                                                        192.168.0.0/16
V4_WAN      network_group       N/D                     0.0.0.0/0
                                                        192.168.1.0/24
  1. show firewall statistics packets and bytes counters works (except default-action aka last rule in each block):
vyos@vyos:~$ show firewall statistics 
...
---------------------------------
IPv4 Firewall "input filter"

Rule     Packets    Bytes    Action    Source       Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  -----------  -------------  -------------------  --------------------
10       12666      714457   accept    any          any            any                  any
20       0          0        accept    any          any            any                  any
30       0          0        drop      any          any            any                  any
999999   1          60       accept    127.0.0.0/8  any            lo                   any
default  N/A        N/A      accept    any          any            any                  any
...
---------------------------------
IPv4 Firewall "output filter"

Rule     Packets    Bytes    Action    Source    Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  --------  -------------  -------------------  --------------------
10       24287      5380968  accept    any       any            any                  any
20       0          0        accept    any       any            any                  any
30       0          0        drop      any       any            any                  any
999999   1          60       accept    any       127.0.0.0/8    any                  lo
default  N/A        N/A      accept    any       any            any                  any

Note, shouldnt the default action (last row in each block) have counters other than "N/A"?

  1. show firewall statistics shows correct IPv4 and IPv6 information:
vyos@vyos:~$ show firewall statistics
...
---------------------------------
IPv4 Firewall "input filter"

Rule     Packets    Bytes    Action    Source       Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  -----------  -------------  -------------------  --------------------
10       12666      714457   accept    any          any            any                  any
20       0          0        accept    any          any            any                  any
30       0          0        drop      any          any            any                  any
999999   1          60       accept    127.0.0.0/8  any            lo                   any
default  N/A        N/A      accept    any          any            any                  any
...
---------------------------------
IPv6 Firewall "input filter"

Rule     Packets    Bytes    Action    Source    Destination    Inbound-Interface    Outbound-interface
-------  ---------  -------  --------  --------  -------------  -------------------  --------------------
10       0          0        accept    any       any            any                  any
20       0          0        accept    any       any            any                  any
30       0          0        drop      any       any            any                  any
999999   0          0        accept    ::1/128   any            lo                   any
default  N/A        N/A      accept    any       any            any                  any

Still needs fixing:

See "note" for case 2 above.

Packets and bytes counters are missing for the default-action:

default  N/A        N/A      accept    any       any            any                  any

Checked with #netfilter irc-channel.

There is currently no way to get counters for default-action in nftables.

Only workaround is to setup a last rule in current filter/chain to mimicking the default-action.

I would propose that VyOS would do this automatically since default-action is defined in the VyOS firewall engine:

set firewall ipv4 forward filter default-action 'drop'

Since the manually allowed ruleid is up to 999999 such default created rule could be ruleid 1000000 similar to:

set firewall ipv4 forward filter rule 1000000 action 'drop'

This would also solve that packets are not properly being logged when hitting default-action.

For example having a default-action of "drop-log" would generate the following as last rule:

set firewall ipv4 forward filter rule 1000000 action 'drop'
set firewall ipv4 forward filter rule 1000000 log 'enable'

With the above "hidden" default-action last rule (ruleid 1000000) would be mapped to "default" as in the `show firewall statistics'.

That is the view of show firewall statistics would be the same but with working counters for packets and bytes.

100% agree. If this isn't too big of a hassle to implement, I would very much appreciate the approach/workaround of @Apachez, until nftables supports this feature ootb...

I'm not too big of a fan of having to implement "catch-all-rules" all the time, just because I'm not able to use certain features by default.. (Many vendors like Sophos are also having this problem..)

Related: T5507

Suggestion of "hidden" ruleset (visible when doing show firewall and show firewall statistics):

For below example "default-action: log-drop" is being used for all filters.

set firewall ipv4 forward filter rule 1999999 action 'drop'
set firewall ipv4 forward filter rule 1999999 log 'enable'

set firewall ipv4 input filter rule 1999998 action 'accept'
set firewall ipv4 input filter rule 1999998 inbound-interface interface-name 'lo'
set firewall ipv4 input filter rule 1999998 source address '127.0.0.0/8'
set firewall ipv4 input filter rule 1999999 action 'drop'
set firewall ipv4 input filter rule 1999999 log 'enable'

set firewall ipv4 output filter rule 1999998 action 'accept'
set firewall ipv4 output filter rule 1999998 destination address '127.0.0.0/8'
set firewall ipv4 output filter rule 1999998 outbound-interface interface-name 'lo'
set firewall ipv4 output filter rule 1999999 action 'drop'
set firewall ipv4 output filter rule 1999999 log 'enable'

set firewall ipv6 forward filter rule 1999999 action 'drop'
set firewall ipv6 forward filter rule 1999999 log 'enable'

set firewall ipv6 input filter rule 1999998 action 'accept'
set firewall ipv6 input filter rule 1999998 inbound-interface interface-name 'lo'
set firewall ipv6 input filter rule 1999998 source address '::1/128'
set firewall ipv6 input filter rule 1999999 action 'drop'
set firewall ipv6 input filter rule 1999999 log 'enable'

set firewall ipv6 output filter rule 1999998 action 'accept'
set firewall ipv6 output filter rule 1999998 destination address '::1/128'
set firewall ipv6 output filter rule 1999998 outbound-interface interface-name 'lo'
set firewall ipv6 output filter rule 1999999 action 'drop'
set firewall ipv6 output filter rule 1999999 log 'enable'

That is the custom ruleid range is 1-999999.

Where the "hidden" (made by VyOS) ruleid range is 1000000-1999999 where it starts with the last rule at 1999999 (aka "default-action") and the rule before that is 1999998 and so on.

Rule 1999999 is also in the show firewall and show firewall statistics renamed in output as "default".

The "hidden" rules 1000000-1999999 cannot be removed or changed (other than when changing "default-action").

The above would solve the bytes and packets counters for default-action along with including the needed "allow localhost to speak to itself".

n.fort changed the task status from Open to In progress.Nov 2 2023, 9:06 PM
n.fort claimed this task.