Page MenuHomeVyOS Platform

Disable conntrack by default
Closed, ResolvedPublicBUG

Description

Reported on the forum: https://forum.vyos.io/t/conntrack-is-enabled-by-default-on-1-4-rr/10586

Possibly missed during my firewall refactor, vyatta made use of FW_CONNTRACK and NAT_CONNTRACK chains to enable/disable conntrack depending if rules are found to match on state in firewall/nat modules.

Details

Difficulty level
Unknown (require assessment)
Version
1.4-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Event Timeline

sarthurdev triaged this task as High priority.
sarthurdev created this task.

I don't think this ever worked as intended: see T3275#103228, vyos-build PR 185, and T3821.

Bellow, there are some procedures that I used to unload the conntrack module in my environment.

1 - Create an empty config file of NFT Ruleset and save in /config/scripts/nft_empty_ruleset.conf

flush ruleset

table ip nat {
        chain VYOS_PRE_SNAT_HOOK {
                return
        }
}
table inet mangle {
        chain FORWARD {
                return
        }
}
table ip raw {
        chain VYOS_TCP_MSS {
                return
        }

        chain PREROUTING {
                return
        }

        chain OUTPUT {
                return
        }

        chain VYOS_CT_HELPER {
                return
        }

        chain VYOS_CT_IGNORE {
                return
        }

        chain VYOS_CT_TIMEOUT {
                return
        }

        chain VYOS_CT_PREROUTING_HOOK {
                return
        }

        chain VYOS_CT_OUTPUT_HOOK {
                return
        }

        chain FW_CONNTRACK {
                return
        }
}
table ip6 raw {
        chain VYOS_TCP_MSS {
                return
        }

        chain PREROUTING {
                return
        }

        chain OUTPUT {
                return
        }

        chain VYOS_CT_PREROUTING_HOOK {
                return
        }

        chain VYOS_CT_OUTPUT_HOOK {
                return
        }

        chain FW_CONNTRACK {
                return
        }
}

2 - Edit the file /config/scripts/vyos-postconfig-bootup.script and then add these lines in the end of file:

nft -f /config/scripts/nft_empty_ruleset.conf
sleep 15
rmmod nft_chain_nat
rmmod nf_nat
rmmod nft_ct
rmmod nfnetlink_cthelper
rmmod nf_conntrack_netlink
rmmod nf_conntrack
rmmod nf_defrag_ipv4
rmmod nf_defrag_ipv6

Explaining
The first line reload the config of the NFT, mading kill the connections states.
This process need to have time to conclusion, so we wait 15 seconds with the sleep command in the second line.
The next lines, must be in the suggestered order because they have some interdependencies that lock the unload command module "rmod" and cause failure.

3 - Save all files and do a reboot of the system.

After this procedures above, now I can do commits in the interfaces configs normally.
If I just make a flush in the NFT's rulesets these part of the system will broken because don't pass in some part of the validation's on the process of commit.

Yet about the before comment, when I make a flush in the NFT without maintain the empty structure this error occurs:

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your 
  business policy requires it)
- and include all the information presented below

Report time:      2023-03-30 20:10:05
Image version:    VyOS 1.4-rolling-202302060317
Release train:    current

Built by:         [email protected]
Built on:         Mon 06 Feb 2023 03:17 UTC
Build UUID:       c92374c9-13b4-442a-99fc-8d987b0fe5d8
Build commit ID:  2767ac346e4d1a

Architecture:     x86_64
Boot via:         installed image
System type:      bare metal

Hardware vendor:  HP
Hardware model:   ProLiant DL360 G6
Hardware S/N:     BRC028N0G6
Hardware UUID:    36343035-3533-5242-4330-32384e304736

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/interfaces-ethernet.py", line 220, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/interfaces-ethernet.py", line 209, in apply
    e.update(ethernet)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/ethernet.py", line 382, in update
    super().update(config)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 1482, in update
    self.set_tcp_ipv4_mss(value)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 606, in set_tcp_ipv4_mss
    self._cleanup_mss_rules('raw', self.ifname)
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py", line 588, in _cleanup_mss_rules
    results = self._cmd(f'nft -a list chain {table} VYOS_TCP_MSS').split("\n")
  File "/usr/lib/python3/dist-packages/vyos/ifconfig/control.py", line 52, in _cmd
    return cmd(command, self.debug)
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 161, in cmd
    raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: nft -a list chain raw VYOS_TCP_MSS
returned: 
exit code: 1

noteworthy:
cmd 'nft -c delete element inet vrf_zones ct_iface_map { "eth3" }'
returned (out):

returned (err):
Error: No such file or directory
delete element inet vrf_zones ct_iface_map { eth3 }
                    ^^^^^^^^^
cmd 'nft -a list chain raw VYOS_TCP_MSS'
returned (out):

returned (err):
Error: No such file or directory
list chain raw VYOS_TCP_MSS
           ^^^

[[interfaces ethernet eth3]] failed
Commit failed

I think one of the problems is that all tables are generated even if there are no rules in it.

So, the if the ip nat table is generated, the nat modules are loaded which also load conntrack etc...

I discovered that conntrack are enabled by default in two files:

/usr/libexec/vyos/init/vyos-router line 309

# Init firewall
 nfct helper add rpc inet tcp
 nfct helper add rpc inet udp
 nfct helper add tns inet tcp
 nft -f /usr/share/vyos/vyos-firewall-init.conf || log_failure_msg "could not initiate firewall rules"

And in /usr/share/vyos/vyos-firewall-init.conf Line 3

# Required by wanloadbalance
table ip nat {
    chain VYOS_PRE_SNAT_HOOK {
        type nat hook postrouting priority 99; policy accept;
        return
    }
}

and Line 40

ct helper rpc_tcp {
    type "rpc" protocol tcp;
}

ct helper rpc_udp {
    type "rpc" protocol udp;
}

ct helper tns_tcp {
    type "tns" protocol tcp;
}

Removing NAT section and CT Helpers from /usr/share/vyos/vyos-firewall-init.conf seems to resolve the problem of conntrack enabled by default.
I can't test the wanloadbalance or the statefull firewall after these changes, because in my enviroment these functions aren't used.

syncer lowered the priority of this task from High to Low.Jul 11 2023, 12:27 PM
syncer added a subscriber: syncer.

@sdev, so what is the state of this?

sarthurdev changed the task status from Open to In progress.Aug 26 2023, 9:35 PM
sarthurdev moved this task from Need Triage to In Progress on the VyOS 1.4 Sagitta board.

How come these helpers (pointed out by @saintclairpcarvalho but also )https://vyos.dev/T5479) are always enabled?

I just tested this with a firewall config with no connection tracking config enabled, still the conntrack modules are loaded and used.

Just confirming this is still an issue, as mentioned in slack, and is really adding a lot of cpu load to pure routers

vyos@mtl2-core-fw2:~$ show configuration commands | egrep -i '\ (firewa|nat)'
set service snmp community firewalls authorization 'ro'
set service snmp community firewalls client 'xx.xx.xxx.xxx'
vyos@mtl2-core-fw2:~$ show conntrack table ipv4 | wc -l
3079
vyos@mtl2-core-fw2:~$

It is apparently being caused by these ct entries

table ip raw {
        ct helper rpc_tcp {
                type "rpc" protocol tcp
                l3proto ip
        }

        ct helper rpc_udp {
                type "rpc" protocol udp
                l3proto ip
        }

        ct helper tns_tcp {
                type "tns" protocol tcp
                l3proto ip
        }

Various other tickets have suggested patching vyos to remove these, and that may be the only way forward until the other tickets are resolved, as this is making anything newer than august-ish require far more resources than they should!

Also adding these lines as to "completely ignore conntrack for all traffic" doesnt seem to help:

set system conntrack ignore ipv4 rule 10 inbound-interface any
set system conntrack ignore ipv6 rule 10 inbound-interface any

Doing the above and rebooting I can still see tracking in action through "conntrack -L".

Verified using VyOS 1.5-rolling-202310020022.

Try this

delete  system conntrack ignore
set system conntrack ignore ipv4 rule 10 source address 0.0.0.0/0
sarthurdev moved this task from In Progress to Finished on the VyOS 1.4 Sagitta board.

Closing this as resolved, please re-open if any further issues found.

runar added a subscriber: runar.

Reopened on @Apachez request

Not a regression as far as this task is concerned. Will update T6073

dmbaturin renamed this task from Conntrack enabled by default to Disable conntrack by default.Sat, May 11, 4:49 PM
dmbaturin changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.