Page MenuHomeVyOS Platform

Wan Load Balancing - Error on firewall NAT rules
Closed, ResolvedPublicBUG

Description

Tested on VyOS 1.4-rolling-202201100317

Commands for Wan Load Balancing:

# Load balancing config
set load-balancing wan interface-health eth0 nexthop '10.0.0.1'
set load-balancing wan interface-health eth1 nexthop '10.1.1.1'
set load-balancing wan rule 10 inbound-interface 'eth3.100'
set load-balancing wan rule 10 interface eth0
set load-balancing wan rule 10 interface eth1

This results on next nat rules:

vyos@vyos# sudo nft list table ip nat
table ip nat {
	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
		counter packets 215 bytes 18124 jump VYOS_PRE_DNAT_HOOK
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		counter packets 273 bytes 21364 jump VYOS_PRE_SNAT_HOOK
	}

	chain VYOS_PRE_DNAT_HOOK {
		return
	}

	chain VYOS_PRE_SNAT_HOOK {
		return
	}

	chain WANLOADBALANCE {
		ct mark 0xc9 counter packets 0 bytes 0 snat to 10.0.0.2
		ct mark 0xca counter packets 0 bytes 0 snat to 10.1.1.2
	}
}

There's a missing rule in chain VYOS_PRE_SNAT_HOOK that jumps to WANLOADBALANCE. So, no source nat occurs at all.

Same config con VyOS 1.3, give us next nat rules:

vyos@Customer:~$ sudo nft list table ip nat
table ip nat {
	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
		counter packets 32 bytes 2784 jump VYATTA_PRE_DNAT_HOOK
	}

	chain INPUT {
		type nat hook input priority 100; policy accept;
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		counter packets 23 bytes 1956 jump VYATTA_PRE_SNAT_HOOK
	}

	chain OUTPUT {
		type nat hook output priority -100; policy accept;
	}

	chain VYATTA_PRE_DNAT_HOOK {
		counter packets 32 bytes 2784 return
	}

	chain VYATTA_PRE_SNAT_HOOK {
		counter packets 23 bytes 1956 jump WANLOADBALANCE
		counter packets 0 bytes 0 return
	}

	chain WANLOADBALANCE {
		ct mark 0xc9 counter packets 14 bytes 1192 snat to 10.0.0.2
		ct mark 0xca counter packets 9 bytes 764 snat to 10.1.1.2
	}
}

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202303290849
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Forgot that my PR for WLB was still a draft. That the jump does seem to be created properly with this PR in place.

PR: https://github.com/vyos/vyatta-wanloadbalance/pull/12

Viacheslav changed the task status from Open to In progress.Jan 12 2022, 4:37 PM
Viacheslav changed the task status from In progress to Needs testing.
Viacheslav assigned this task to sarthurdev.

Tested and working as expected on VyOS 1.4-rolling-202201150317

Re Opening this task, since same error is present again

vyos@vyos:~$ sudo nft list table ip nat
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
        chain VYOS_PRE_SNAT_HOOK {
                type nat hook postrouting priority srcnat - 1; policy accept;
                return
        }

        chain WANLOADBALANCE {
                ct mark 0xc9 counter packets 0 bytes 0 snat to 217.146.108.9
                ct mark 0xca counter packets 0 bytes 0 snat to 10.55.55.63
        }
}
vyos@vyos:~$ show ver | grep Ver
Version:          VyOS 1.4-rolling-202303290849
vyos@vyos:~$

Also on version 1.4-rolling-202303170317 it's not working.

Last version I proved it worked properly: 1.4-rolling-202302080317

n.fort changed Version from VyOS 1.4-rolling-202201100317 to VyOS 1.4-rolling-202303290849.

The possible reason https://github.com/vyos/vyatta-wanloadbalance/blob/02f9e8fbee873f9ca1111a69761546c758001b24/src/lbdecision.cc#L116

vyos@r14# sudo nft list table ip nat
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
	chain VYOS_PRE_SNAT_HOOK {
		type nat hook postrouting priority srcnat - 1; policy accept;
	}

	chain WANLOADBALANCE {
		ct mark 0xc9 counter packets 0 bytes 0 snat to 192.168.122.14
	}
}
[edit]
vyos@r14# 
[edit]
vyos@r14# sudo iptables-nft -t nat -I VYOS_PRE_SNAT_HOOK 1 -j WANLOADBALANCE
iptables: No chain/target/match by that name.
[edit]
vyos@r14#

The same thing to delete https://github.com/vyos/vyatta-wanloadbalance/blob/02f9e8fbee873f9ca1111a69761546c758001b24/src/lbdecision.cc#L115

vyos@r14:~$ sudo nft list table ip nat
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
	chain VYOS_PRE_SNAT_HOOK {
		type nat hook postrouting priority srcnat - 1; policy accept;
		counter packets 1 bytes 84 jump WANLOADBALANCE
		return
	}

	chain WANLOADBALANCE {
		ct mark 0xc9 counter packets 1 bytes 84 snat to 192.168.122.14
	}
}
vyos@r14:~$ 
vyos@r14:~$ 
vyos@r14:~$ sudo iptables-nft -t nat -D VYOS_PRE_SNAT_HOOK -j WANLOADBALANCE
iptables: Bad rule (does a matching rule exist in that chain?).
vyos@r14:~$ 
vyos@r14:~$

So it was some changes in kernel/nft version, and iptables-nft already could not work correctly with those rules.

vyos@r14:~$ show version kernel 
6.1.21-amd64-vyos
vyos@r14:~$ 
vyos@r14:~$ show version all | match nft
ii  libnftables1:amd64                   1.0.6-2                          amd64        Netfilter nftables high level userspace API library
ii  libnftnl11:amd64                     1.2.3-1                          amd64        Netfilter nftables userspace API library
ii  miniupnpd-nftables                   2.3.1-1                          amd64        UPnP and NAT-PMP daemon for gateway routers - nftables backend
ii  nftables                             1.0.6-2                          amd64        Program to control packet filtering rules by Netfilter project
vyos@r14:~$

It shoulb be overwritten to nftables. I don't see any easy workaround

Viacheslav added a subscriber: Viacheslav.

There also additional bugs relates iptables-nft as chain WANLOADBALANCE_OUT is not exist
https://github.com/vyos/vyatta-wanloadbalance/blob/70ee1319c20e083ab407d8a11faa44c74d05f084/src/lbdecision.cc#L312

vyos@r14# sudo nft list table ip mangle
# Warning: table ip mangle is managed by iptables-nft, do not touch!
table ip mangle {
	chain PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
		counter packets 851 bytes 179376 jump WANLOADBALANCE_PRE
	}

	chain WANLOADBALANCE_PRE {
	}

	chain ISP_veth1 {
		counter packets 0 bytes 0 ct mark set 0xc9
		counter packets 0 bytes 0 meta mark set 0xc9
		counter packets 0 bytes 0 accept
	}

	chain ISP_veth2 {
		counter packets 0 bytes 0 ct mark set 0xca
		counter packets 0 bytes 0 meta mark set 0xca
		counter packets 0 bytes 0 accept
	}
}
[edit]
vyos@r14# 
[edit]
vyos@r14# sudo iptables-nft -t mangle -A WANLOADBALANCE_OUT -m mark ! --mark 0 -j ACCEPT
iptables: No chain/target/match by that name.
[edit]
vyos@r14#
Viacheslav changed the task status from Confirmed to Needs testing.Mar 31 2023, 8:20 AM
Viacheslav claimed this task.
Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.