I'm thinking about trying to run either tayga or jool as a docker container inside of VyOS. Has anyone tried something like this? If I wrote a guide on how to do this I wonder if it would be an OK temporary solution until it's integrated into VyOS.

Isn't anyone implementing this feature right now?

@jack9603301 I tried building jool into the VyOS image but because their deb packages uses DKMS which is not compatible with VyOS so somebody has to create a VyOS specific deb package.

Jool can handle 2 times as many packets.
https://link.springer.com/article/10.1007/s11235-020-00681-x

@dmbaturin @artooro Come on, remember not to forget NAT46

They are a pair

Regarding the command line, I recommend the following:

set nat64
set nat46

Because I have completed the implementation of NAT66, the command line is as follows:

set nat66

Although it cannot be merged into the main line for the time being

I would suggest going with tayga if this feature is planned to be implemented.

I tried jool in my testing environment.
Jool is feature-rich and very performant, however it is tricky to get the firewall setup correct because jool itself is a kernel module that extends the netfilter subsystem.
When jool is enabled, normal firewall filtering on INPUT and FORWARD is skipped. So NAT64 traffic will always be permitted.
In order to get NAT64 traffic going to the firewall, you have to run jool in a separate network namespace and filter on the interfaces that connects the network namespaces.
This will add a lot of complexity to VyOS to maintain and such a complicated iptables/nftables configuration and such an out-of-tree kernel module.

However for tayga, things will be much more easier. Tayga just adds a new tap interface to the host network stack. We can reuse the existing VyOS firewall commands to filter traffic on that interface.

I've been running TAYGA inside VyOS with absolutely no problems for the last several years.

In my own builds, I just added the "tayga" package so that it would automatically be there without having to add it manually each time I upgrade.

JOOL looks nice but is a much bigger effort to integrate.

TAYGA could be integrated NOW.

Please consider adding TAYGA now and then if someone can take the time to integrate JOOL then that is fine as well, as long as it has the same capabilities as TAYGA.

Prototype PR using Tayga: https://github.com/vyos/vyos-1x/pull/1408

While i like the inclusion of NAT64 inside vyos (And the effort vfreex has made), i believe that tayga is not the way to go, it was last updated on 2010-12-12 according to the readme in it. Jool on the other hand has a bigger throughput being kernel module. The only issue i believe is the module compilation cause configuration is quite easy.

@aaliddell I am not too concerned about tayga's maintenance. It have been proved to work well for years, and the package is already a part of the official repository of debian. Actually debian's tayga package includes a few patches: https://salsa.debian.org/debian/tayga/-/tree/debian/master/debian/patches

The only downside of tayga is its performance. I would like to see Jool to be included in VyOS however that will be a muck bigger effort to integrate. At least kernel module integration and firewall filtering for jool is complex. Let me know if you have more insights more about Jool.

Just adding here that Jool.mx is no longer maintained.

"The project’s development has slowed down to essential maintenance. Bugfixing and support will remain active, but there will be no new features in the foreseeable future." - Source: https://nicmx.github.io/Jool/en/index.html

However latest release has been on Mar 21, 2022.

It sounds like people prefer jool, and in my personal opinion, if there is a package that is still active up to now, like jool, then jool is probably better. But whatever it is, it's foreign to me

Jool is still being maintained for bugfixes etc. and it has all the features we're looking for, then it sounds fairly ideal. If no new features are being added to it, it's less likely to break in future releases too

Jool package was added to the vyos-build repo in https://github.com/vyos/vyos-build/commit/d9f711f500ea21288a50f54640dff833cd1da153

Great, maybe we can use jool to build it!

In T160#147889, @frebib wrote:

Jool package was added to the vyos-build repo in https://github.com/vyos/vyos-build/commit/d9f711f500ea21288a50f54640dff833cd1da153

I started working on a jool nat64 implementation at https://github.com/vyos/vyos-1x/pull/1993 for anyone that would like to try it out. Not everything is implemented yet, but I have been running this for the past few weeks with no issues. Any comments or feedback on the UX would be appreciated

PR https://github.com/vyos/vyos-1x/pull/2573

set interfaces ethernet eth0 address '192.168.122.14/24'
set interfaces ethernet eth0 address '192.168.122.10/24'
set interfaces ethernet eth2 address '2001:db8::1/64'

set nat64 source rule 100 source prefix '64:ff9b::/96'
set nat64 source rule 100 translation pool 10 address '192.168.122.10'
set nat64 source rule 100 translation pool 10 port '1-65535'

Is it possible to add a feature of setting the translation pool address to follow an interface IP address similar to nat44's source masquerade for those with dynamic public IPv4 addresses?

Not sure if Jool supports it.
As I understand, it is better to have a separate address for translations, otherwise, you should define ports for 64 translations. If you set 1-65535 for the "main" address, you will lose connections and can't use those ports for the system (they will be available only for translations)
Jool uses its own bib table instead of conntrack to manage nat mappings.
So you should add a separate pool only for translations.

Is it possible to add support for policy routing?

I want to be able to route some ipv6 source addresses/lan computers to a specific pool, so I can send their outgoing traffic to a specific ipv4 address on which I do source/ policy routing.

I have this working today on a debian box. With the conf below I say that <IPV6-COMPUTER1/128> should send all its traffic via 172.21.8.203, and <IPV6-NETWORK2/64> sends all its traffic via 172.21.8.202
The important bits is to be able to set MARK/fwmark.

jool -i default pool4 add --tcp 172.21.8.202 1-65535 --mark 4
jool -i default pool4 add --udp 172.21.8.202 1-65535 --mark 4
jool -i default pool4 add --icmp 172.21.8.202 1-65535 --mark 4

jool -i default pool4 add --tcp 172.21.8.203 1-65535 --mark 5
jool -i default pool4 add --udp 172.21.8.203 1-65535 --mark 5
jool -i default pool4 add --icmp 172.21.8.203 1-65535 --mark 5



ip6tables -t mangle -A PREROUTING --destination 64:ff9b::/96 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p tcp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p udp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p icmp -j JOOL --instance default

iptables  -t mangle -A PREROUTING --destination 172.21.8.203 -p tcp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.203 -p udp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.203 -p icmp -j JOOL --instance default


ip6tables -t mangle -I PREROUTING -s <IPV6-COMPUTER1/128> -j MARK --set-mark 4
ip6tables -t mangle -I PREROUTING -s <IPV6-NETWORK2/64>  -j MARK --set-mark 5

@danielpo marking IPv6 packet is possible

set policy route6 PBR6 interface 'eth1'
set policy route6 PBR6 rule 100 set mark '4'
set policy route6 PBR6 rule 100 source address '2001:db8::/64'

That translated to nft:

table ip6 vyos_mangle {
	chain VYOS_PBR6_PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
		iifname "eth1" counter packets 0 bytes 0 jump VYOS_PBR6_UD_PBR6
	}

	chain VYOS_PBR6_POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
	}

	chain VYOS_PBR6_UD_PBR6 {
		ip6 saddr 2001:db8::/64 counter packets 0 bytes 0 meta mark set 0x00000004 return comment "ipv6-route6-PBR6-100"
	}
}

What needs to be added is:

set nat64 source rule 100 source mark 4

or

set nat64 source rule 100 match mark 4

And it will be easy.

But this logic is not implemented for CLI

ip6tables -t mangle -A PREROUTING --destination 64:ff9b::/96 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p tcp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p udp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p icmp -j JOOL --instance default

It probably should be implemented in the firewall like this

set firewall ipv6 forward mangle rule 10 action jump

@n.fort Let us know if you have some ideas for mangle/prerouting CLI :)

Another thing that we don't use jool iptables plugin, but use netfilter if it makes sense.

I stil haven't tried nat64, but quick config example, for nat64 for single ipv6 address is not allowed by our cli:

vyos@Upstream# compare commands 

set nat64 source rule 10 source prefix '2001:db8::1/128'
set nat64 source rule 10 translation pool 5 address '203.0.113.55'

[edit]
vyos@Upstream# commit
[ nat64 ]
Source NAT64 rule 10 source prefix must be /96

[[nat64]] failed
Commit failed
[edit]
vyos@Upstream#

Regarding mangle, I would love to see mangle priority in ipv4 and ipv6. So far it's implemented in policy route|route6, with several limitations.

if you are running the netfilter implementation, I think the iptables mangle is handled automatically.(I verified this by changing my Jool to netfilter and removed these lines below:

ip6tables -t mangle -A PREROUTING --destination 64:ff9b::/96 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p tcp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p udp --dport 1:65535 -j JOOL --instance default
iptables  -t mangle -A PREROUTING --destination 172.21.8.202 -p icmp -j JOOL --instance default

So the only thing needed is:

set nat64 source rule 100 match mark 4

PR https://github.com/vyos/vyos-1x/pull/2677

set interfaces ethernet eth0 address '192.168.122.14/24'
set interfaces ethernet eth0 address '192.168.122.10/24'
set interfaces ethernet eth2 address '2001:db8::1/64'

set nat64 source rule 100 match mark '23'
set nat64 source rule 100 source prefix '64:ff9b::/96'
set nat64 source rule 100 translation pool 10 address '192.168.122.10'
set nat64 source rule 100 translation pool 10 port '1-65535'

test

vyos@r4# sudo jool -i instance-100 pool4 display --tcp
+------------+-------+--------------------+-----------------+-------------+
|       Mark | Proto |     Max iterations |         Address |       Ports |
+------------+-------+--------------------+-----------------+-------------+
|         23 |   TCP |       1024 ( auto) |  192.168.122.10 |     1-65535 |
+------------+-------+--------------------+-----------------+-------------+
[edit]
vyos@r4#

@danielpo could you check it?

That's great!
I tried running make interface_definitions in vyos-1x and copy in the changed files into my vyos install, but the files kept disappearing when rebooting, and couldn't figure out how to build an iso with your vyos 1-x repo.

So I wil try it when there is a vyos rolling iso available.

Latest 1.5 rolling ISO has it

Actually, it doesn't! :(
{F4050258}

In T160#168646, @danielpo wrote:

That's great!
I tried running make interface_definitions in vyos-1x and copy in the changed files into my vyos install, but the files kept disappearing when rebooting, and couldn't figure out how to build an iso with your vyos 1-x repo.

So I wil try it when there is a vyos rolling iso available.

@danielpo You can build .deb from vyos-1x and install the package

1. do changes XML/python in vyos-1x
2. Buid package sudo debuild -us -uc -b
3. Send generated vyos-1xxxx.deb to the VyOS node
4. On the vyos-node sudo dpkg -i *.deb && rm *.deb
5. Restart the service sudo systemctl restart vyos-configd or reboot the instance

I've tinkered with this a bit , but a problem I have is that I need multiple ip addresses on the outgoing interface(s). I understand that Jool takes all ports, so It's needed for everything to work properly.
On my own WAN interface, this is not a problem, I can just assign another ip from my ISP.

But this doesn't work if I want to use a VPN service I don't control myself, via nat64, I only get one ip address from the VPN.
When running Jool on virtual machine behind vyos, I just use NAT:ed local addresses for the outgoing ip:s, but this doesn't work then the Jool traffic is originating from vyos itself.

Not sure if this is even possible to fix..?

It is more of a question for Jool.
Or you can use a range of required ports that aren't used by the system

Since it has been implemented, can also global options be implemented?

https://nicmx.github.io/Jool/en/usr-flags-global.html

Thing like tcp timeout and icmp timeout being configurable should be available