@maznu I think this will be the default only after BGP switches to a new XML/python format.
https://phabricator.vyos.net/T2174
- Feed Queries
- All Stories
- Search
- Feed Search
- Transactions
- Transaction Logs
All Stories
May 6 2020
The good news is that this can be fixed with:
To reproduce it in a test lab.
R1 Originate 1000 prefixes
R2 import 1000 and export 40 prefixes to R3
R3 prefix-limit 40 prefixes.
Tested it right now, set interfaces bridge br0 member interface eth3.4 worked fine with latest current vyos-1x, so the next ISO should definitely have this fixed.
See T2427, the fix for that was merged today and will be in the next ISO build. I'll test that new ISO too to see if I catch any migration errors (I tested it just on a test VM without upgrading and I didn't see any VLAN errors, several configured VLANs seemed to work)
Edit: I see the error is for eth vif in a bridge, I didn't test that, just vif on ethernet and eth in a bridge. I'll test it today.
Just wanted to add an update on this, as of today May-6-2020
It appears you are sending a redundant 'set' within the path itself.
May 5 2020
ack - I will change this to make sure it is safe !
Why do we need to remove all addresses from the interface when it is disabled? That causes a commit error on dhcp:
vyos@vyos# set service dhcp-server disable [edit] vyos@vyos# commit [edit] vyos@vyos# delet service dhcp-server disable [edit] vyos@vyos# commit [ service dhcp-server ] DHCP server configuration error! None of configured DHCP subnets does not have appropriate primary IP address on any broadcast interface.
@thomas-mangin there was a bug in PR #383 that prevented bond interfaces from coming up - it thought the members had addresses assigned, but they didn't. Something in my PR#384 fixed it - I'm not exactly sure what - probably assigning the intf['intf'] in get_intf_from_dict or fixing the _basename function. There is still a bug with IPv6 link-locals that worked fine before - I'll see if it's easy to track down, if not I'll open a new task for it.
It seems to be reading the output of something that thinks it's talking to a VT100 terminal.
Example of output containing control characters.
May 4 2020
@Viacheslav I confirmed the bug you have with assigning 'fe80::/64' when it shouldn't. It appered after https://github.com/vyos/vyos-1x/pull/383 - I'll investigate once my other PR gets merged and open a task for it. (note that it should add the fe80::(eui-64)/64 based on mac address by default and no-default-link-local did disable it before PR#383)
This is a result of a regression in unionfs-fuse, fixed in commit 93c2f9b, but not contained in the debian buster package v1.0. The naming convention for directories was changed from unionfs to unionfs-fuse, and then reverted, forgetting to revert the meta directory name. Consequently, bad things can happen for packages that had defined the meta directory consistent with the pre-v1.0 naming, for example, vyatta-cfg. Two fixes have been tested with initial success: (1) patch unionfs-fuse (2) change the meta directory name in vyatta-cfg. Either fix passes initial tests, although closer scrutiny is warranted, as it is a critical issue. The downside of (1) is the need to maintain an upstream or patched version of unionfs-fuse; the downside of (2) is that we are accommodating a regression, which will break again when/if the upstream package is available. Likely the quickest solution is preferred, as this issue is implicated in several recent bug reports.
Need to check again with 1.3, as may be solved by: https://phabricator.vyos.net/T1291
@Gislaved, I have just tested it on the KVM (proxmox) with VirtIO and I coudn't reproduce it. It works with exactly the same defects as I mentioned in my previous message. Tested on the both LTS and 1.3-rolling-202005031216.
@brussell @Viacheslav @dmbaturin @zsdc
FRR devs agree to change the default behavior in 7.4
so we need to prepare change on our side and also document that change in FRR docs
Would love to see this resolved — a large (but reasonable) configuration doing IRR-based filtering from BGP peers took 9 hours to boot up.
It is already in the manual. Not at the Troubleshooting section, but at Configuration Overview.
https://docs.vyos.io/en/latest/configuration-overview.html#comment
That was done.
In T2404#62721, @jjakob wrote:@c-po is it possible to detect the exact networking setup used on the host? If not, I don't think that'll do any good, because the hypervisor is just that - a hypervisor. There can be many different types of networking. Like in this case, Openstack's SDN supports a lower MTU than a bridged or NATed connection with the same hypervisor would. The only reliable way is to detect the max MTU in some way (the kernel/device driver must know this, the question how can we read it) - the first step is to not touch the default MTU if the user doesn't set it.
@c-po is it possible to detect the exact networking setup used on the host? If not, I don't think that'll do any good, because the hypervisor is just that - a hypervisor. There can be many different types of networking. Like in this case, Openstack's SDN supports a lower MTU than a bridged or NATed connection with the same hypervisor would. The only reliable way is to detect the max MTU in some way (the kernel/device driver must know this, the question how can we read it) - the first step is to not touch the default MTU if the user doesn't set it.
Wireless AP completely rewritten. If issue reappears please reopen or file a new bug.
Forum says issue is fixed. Also WWAN underwent a major rewrite - should be all good now.
@MikeLupe Are you using a custom dyndns provider/script? It ifs compatible with the dyndns2 "API" this is already possible using the "custom" dns dynamic service.
This has been completely rewritte in Python/XML. Please reopen if this is still a problem - and if so what do you think should be the "real" rendered configuration.
I suggest adding a "validate" function to vyos.validate. We should determine the hypervisor used and if a setting is allowed on that hypervisor or not. Thus we can throw an appropriate erorr message by raise ConfigError() in the Python file changing the interface setting. This would be the cleanest way then the "false" setting never reaches the configuration backend at all.
So the minimum MTU for IPv6 is 1280. Setting ipv6 address no-default-link-local will probably fix this.
@jjakob I don't set any ip address on eth1
@Viacheslav do you have IPv6 disabled per chance? What's the output of ip addr dev eth1 after the failed commits? What's the full eth1 config?
The odd thing to me here is that it shouldn't ever be adding just fe80::/64, it should get the mac (using self.get_mac()) and create a single EUI64 address. The longer fe80:: address looks as it should be.
Confirmed with the simplest 'login as vyos; set interfaces ... description; login as other; delete interfaces .. description; login as vyos; set ...". T2203 is likely a result of this issue. Will reassign to me.
Fix path to mac-address node. PR https://github.com/vyos/vyos-1x/pull/392
In T1230#62672, @Viacheslav wrote:@csalcedo maybe you use large port-ranges https://phabricator.vyos.net/T2189
We don't do any firewalling — we have lots of prefix-lists for filtering eBGP sessions. Right now we're looking at a router that's taken more than 1h20minutes to boot up — and it is still not finished — on modern Xeon CPUs. That's doubled in length since adding a prefix-list of around 5000 entries (roughly double the total number of prefix-list entries as before).
I definitely am not using large port-ranges. A pretty standard setup using a zone-based firewall.
@csalcedo maybe you use large port-ranges https://phabricator.vyos.net/T2189
sever@vyos-roll# set interfaces ethernet eth1 description testeth [edit] sever@vyos-roll# commit [edit] sever@vyos-roll# set interfaces ethernet eth1 mtu 1200 [edit] sever@vyos-roll# commit [ interfaces ethernet eth1 ]
All works on the rolling 1.3-rolling-202005030117
vyos@vyos:~$ ping 100.64.0.1 flood count 300 PING 100.64.0.1 (100.64.0.1) 56(84) bytes of data.
Another affected by this! Several devices with more than 20K config lines cause our automation scripts to take really really long to complete, and also the devices take much time to boot.
It would be great having this fixed in 1.3 :-)
Also affected by this. Reboots take almost 10 minutes before the device is usable. Commits take a long time as well.
May 3 2020
This issue seems to still be reproducing on static mappings with 1.2.5. I have noticed that the issue is fixed for non static mapping DHCP clients but all my clients who are getting assigned their IPs via static mappings are still getting their hostnames prefixed with LAN_ (LAN being my shared network name).
I think not setting the MTU at all if the user doesn't configure it is the right way to do it. It would be nice if we could detect an interface's max MTU but I couldn't find a way to do it either (I didn't really search that hard other than looking under /sys/net/{ifname} and /proc/sys/net).
@thomas-mangin Which other dependent interfaces do you mean? All other interfaces that depend on the ethernet interface should leave the MTU untouched too, let the OS choose the defaults.
My suggestion:
- if someone set a higher MTU than OS support, commit fail will be ok and that's OK, each vyos admin should know his OS limitations
Some code should also be added to detect the parent MTU size and make sure a clan MTU is smaller than the parent
Other ubuntu servers inside same openstack max mtu OS leave at 1450 too
In T2404#62637, @robertoberto wrote:I tried to google to find out how to discover max mtu an interface accept but not found.
I guess limitation is because we are using OpenStack KVM or the other guy is using VMware and they use some bytes to their SDN
I tried to google to find out how to discover max mtu an interface accept but not found.
The mtu provided are reasonable and should not fail IMHO. I need to understand why Linux does not want to honour it.. hw limitation or other. If it is a limitation then we should detect it and report it to the user
That would only hide the problem. The MTU bug would still be here.
This should probably be:
if eth['mtu']:
e.set_mtu(eth['mtu'])Then if the user doesn't set a mtu, we don't try to change it. If the config set mtu is wrong the commit should fail, but it should get verified in the verify section by a max mtu read from the interface (if that's possible), or leaving it as it is now. It shouldn't silently (or even loudly) continue as that would make the failure undetectable at boot.
In T2404#62624, @thomas-mangin wrote:An issue to change the MTU should not fail the whole interface change but just this one but the code does not not allow this easily. I will try to work on this.
An issue to change the MTU should not fail the whole interface change but just this one but the code does not not allow this easily. I will try to work on this.
vyos@vyos:~$ show version
Version: VyOS 1.3-rolling-202004230117
Release Train: equuleus
In T2404#62621, @robertoberto wrote:Here:
root@vyos:~# cat /sys/class/net/eth0/mtu
1450
root@vyos:~# echo 1500 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1460 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1440 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1441 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1442 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1443 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1449 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1450 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1460 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1455 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1453 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1452 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1451 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~#
root@vyos:~# cat /sys/class/net/eth0/mtu
1450
root@vyos:~# echo 1500 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1460 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1440 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1441 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1442 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1443 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1449 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1450 > /sys/class/net/eth0/mtu
root@vyos:~# echo 1460 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1455 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1453 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1452 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~# echo 1451 > /sys/class/net/eth0/mtu
-bash: echo: write error: Invalid argument
root@vyos:~#
Sure it can (was in the initial ifconfig.py rewrite IIRC)
The reason seems to be that when the kernel can not change the MTU when writing to the sysfs, it will raise a OSError with the reason why.