User Details
- User Since
- Jan 18 2019, 4:03 PM (245 w, 1 d)
Mon, Sep 18
I haven't tried anything else since I rebooted back into 1.4, but I did try sudo su - which gave the same error.
I think this broke a whole lot of things for RADIUS users (these work fine in 1.4-rolling-202308040317, but are broken in 1.5-rolling-202309170024):
Jul 11 2023
Why is this closed? If you don't want the functionality, fine, but don't leave broken functionality in the installer....at least take it out so you're not confusing your users when it doesn't work.
Jul 9 2023
@Viacheslav So I figured out what's causing it..it looks like for some reason my commit-archive configuration on the secondary side (which works fine normally) is causing the hanging. As soon as I remove the set system config-management commit-archive on the secondary side, everything starts working fine, even with my full firewall configuration. Has this been tested at all with commit-archive? Could there be some sort of bug happening with it? Here's the section of the secondary side config, for reference:
[system config-management]
- commit-archive {
- location "sftp://<someUser>:<somePass>@stor01a-rh9.int.trae32566.org/int/cr01b-vyos"
- source-address "fd52:d62e:8011:fffe::3"
- }Yeah I tried increasing the timeout to the maximum (300) and it still timed out, but I'll try config-sync mode 'set' I guess. The config is fairly large; 549 lines of just set firewall.
@Viacheslav I'm not sure why, but it appears that after doing this, there is high CPU usage on the secondary side, and eventually it stops responding entirely (bgp sessions go down, no response to anything via icmp) and has to be hard reset; it won't even respond to a console login attempt:
@Viacheslav I think that fixed it...sorta. It looks like now it does sync successfully, though it appears to time out after awhile for some reason:
trae@cr01a-vyos:~$ sudo nano -c +140 /run/scripts/commit/post-hooks.d/vyos_config_sync trae@cr01a-vyos:~$ sudo systemctl restart vyos-configd trae@cr01a-vyos:~$ configure [edit] trae@cr01a-vyos# set firewall name INT_TO_LOCAL rule 80 destination address 192.168.253.2-192.168.253.3 [edit] trae@cr01a-vyos# commit INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=cr01b-vyos.int.rtr.trae32566.org
Same version on both, 1.4-rolling-202307070317. Also, if you can disable 2 factor on my Slack account ([email protected]) we can talk in Slack about this (lost my 2 factor app / backup codes).
@Viacheslav
@Viacheslav Thanks for all the work on this, I'm glad to see it moving along! This doesn't appear to work for me on 1.4-rolling-202307070317; I've configured it for both firewall and NAT and it appears to not be getting triggered (though I've only tried firewall changes so far). Here's the primary side (cr01a-vyos.int) config:
trae@cr01a-vyos# show service config-sync
mode load
secondary {
address cr01b-vyos.int.rtr.trae32566.org
key <MyKey>
}
section nat
section firewallFeb 13 2023
Nov 22 2022
This breaks a perfectly valid use case which I utilize regularly: using IPv4 + IPv6 peers with the same public key. Why would I want to create multiple keys for the exact same devices going over IPv4 and IPv6? If you want to include a warning, fine, but don't limit functionality based on someone's interpretation of how something will be used. I understand where this came from, but any time you limit functionality, you limit your users. As Donald Knuth once said:
Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Sep 30 2022
Jun 30 2022
This seems to be an issue in 1.4 as well, I have the exact same symptoms, and removing the accept-protocol fixes the issue.
Jun 16 2022
Dec 17 2021
Still broken:
trae@cr01b-vyos:~$ show conf com | grep arch set service dhcp-server shared-network-name INT subnet 192.168.1.0/24 domain-search 'int.trae32566.org' set service dhcp-server shared-network-name INT subnet 192.168.1.0/24 domain-search 'ipa.trae32566.org' set service dhcp-server shared-network-name INT subnet 192.168.1.0/24 domain-search 'trae32566.org' set system config-management commit-archive location 'sftp://USER:[email protected]:/int/cr01b-vyos' set system config-management commit-archive source-address 'lo' set system domain-search domain 'int.trae32566.org' set system domain-search domain 'ipa.trae32566.org' set system domain-search domain 'trae32566.org' trae@cr01b-vyos:~$ configure [edit] trae@cr01b-vyos# set system host-name temp [edit] trae@cr01b-vyos# commit Using source address lo Archiving config... sftp://stor01z-rh8.int.trae32566.org:/int/cr01b-vyos Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3/dist-packages/vyos/remote.py", line 312, in upload urlc(urlstring, *args, **kwargs).upload(local_path) File "/usr/lib/python3/dist-packages/vyos/remote.py", line 202, in upload with self._establish() as ssh, ssh.open_sftp() as sftp: File "/usr/lib/python3/dist-packages/vyos/remote.py", line 189, in _establish sock = socket.create_connection((self.hostname, self.port), socket.getdefaulttimeout(), self.source) File "/usr/lib/python3.9/socket.py", line 843, in create_connection raise err File "/usr/lib/python3.9/socket.py", line 830, in create_connection sock.bind(source_address) socket.gaierror: [Errno -5] No address associated with hostname [edit]
This is on 1.4-rolling-202112160318
Dec 6 2021
I think this actually inadvertently broke things even more, because now:
Nov 20 2021
I think this is what it would look like in service dhcp server. I left some comments to explain my thinking a bit, and I tried to make it as flexible as possible (for example the way match options are strings, so future DHCP options can be supported as soon as ISC supports them):
failover {
name INT
remote 192.168.15.4
source-address 192.168.15.3
status primary
}
shared-network-name INT {
description "Internal connection to ir01"
class CLIENT_MAP {
rule 10 {
action permit # This is equivalent to dhcpd's allow/deny members of
match option "agent.circuit_id" value "Vlan200" # This could match any option (ex: dhcp-client-identifier)
}
}
class GUEST_MAP {
rule 10 {
action permit
match option "agent.circuit_id" value "Vlan240"
}
}
subnet 192.168.1.0/24 {
class CLIENT_MAP
default-router 192.168.1.1
domain-name int.trae32566.org
domain-search int.trae32566.org
domain-search ipa.trae32566.org
domain-search trae32566.org
enable-failover
name-server 192.168.255.1
name-server 192.168.15.10
name-server 192.168.31.3
ntp-server 192.168.255.2
ntp-server 192.168.15.11
ntp-server 192.168.31.4
range CLIENTS {
start 192.168.1.2
stop 192.168.1.240
}
server-identifier 192.168.15.2
static-mapping QUEST {
ip-address 192.168.1.17
mac-address 80:f3:ef:11:e7:e7
}
}
subnet 192.168.6.0/24 {
class GUEST_MAP
default-router 192.168.6.1
enable-failover
name-server 1.1.1.1
name-server 1.0.0.1
name-server 8.8.8.8
ntp-server 50.205.57.38
ntp-server 64.225.34.103
ntp-server 129.250.35.251
server-identifier 192.168.15.2
range GUESTS {
start 192.168.6.2
stop 192.168.6.254
}
}
subnet 192.168.15.0/29 { # This tells it indirectly to use the interface eth2, which is on this subnet (is there a better way?)
default-router 192.168.15.1
enable-failover
range DUMMY {
start 192.168.15.2
stop 192.168.15.7
}
}
}Oct 23 2021
Oct 14 2021
Oct 9 2021
Jul 8 2021
trae@cr01a-vyos# show system config-management
commit-archive {
location sftp://cr01a-vyos.int:<somePassword>@stor01z-rh8.int.trae32566.org:/int/cr01a-vyos
source-address lo
}
commit-revisions 10000Jul 7 2021
This is still broken on the most recent rolling release:
trae@cr01a-vyos# commit
Using source address lo
Archiving config...
sftp://stor01z-rh8.int.trae32566.org:/int/cr01a-vyos Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 315, in upload
upload_sftp(local_path, url.hostname, url.path, username, password, port, source, progressbar)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 190, in upload_sftp
transfer_sftp('upload', *args, **kwargs)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 162, in transfer_sftp
sock.connect((hostname, port))
OSError: [Errno 22] Invalid argument
[edit protocols bgp]Jun 23 2021
Jun 22 2021
Confirmed that's what is happening:
vyos@cr01a-vyos# TEST='variable' [edit] vyos@cr01a-vyos# set system login user vyos authentication plaintext-password HqNzXaK27k19$TEST [edit] vyos@cr01a-vyos# comp [edit system login user vyos authentication] +plaintext-password HqNzXaK27k19variable
@fernando Are you sure you're testing this on 1.3?
vyos@cr01a-vyos# run show ver
Jun 20 2021
Yes, though the error has changed:
trae@cr01b-vyos# commit
Using source address fd52:d62e:8011:fffe:192:168:253:3
Archiving config...
sftp://stor01z-rh8.int.trae32566.org:/int/cr01b-vyos Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 299, in upload
upload_sftp(local_path, url.hostname, url.path, username, password, port, source, progressbar)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 178, in upload_sftp
transfer_sftp('upload', *args, **kwargs)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 174, in transfer_sftp
sock.shutdown(socket.SHUT_RDWR)
OSError: [Errno 9] Bad file descriptorJun 16 2021
Jun 12 2021
This appears to be fixed in the most recent rolling releases; I'm not sure how, but it's fixed.
Jun 11 2021
This does not appear to be fixed; I think it's something specific to 1.4:
trae@cr01a-vyos# commit
Using source address fd52:d62e:8011:fffe:192:168:253:2
Archiving config...
sftp://stor01z-rh8.int.trae32566.org:/int/cr01a-vyos Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 287, in upload
upload_sftp(local_path, url.hostname, url.path, username, password, port, source, progressbar)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 166, in upload_sftp
transfer_sftp('upload', *args, **kwargs)
File "/usr/lib/python3/dist-packages/vyos/remote.py", line 162, in transfer_sftp
sock.shutdown()
TypeError: shutdown() takes exactly one argument (0 given)
[edit policy route-map BGP-BACKBONE-OUT]
trae@cr01a-vyos# run show verI have a similar problem, but different, in T3563. I've reopened it and added information, but basically 1.4 still has the issue reported in that bug report.
Jun 10 2021
Again, the problem is not the time it takes to commit, but the time it takes to set. I will try reproducing it again and see if I can come up with an easier way. I would suggest trying adding the BGP config and prefix lists and route-maps without the export route-map applied, then commit, then try applying the export route-maps.
FRR appears to have no problems processing this extremely quickly:
vyos@cr01b-vyos# time vtysh -c "conf t" -c "router bgp 4242420670" -c "address-family ipv4 unicast" -c "neighbor BACKBONE route-map BGP-BACKBONE-OUT out"
Jun 9 2021
I'm wondering if perhaps it's my prefix lists or route-maps maybe? I can upload those in a bit once I test how fast / slow it is in frr.
Jun 7 2021
This is indeed fixed!
May 23 2021
May 20 2021
Mar 27 2021
Mar 3 2021
I've had this bite me a few times now as well, but I wasn't able to pin it down before to being a bug.
Mar 2 2021
Feb 15 2021
Jan 22 2021
Jan 18 2021
Interesting. I agree, I think that defeats the purpose of VRRP if both have to be running. I guess we can go ahead and close this as not feasible at this time.
Jan 15 2021
This can be fixed by copying the two profile directories you're using (/usr/lib/tuned/network-throughput and /usr/lib/tuned/network-latency) to /etc/tuned and changing the sysctl section to only contain enabled=false. Here's it working:
root@cr01a-vyos:~# diff /usr/lib/tuned/network-latency/tuned.conf /etc/tuned/network-latency/tuned.conf 13,16c13 < net.core.busy_read=50 < net.core.busy_poll=50 < net.ipv4.tcp_fastopen=3 < kernel.numa_balancing=0 --- > enabled=false root@cr01a-vyos:~# diff /usr/lib/tuned/network-throughput/tuned.conf /etc/tuned/network-throughput/tuned.conf 10,16c10 < # Increase kernel buffer size maximums. Currently this seems only necessary at 40Gb speeds. < # < # The buffer tuning values below do not account for any potential hugepage allocation. < # Ensure that you do not oversubscribe system memory. < net.ipv4.tcp_rmem="4096 87380 16777216" < net.ipv4.tcp_wmem="4096 16384 16777216" < net.ipv4.udp_mem="3145728 4194304 16777216" --- > enabled=false
Jan 14 2021
Yep, I use RADIUS.
Jan 9 2021
This is fixed:
trae@cr01b-vyos:~$ restart flow-accounting trae@cr01b-vyos:~$ show ver
Jan 7 2021
Jan 6 2021
Nov 3 2020
Yes! Turns out the following is what fixed it:
https://phabricator.vyos.net/T2980
Oct 26 2020
All of ours do as far as I know, which includes 10,40, and 100g. I'm pretty sure 2.5g and 5g will as well.
Oct 18 2020
Works for me!
trae@cr01b-vyos:~$ show protocols bfd peers Session count: 11 SessionId LocalAddress PeerAddress Status ========= ============ =========== ====== 3776760774 192.168.253.3 192.168.253.7 up 1851352402 fd52:d62e:8011:fffe:192:168:253:3 fd52:d62e:8011:fffe:192:168:253:6 up 3344115206 192.168.253.3 192.168.253.2 down 1252680903 fd52:d62e:8011:fffe:192:168:253:3 fd52:d62e:8011:fffe:192:168:253:2 down 3664188082 192.168.253.3 192.168.253.6 up 2809207409 fd52:d62e:8011:fffe:192:168:253:3 fd52:d62e:8011:fffe:192:168:253:1 up 2086113021 192.168.253.3 192.168.253.12 up 1362288442 unknown fd52:d62e:8011:fffe:192:168:253:12 down 3846665654 fd52:d62e:8011:fffe:192:168:253:3 fd52:d62e:8011:fffe:192:168:253:7 up 276439511 fd52:d62e:8011:fffe:192:168:253:3 fd52:d62e:8011:fffe:192:168:253:12 down 1342044518 192.168.253.3 192.168.253.1 up
Oct 16 2020
That sounds great to me! I actually like that more.
Oct 15 2020
awesome, thanks!
Oct 13 2020
Oct 12 2020
Oct 7 2020
Oct 6 2020
Sep 12 2020
Try a show ipv6 route ospfand look at the routes; they're probably being rejected:
trae@cr01b-vyos# run show ipv ro ospf
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route