Phabricator didn't get patches without task number, but this already in rolling
https://github.com/vyos/vyatta-op/commit/8dd886c9542cefe09d5a722307e0acbbf41ee9b8
I propose to backport this into crux.
- Feed Queries
- All Stories
- Search
- Feed Search
- Transactions
- Transaction Logs
All Stories
Apr 26 2020
Bug introduced in https://github.com/vyos/vyos-1x/commit/6ae6c5b
@jjakob as you seem to be the OpenVPN pro - you mind taking a look?
Apr 25 2020
@jjakob if you enable debugging and record what is happening to the interface when it drops would be useful.
@syncer Re-opening this. Had the first exact same incident on a different router with an IPv6 BGP session on a RJ45 connection, so that would rule out any issues with the Intel X710 card in relation to this issue.
I'll try out the latest code then. But on 0421 with vyos-1x from 04-23
15:11 the issue is still there. I had most of the fix ready, it re-added
the interface back to the bridge it is supposed to be in after applying
all settings. I've thought about if this could be some function/method of
ifconfig, but I don't think it's necessary or how it would be
implemented. If the issue is still there, I'll open a PR. Then if there
is a bigger need to add this to ifconfig you can do it later.
For clarity, I do not believe the rewrite in the parent task is the root of the issue, as the behaviour on configuration was not changed. The code was restructured but the action logic remained the same. The same commands/actions will be run now as were run before the patchset. If something changed it will be in the conf mode interface code and not ifconfig AFAICS
Yes the code always does down / up. It is something @cpo and I discussed and it was deemed safer when performing changes. There could be a way to indicate that a change occur which should cause a trigger in another section. Something all sections should run on every change. This would be a major design change as we would not anymore apply everything, would need to decide what part of the change trigger this update process and implement a new section and make sure it is run !
These two days I will try to simulate the failure in the network simulator.
Should be fixed within parent task
Apr 24 2020
@jjakob No, it's not logged in the journal either:
set firewall all-ping 'enable' set firewall broadcast-ping 'enable' set firewall config-trap 'disable' set firewall ipv6-receive-redirects 'enable' set firewall ipv6-src-route 'enable' set firewall ip-src-route 'enable' set firewall log-martians 'enable' set firewall name wan default-action 'drop' set firewall name wan rule 1 action 'accept' set firewall name wan rule 1 state established 'enable' set firewall name wan rule 1 state related 'enable' set firewall name wan-local default-action 'drop' set firewall name wan-local rule 1 action 'accept' set firewall name wan-local rule 1 state established 'enable' set firewall name wan-local rule 1 state related 'enable' set firewall name wan-local rule 2 action 'accept' set firewall name wan-local rule 2 icmp type-name 'echo-request' set firewall name wan-local rule 2 protocol 'icmp' set firewall name wan-local rule 2 state new 'enable' set firewall name wan-local rule 3 action 'drop' set firewall name wan-local rule 3 destination port '22' set firewall name wan-local rule 3 protocol 'tcp' set firewall name wan-local rule 3 recent count '4' set firewall name wan-local rule 3 recent time '60' set firewall name wan-local rule 3 state new 'enable' set firewall name wan-local rule 4 action 'accept' set firewall name wan-local rule 4 protocol 'tcp' set firewall name wan-local rule 4 state new 'enable' set firewall options interface pppoe0 adjust-mss '1452' set firewall options interface pppoe0 adjust-mss6 '1280' set firewall receive-redirects 'disable' set firewall send-redirects 'enable' set firewall source-validation 'disable' set firewall state-policy established action 'accept' set firewall state-policy established log enable set firewall state-policy invalid action 'accept' set firewall state-policy invalid log enable set firewall state-policy related action 'accept' set firewall state-policy related log enable set firewall syn-cookies 'enable' set firewall twa-hazards-protection 'disable' set interfaces bridge br1 address '192.168.0.1/24' set interfaces bridge br1 address 'fc00:470:f1cd::1/64' set interfaces bridge br1 description 'lan 0' set interfaces bridge br1 dhcpv6-options temporary set interfaces bridge br1 ipv6 address autoconf set interfaces bridge br1 ipv6 address eui64 'fc00:470:f1cd::/64' set interfaces bridge br1 ipv6 dup-addr-detect-transmits '1' set interfaces bridge br1 member interface eth1.1 set interfaces bridge br1 member interface eth2 set interfaces bridge br1 stp set interfaces bridge br2 address '192.168.101.1/24' set interfaces bridge br2 address 'fc00:470:f1cd:101::1/64' set interfaces bridge br2 description 'lan 101' set interfaces bridge br2 dhcpv6-options temporary set interfaces bridge br2 dhcpv6-options temporary set interfaces bridge br2 ipv6 address autoconf set interfaces bridge br2 ipv6 address eui64 'fc00:470:f1cd:101::/64' set interfaces bridge br2 ipv6 dup-addr-detect-transmits '1' set interfaces bridge br2 member interface eth0 set interfaces bridge br2 member interface eth1.2 set interfaces bridge br2 member interface eth3 set interfaces bridge br2 member interface eth4 set interfaces bridge br2 stp set interfaces ethernet eth0 description 'LAN' set interfaces ethernet eth0 hw-id '00:98:2b:f8:3f:11' set interfaces ethernet eth0 ipv6 address set interfaces ethernet eth0 ipv6 dup-addr-detect-transmits '1' set interfaces ethernet eth1 description 'Trunk' set interfaces ethernet eth1 hw-id '00:98:2b:f8:3f:12' set interfaces ethernet eth1 vif 1 description 'vlan 1 of eth1' set interfaces ethernet eth1 vif 2 description 'vlan 2 of eth1' set interfaces ethernet eth2 description 'LAN' set interfaces ethernet eth2 hw-id '00:98:2b:f8:3f:13' set interfaces ethernet eth3 description 'LAN' set interfaces ethernet eth3 hw-id '00:98:2b:f8:3f:14' set interfaces ethernet eth4 description 'LAN' set interfaces ethernet eth4 hw-id '00:98:2b:f8:3f:15' set interfaces ethernet eth5 description 'WAN' set interfaces ethernet eth5 disable-flow-control set interfaces ethernet eth5 firewall in name 'wan' set interfaces ethernet eth5 firewall local name 'wan-local' set interfaces ethernet eth5 hw-id '00:98:2b:f8:3f:16' set interfaces pppoe pppoe0 authentication password 'user' set interfaces pppoe pppoe0 authentication user 'passwd' set interfaces pppoe pppoe0 default-route 'auto' set interfaces pppoe pppoe0 description 'ISP' set interfaces pppoe pppoe0 ipv6 address autoconf set interfaces pppoe pppoe0 ipv6 enable set interfaces pppoe pppoe0 mtu '1492' set interfaces pppoe pppoe0 source-interface 'eth5' set nat nptv6 rule 2 description 'IPv6 NPT' set nat nptv6 rule 2 outbound-interface 'pppoe0' set nat nptv6 rule 2 source prefix 'fc00:470:f1cd::/48' set nat nptv6 rule 2 translation prefix '240e:fc:7d:64b9::/64' set nat source rule 1 description 'PUBLIC' set nat source rule 1 log 'enable' set nat source rule 1 outbound-interface 'pppoe0' set nat source rule 1 protocol 'all' set nat source rule 1 source address '0.0.0.0/0' set nat source rule 1 translation address 'masquerade' set protocols static interface-route 0.0.0.0/0 next-hop-interface pppoe0 set protocols static interface-route6 ::/0 next-hop-interface pppoe0 set protocols static table 150 interface-route 0.0.0.0/0 next-hop-interface pppoe0 set protocols static table 150 interface-route6 ::/0 next-hop-interface pppoe0 set service dhcp-server shared-network-name pri101 description 'DHCP 101' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 default-router '192.168.101.1' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 dns-server '192.168.0.254' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 dns-server '192.168.101.1' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 dns-server '192.168.0.1' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 dns-server '192.168.0.1' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 lease '86400' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 ntp-server '192.168.101.1' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 range 0 start '192.168.101.8' set service dhcp-server shared-network-name pri101 subnet 192.168.101.0/24 range 0 stop '192.168.101.254' set service dhcpv6-server preference '0' set service dhcpv6-server shared-network-name pri101 subnet fc00:470:f1cd:101::/64 address-range start fc00:470:f1cd:101::8 stop 'fc00:470:f1cd:101:ffff:ffff:ffff:ffff' set service dhcpv6-server shared-network-name pri101 subnet fc00:470:f1cd:101::/64 name-server 'fc00:470:f1cd::ff00' set service dhcpv6-server shared-network-name pri101 subnet fc00:470:f1cd:101::/64 name-server 'fc00:470:f1cd::1' set service dhcpv6-server shared-network-name pri101 subnet fc00:470:f1cd:101::/64 name-server 'fc00:470:f1cd:101::1' set service dns forwarding allow-from '192.168.0.0/16' set service dns forwarding allow-from '2001:470:f1cd::/48' set service dns forwarding cache-size '1024' set service dns forwarding domain pve. server '192.168.0.47' set service dns forwarding domain pve. server '2001:470:f1cd::47' set service dns forwarding listen-address '0.0.0.0' set service dns forwarding listen-address '::' set service dns forwarding name-server '2001:470:f1cd::ff00' set service dns forwarding name-server '192.168.0.254' set service dns forwarding name-server '202.96.134.33' set service dns forwarding name-server '202.96.128.86' set service dns forwarding name-server '114.114.114.114' set service dns forwarding name-server '1.1.1.1' set service dns forwarding name-server '1.0.0.1' set service dns forwarding system set service https virtual-host vhost0 listen-address '*' set service https virtual-host vhost0 server-name '192.168.0.1' set service mdns repeater interface 'br1' set service mdns repeater interface 'br2' set service router-advert interface br1 set service router-advert interface br2 name-server 'fc00:410:f1cd:101::1' set service ssh listen-address '0.0.0.0' set system acceleration set system config-management commit-revisions '100' set system console device ttyS0 speed '115200' set system domain-name 'router' set system host-name 'vyos' set system ip arp table-size '2048' set system ip multipath layer4-hashing set system ipv6 multipath layer4-hashing set system ipv6 neighbor table-size '2048' set system ipv6 strict-dad set system login user vyos authentication encrypted-password '$6$UaXQViDvJ.Hr$85U/9Q5d/tc9hdtrnntMVgrztOCext..OJCHaJYZUo82GAdD95lchvSjI3vCZJTNte7cIAs87YctYlXODGXAz1' set system login user vyos authentication plaintext-password '' set system name-server '192.168.0.1' set system ntp allow-clients address '192.168.0.0/16' set system ntp allow-clients address 'fc00:470:f1cd::/48' set system ntp listen-address '::' set system ntp listen-address '0.0.0.0' set system ntp server 0.debian.pool.ntp.org set system ntp server 1.debian.pool.ntp.org set system ntp server 2.debian.pool.ntp.org set system ntp server 3.debian.pool.ntp.org set system sysctl custom net.ipv4.conf.all.rp_filter value '0' set system sysctl custom net.ipv4.conf.default.rp_filter value '0' set system sysctl custom net.ipv4.conf.eth0.rp_filter value '0' set system sysctl custom net.ipv6.conf.all.accept_ra value '2' set system sysctl custom net.ipv6.conf.all.forwarding value '1' set system syslog global facility all level 'info' set system syslog global facility protocols level 'debug' set system time-zone 'Asia/Shanghai'
Please always include a copy of your configurationw which would make labbing much easier
It seems that IPv6 is not only the case even in IPv4, but I don't understand that this happens. I remember that 1.2-rolling-20191203 has no such problem. If so, it shouldn't be the problem of my local ISP network. Moreover, this problem also exists in the PPPoE of IPv4. It's more doubtful that there may be a bug in the PPPoE of vyos. Although the bug may occur in a specific environment?
I think it's strange, but in fact, I'll try to see it in GNS tomorrow!
@jack9603301 unfortunately I can not reproduce this behavior on my PPPoE link - it runs stable with the latest rolling versions and I have 0 disconnects - neither in a LAB with VyOS PPPoE server not Deutsche Telekom or M-Net links.
@jjakob this is the wrong phabricator entry to discuss it but I got your point ;-)
This is correct.
I have extracted this question to T2380 alone!
Do they appear in the journal? sudo journalctl -f
I think logs go to systemd-journald first, then are forwarded to rsyslog,
which logs them to files. I've said some time ago, ever since 1.2 moved
to Buster and journald came in, that we should move more to native
journald logging away from rsyslog. I enable persistent journal storage
in /etc/systemd/journald.conf after every upgrade just because of this,
the rsyslog config is buggy/broken, it doesn't log half of what it
should, it stays in journald, so I just use journalctl to look at all
logs. I don't know if journald supports logging to separate files like
rsyslog though, by default it uses its own binary format, I think that
can be changed and maybe rsyslog file behavior moved to journald.
At present, relevant functions are normal after virtual machine test! New bug found, see T2379 for details
If this happens to you, you don't need to delete and re-add the members in the config, you can run sudo ip link set dev ethX master brX to add the eth interfaces back to the bridge.
We ave no RPKI filtering active yet, so https://github.com/FRRouting/frr/issues/5458 seems not related.
Thanks for confirming this.
660 is not enough as the created file is already 660 root:root, which
means just root has the rights to write to it. The failing script isn't
run as root while in a config session (validate-value.py). I'm not
exactly sure which user (vyos or vyattacfg), but as this is a debug log,
it's not security critical and can be set to 666. Otherwise it'd need to
be owned by a group into which all the writing users must be put into,
and can stay 660.
The file should have been 660 and it should have worked .. I will check
Apr 23 2020
sorry, i'm not a hacker, so i dont know any attach vector. :( But it does not mean that it does not exist.
All that's needed is to chmod 666 the file after creation - I did that
after I found the error and all scripts could write to it fine.
Yes, I also have a patch ... somewhere .. which prevent to change any of the /config files on the system (so that a user can not make damage to the system).
I also need to add a check to only use the file if the user and permission are what is expected. I will do it :-)
The file should be created as group vyattacfg tho with rights allowing both the user and root to write to it.
@jjakob There is no need to check if an interface exists before creation, the code has always tried to find the interface and use it if there, otherwise it will create it.
Please test with a image built after April 23 20:44 (no image is built yet, the next one should have the fix for T2362 included)
Fixed by T2362
@jjakob yes, what you propose to check if an interface exists is good. If you know the type (as defined in the class which as the same name as the "set interface" section such as ethernet) you can use Section.interfaces('ethernet') to only get what you want.
PR https://github.com/vyos/vyos-1x/pull/374
Possible commands:
vyos@R1-VM# set service pppoe-server authentication protocols Possible completions: pap Allow PAP authentication [Password Authentication Protocol] chap Allow CHAP authentication [Challenge Handshake Authentication Protocol] mschap Allow MS-CHAP authentication [Microsoft Challenge Handshake Authentication Protocol, Version 1] mschap-v2 Allow MS-CHAPv2 authentication [Microsoft Challenge Handshake Authentication Protocol, Version 2]
I'm still seeing this in VyOS 1.3-rolling-202004170117
In T2366#61424, @runar wrote:"This is fine if used in conf_mode scripts that'll create it after that anyway" if the intention of the code was not to create the interdace this is not fine if you ask me. :)
"This is fine if used in conf_mode scripts that'll create it after that anyway" if the intention of the code was not to create the interdace this is not fine if you ask me. :)
thats correct @jjakob , when a mac is changed on a interface the ONLY place to find the original mac address for a interface is using the hw-id, this is because the kernel does not hold track of the original mac anywhere. for now on saving the config it reverts back to the original hw-id mac when the mac node is deleted witch should be quite fine to do. When the old boot interface mapping code is rewritten these pointers also need to match the new scripts. but thats another storry :)
is if iface in Section.interfaces() the correct way to check if the interface already exists?
Actually, if an interface doesn't exist and if we try to get its properties (for example if mac != BridgeIf('br0', create=False).get_mac():) that raises an exception. I'm not sure what the best way to do here is. Use create=True anyway? This is fine if used in conf_mode scripts that'll create it after that anyway, but what if we're checking another interface that we don't know if it exists or not? First do an existence check? How do we do that?
Bug: if I create '/config/vyos.log.debug' with contents '/config/vyos.debug.log', /config/vyos.debug.log is created as root:root rw-rw-r--, and later scripts that use call() but aren't root, don't have the permission to write to it, and fail. For example any validator like mac-address fails as if the value is invalid, but if we turn on its debug flag, we see the real error is [Errno 13]: Permission denied: '/config/vyos.debug.log' because 'src/helpers/validate-value.py' uses call(), but it isn't ran with root privileges. The debug log should be probably created with 666 permissions or its group changed so that all scripts, even non-root, can write to it.
Oh, I see what that's supposed to do - if there was a 'mac' set to change the interface's mac address, it sets that back to the real factory mac. In that case, that's fine (it's done every time if no 'mac' node is there, which is rather pointless, but shouldn't harm anything), the bug with interface renaming is a separate bug that's not applicable here.
@c-po looking at your commit d80398b6, I don't think that's how the hw-id node was supposed to be used. It was never used as a way to change the interface's mac address, rather it was used to rename the interface with the matching mac address to the name it has in config.boot. This is done by some vyatta scripts at boot time. If we later set the same mac on the interface that it should already have (as it was renamed by the vyatta scripts), that's pretty pointless to do IMO.