If you think it's possible to delay dhcp6c, I'll start making patches.

@c-po @gadams I have a remedy in mind. Can I use the delay command to execute the delay startup in the startup service file?

@CRCinAU @c-po I have already submitted a PR, to repair the problems caused by the first load after boot, because the interface has not yet been initialized, but this completely depends on the automatic restart of systemctl for fault recovery. I think this is not enough. It is natural to recover from the failure, but if there is a mechanism and processing, it would be better to postpone it until all interfaces have been initialized.

@gadams Is there any way to dynamically increase the dependency list in vyos's current configuration?

There is no error in the configuration itself, and the key is the first boot load when rebooting. In this case, some of the interfaces are not initialized.

So - just to refocus for a minute...

@c-po if the interface dependency system that @jjakob describes works as I might imagine, then perhaps it's just a matter of adding the interfaces that appear in prefix-delegation configs to the dependency lists. (There would be some subtleties dealing with things like vifs within an interface, but that can be sorted out.)

In T421#65476, @gadams wrote:

Recovery from failures does seem generally desirable, but it would also be preferable to discover errors in configuration while in conf code. For this reason, it seems like the best way to handle this would be to defer starting dhcp6c until the very end of configuring all the interfaces, if that's possible. Is there a mechanism already to do this, or should I look into restructuring things slightly.

I've been trying to find the ultimate solution to this problem, and it seems that the best solution is to postpone dhcp6c until the end of all interfaces to start it. Before that. I have done enough testing, and the previous comments in the error log have been released.

@jjakob I understand what you mean, so let me explain to you carefully. Over the past two days, I have been trying to find out why the first automatic startup of dhcp6c fails (this is usually due to the automatic call when vyos automatically loads the configuration). At present, the root cause has been determined. I just call dhcp6c manually by the script of vyos, and I also understand this process, but because of the particularity of prefix delegation, such as my case, When pppoe completes the call to execute dhcp6c@pppoe0 execution prefix delegation, I give the prefix to br1 and br2, respectively, which are not created at the startup of their pppoe0 and dhcp6c@pppoe0, so the dhcp6c launcher will fail. I currently have a pudding configuration that when the vyos script manually invokes and starts dhcp6c and fails, it restarts at regular intervals until the service runs or the user stops it manually. But this is far from enough, it only depends on the recovery from the failure of the systemctl startup service to complete the recovery operation. As @gadams said, due to the particularity of dhcp6c's configuration file, when its prefix is delegated, its dependency will change from configuration to change, and its behavior allows it to depend on all interfaces. Therefore, it is a good idea to postpone the startup of dhcp6c to the last call (please note that fault exploration has been completed, the root cause of this failure has been determined, and its temporary patch configuration has been tested in the local environment. Therefore, we can first approve the patch configuration merge, and then discuss the issues related to the postponement of dhcp6c)

I haven't looked at how dhcp6c gets started currently. VyOS uses systemd to manage the services, but none of them should be set to enabled, they're all started manually via VyOS scripts. It's possible it's done differently in this case, I'm not going to speculate on something I don't know. I assumed it got started the same, when the interface script starts it.
On the dependency problem, I don't know how dhcp6c behaves when it's started with configured nonexistent interfaces. If it does cause a failure to start, that is an issue that needs to be fixed via another way. I'm not the implementer of this code so I'm not going to speculate on the best way to do it.

That's basically re-implementing and duplicating code from the ethernet script. It would work for bonding and for the link-local, but I'm thinking there may be other attributes that enslaving it to a bond (or bridge) may have changed (MTU?) that I don't know if they're changed back by the kernel after unenslaving it. It would quickly become a kludge.
You'd also need to do the same in the bridge interface, but there there can be any interface type enslaved, so you'd need to first get the interfaces config path (via Section). You'd end up with 2 pieces of code that are slightly different that duplicate code from the interface scripts (rather I think it's been moved into configdict.intf_from_dict).
It is possible, but I don't like it at all.

I just did a few more tests and the reason why IPv6 is gone is that the interface is being automatically disabled after it leaves bond membership. I will change the description accordingly.

Of course, if there is a better way to solve this problem, you are welcome to put forward

When using dhcp6c for prefix delegation fetch, it can rely on any interface and should be called after all interfaces have finished starting

Because it is impossible to determine the user's dependence on the configuration interface of dhcp6c, the dependency problem has already occurred, which will cause the startup program of dhcp6c to fail after rebooting the system. I have made a patch configuration so that dhcp6c, can be restarted indefinitely in the event of failure, but this is still not the best way. Of course, the best solution may be to postpone the startup priority of dhcp6c.

Sure, a new task would be very welcome so there's less spam in this task.
Why do you want to postpone dhcp6c startup? All the requirements and dependencies are there when the interface scripts start it. The interface is brought up before it's started. Other than waiting for a pppoe connection, yes, that would be worthwile. Each interface script has a priority so that other interfaces they depend on are configured before the one that depends on them, that's set in the priority tag in the XML definitions and done by vyatta-cfg. They're started sequentially by their priority value, not all at once.

@jjakob, thanks for your opinion. What if we just add a couple of lines into the apply() function in the interfaces-bonding.py:
https://github.com/vyos/vyos-1x/compare/current...L6NqLW:T2515?expand=1

Because I need to recover the fault first, so I made a patch. After all, I don't have a good way to postpone its processing.

I think this problem can consider setting up a new task list and studying how to postpone the processing of this. Fault recovery is usually desirable, but we should not push all possible priority-induced failures to recovery in the fault.

I still think the failure recovery mechanism needs to exist, but I agree with you. I think we should postpone the startup mechanism of dhcp6c until all interfaces have been initialized. A better idea is to execute dhcp6c processing uniformly after all interfaces have been initialized.

Recovery from failures does seem generally desirable, but it would also be preferable to discover errors in configuration while in conf code. For this reason, it seems like the best way to handle this would be to defer starting dhcp6c until the very end of configuring all the interfaces, if that's possible. Is there a mechanism already to do this, or should I look into restructuring things slightly.

Please merge this fix.

https://github.com/vyos/vyos-1x/pull/438

The repair settings take effect on tests in the local environment.

@gadams Yes, I thought that since the system CTL automatic restart failed, I might need to write a script to perform the automatic recovery. Now it doesn't seem necessary. I will modify its service file.

OK, I have found the best recovery. I will submit PR immediately. I will modify the service settings of systemctl and use its failure to restart automatically to fix the problem. When dhcp6c service fails to start, it will restart according to the preset settings.

Something else I realized last night: In general, it's not safe to start dhcp6c before all interfaces are configured, as long as PD is specified (whether 'address dhcpv6' is specified). That's because the prefix-delegation stanza can refer to any other interface on the system--even ones that haven't been set up, yet. That might include vif interfaces (such as I noticed last night) or any other virtual interface, like br or tun.

@gadams it makes no sense to use this as a catch-all thread. New requests/bugs should go into dedicated tasks.

@jjakob Yes, exactly my thoughts, and what my last pull request starts. I'll try to catch the remaining cases later this evening my time (in 12 hours or so). I can imagine one case that might be a little tricky.

Yes there have been issues with interface naming in the past. Hopefully they are finally resolved in 1.3 now.

In general, there are several solutions:
a) Add the CLI option of auto repair daemons, and rely on cron to execute the repair program. In case of service failure, the service can be restarted automatically
b) Find the only way to solve the problem thoroughly
Generally, I prefer a + b, so that when the service fails to start in a single time, the daemons can complete the recovery execution.
But it's just an idea. If you have any other suggestions, please let me know.

@zsdc can you try to reproduce this issue on 1.3 rollings or on 1.2.5? I can't reach this behavior.

PR added https://github.com/vyos/vyatta-cfg-vpn/pull/33.

vyos@vyos# commit
[ vpn ]
Warning: local prefix 192.168.34.0/24 specified for peer "192.168.50.2"
is not configured on any interfaces

PR https://github.com/vyos/vyatta-cfg-quagga/pull/48
Also added the second commit which fixes the path to zebra daemon

I try to manually modify the contents of /etc/ppp/ipv6-up.d/1000-vyos-pppoe-pppoe0 , and change the following commands:

@gadams I agree it's confusing, to change the syntax isn't hard, we just have to choose the best user-friendly syntax and behavior. It can be even accomplished without changing the syntax, by:
a) if 'dhcpv6-options delegate' is set, do the same as for 'parameters-only', plus start dhclient by add_addr('dhcpv6')
or
b) start dhclient if either 'dhcpv6-parameters' or 'address dhcpv6' is set but only assign an address in the 'address dhcpv6' case, may be the simplest option.

Ah, I think I see what you, @tbr and @jjakob are getting at. If I want to do DHCPv6-PD, then I need to start the daemon. But if I don't want an address using the protocol, then I can explicitly turn it off.

Reworking my config so that it uses eth0->eth3 instead of eth1->eth4 makes everything work as expected. So something has clearly changed regarding the interface naming/creation logic.

@jjakob Yes, I tried dhcpv6-options parameters-only; it had no effect. I did not try 'address dhcpv6' simply because that doesn't seem like a great configuration. But it would have been worth testing, anyway. And while that would have been good to test, it would be a pretty awkward an unexpected workaround for regular users to think of it.

@gadams have you tried the above 2 settings: 'address dhcpv6' and 'dhcpv6-options parameters-only' without your patch to see if the client doesn't assign an address in that case?

I have sent a pull request: https://github.com/vyos/vyos-1x/pull/437

@tbr thanks for clarifying that, I agree. So the way to do that would be to set 'address dhcpv6' and 'dhcpv6-options parameters-only'. That is slightly confusing at first, as the combination of those 2 options shouldn't actually assign an address. I haven't tried it but that's how I expect it should work, I don't use PD currently. If it does work my comments regarding new methods in scripts are entirely unneeded.

This is difficult to solve with the current config syntax where the bond and bridge members are under the bonding and bridge nodes. When modifying bond or bridge members, only interfaces-bonding.py or interfaces-bridge.py is called, which can't modify the interfaces themselves, as all the interface logic (adding and deleting addresses) is in the interface script itself (e.g. interfaces-ethernet.py). The thing that says which script to run is the owner attribute of the interface node, which is ran by vyatta-cfg scripts on commit.

Just my input: It seems there is some confusion about what DHCPv6-PD actually is. And this reflects in the endless discussions and questions here in the thread...

Maybe we should add new methods to the Interface or DHCP class to allow starting just DHCPv6-PD without assigning an address to it? The way it's done now is by assigning an address with the value "dhcpv6" to the interface through the add/del_addr methods of Interface class. There needs to be a separate method for DHCPv6-PD without addressing (and generate a dhcpc config that doesn't assign the address, of course).

Aha! Thanks, @c-po. As I suspected, there is an assumption in ifconfig/interface.py that the DHCPv6 client need not be started if we're not getting our address via DHCPv6, which of course is not the case, here.

@gadams In my environment, oddly enough, after a reboot dhcp6c@pppoe0 Restart PPPoE 0 by using the vyos command or use systemctl to query and find that the service is manually restarted dhcp6c@pppoe0 Before that, the service was in a failed state and the Restart field of the service did not work. During the configuration loading process after system restart, I cannot make it start normally

https://github.com/vyos/vyos-1x/blob/current/python/vyos/ifconfig/dhcp.py#L113

I'm sorry to say that with the current rolling release, still no dhcp6c is started when I add the PD configuration. Here is my test config:

Test package for 'vyos-api-tools' here:
https://github.com/jestabro/vyos-api-tools

During testing I've found that there is a well known problem (we had for ethernet interfaces) also in the serial ports. They can be enumerated and mapped to /dev/ttyUSBxxx differently from boot to boot. This is especially painful on my development APU4 board which also has a Sierra Wireless MC7710 LTE module installed which operates via ttyUSB2 (when no serial console cable is attached) - on subsequent boots this can become ttyUSB3 or depending on the number of FT232 dongles I attach.

Dependency dropped.

The dependency on flask-restx was dropped, in favor of FastAPI. The move to Flask itself for stability was completed,

The dependencies, such as FastAPI, are to be collected in the debian package 'vyos-api-tools'. Screenshot of the redoc page below:

@c-po It has to be said that this is really a troublesome problem. I'm still stuck in fault exploration, rather than making patches. Therefore, there may be some assumptions and verification processes, but it's really troublesome. When the system is restarted, the automatic operation of the service will fail, and I have to restart the service manually.

@c-po Strangely, after rebooting the system, I specifically dhcp6c@pppoe0 Query and find that this service is manually restarted by using the vyos command to restart PPPoE 0 or by using systemctl dhcp6c@pppoe0 Before, this service was in failure status, and your Restart field of the service didn't work. I can't make it start normally during the configuration loading process after the system is restarted 。

@gadams your mentioned problem is already fixed in the latest rolling image

@jack9603301 your assumptions are invalid. I have a fully reboot-save PPPoE setup. Please stop making wrong assumptions! and search the code properly!

Bug with FRRouting 7.3.1

PR https://github.com/vyos/vyos-1x/pull/435

@c-po Trace the problem that the system cannot automatically perform prefix delegation after restarting:

ps afx shows frr processes still running. systemctl stop vyos-router; pkill -f '*.frr*.'; systemctl start vyos-router makes vyos-router start successfully, but with these errors:

@c-po I didn't find out where to call using wide DHCP. I intend to analyze the reason why restart can't automatically start allocating prefix in the 20200523 image.

Additional information, it also appears to be broken in 1.2.5 (self built image) - seems to be the same problem.

Show ip route without prefix don't work.

VyOS added those during the 2nd boot after upgrade. I would assume bug related to the fact that my config doesn't include eth0.

set policy community-list COM01 rule 10 action permit 
set policy community-list COM01 rule 10 regex 65001:0
commit
[edit]
vyos@r-roll#

commit without errors

vtysh -c "show run"
!
bgp community-list expanded COM01 permit 65001:0
!

Why 2 times hw-id for eth1 eth2 eth3

In T2523#65310, @c-po wrote:

Why is there no eth0 on VyOS 1.2.5?

I included all the bits I thought were relevant in T421#65109 (you'll need to click "Show older changes" at this point to see it).