One compelling idea is to move the implementation of the path argument in get_config_dict out of show_config, and replace it with a function to get a sub dictionary from the full get_config_dict. An example implementation is below. This would not only normalize the result, independent of idiosyncrasies of the current backend, but also future proof the form for the switch to vyconf. Secondly, it would allow us to remove the dependency on show_config itself, since that call is already made at Config initialization. Thirdly, it would allow direct use for off-line config instances, namely those for testing and offload to daemon.

But you make an excellent point, @thomas-mangin ; it may be best to normalize that within config_dict itself, now that we have the xml processing available ...

showConfig produces the following for multi-nodes:

The value vs list is not produced by showConfig, but rather due to matching in configtree. We'll let @dmbaturin consider if this is an issue or not, but I imagine a jinja macro could address the problem where it occurs in template generation.

Changing the api settings (for example: keys, port) will cause a disruptive reload of the http-api service; it will of course succeed, but the response for that set command will reflect the reloading. This is known, if misleading behaviour. I will lower this to normal priority, and consider if we can improve the error/success message.

@jjakob that is in fact what we are doing: for stability, some of the changes are necessarily incremental --- these are not new abstractions, but rather, refinements of the existing ones, which allow the separation of new code from legacy. We are avoiding any patching of the old backend, simply replacing with the structures which will allow a switch to vyconf, while improving performance and extension of the new code.

I agree @jjakob that we are still converging on the best way to structure the salient information, namely the difference between effective and session configs, however this will be internalized: a simple example of use for testing and daemon (and other) is:

I'm not sure that we have to go that far, but let me take a closer look, and we can discuss; the model is to be able to call, in batch, the functions of an arbitrary conf_mode script, without variation in structure or behaviour.

Only that it inherits from Config, hence the form of the Config instance can not be changed, say, to 'off-line' Config, or 'testing' Config, which would have different init functions; if it took Config as an argument instead, or a named argument with default the 'live' Config, then it would not be an issue for generalization. But we should discuss the details further, and collect results and suggestions.

1. commit restrictions/permissions have not changed; all handled in the backend.
2. The daemon can not load the configs, as it does not have/need access to the config session; that's the point.
3. ?
4. the conf_mode script needs to reference the global config; re-setting level is basic hygiene --- the script should ask itself 'what if I am called again?'
5. we want to eat the args in the node.def; so you already have it.

@jjakob. to clarify two points (for my sake as well), there are cases where config fails to the point where the config session can not be initialized, such that one can not enter a config session: here's an example --- say. during development, someone forgets to import a module in an early conf_mode script; config initialization is completely screwed --- this had occurred ages ago, and the only way I found it was that I had happened to be looking at their code a few moments previously and noticed ... that's what vyos-config-debug is for, and why it is hidden on a boot flag. You are talking about a much more reasonable case, where (I imagine) a specific configuration causes a partial failure, but one can still enter a config session. The checks in subtask T2568 may help restore the ability to debug by config/load/commit. Those checks are needed anyway.

https://github.com/vyos/vyos-1x/compare/current...jestabro:config-checks.patch

I agree with the downside of vyos-config-debug; it was provided to allow some analysis when other was not available. What was done before was a straight call to the vyatta backend; what is done now is a essentially what you're suggesting, but in a case where the context of config/configtree is not successfully initialized, hence the failure. This is a good point for analysis, however, and I will restore the behaviour, likely by (a) using a straight call to the backend, and meanwhile (b) investigating if we can have partial context for config/configtree in this case.

The now standard method of debugging was clarified in T2409, with reference to this ticket (vyos-config-debug); the question here is whether there is any way to use 'configure; load; commit' on failed boot config load --- in the worst cases, no, as the CLI has not succssfully initialized, but will investigate. Changing to bug for evaluation.

Yes, let me confirm a few details here ....

@jjakob Thanks for a good summary of one of the issues, namely, not replacing config until 'success' (at least of boot, if not other criteria). Migration does save a backup, but we had recently discussed not doing in place change by default. I would like to assess this in relation to other recent discussions of failsafe and rollback, and then proceed with that compatible change first ...

Test package for 'vyos-api-tools' here:
https://github.com/jestabro/vyos-api-tools

Dependency dropped.

The dependency on flask-restx was dropped, in favor of FastAPI. The move to Flask itself for stability was completed,

The dependencies, such as FastAPI, are to be collected in the debian package 'vyos-api-tools'. Screenshot of the redoc page below:

Cool, thanks for pointing this out; I will take T2501, and will probably rename for clarity.

@jjakob, yes I see the issue there; but firstly, did you try debugging with boot parameter 'vyos-config-debug' ? cf.

@jjakob yes, thanks for pointing this out ...

Thanks, @fetzerms !

low-brow caching:

@fetzerms Have you seen this problem with the fix mentioned above ? If not, I will close.

You want to fetch the repo before building; cf. the 'Generating the container section':

Suggested patch here:
https://github.com/vyos/vyos-1x/compare/current...jestabro:T2049.patch

Okay @jjakob, thanks for pinpointing the concern: (quoting from above) 'IMO this should not read config.boot, but return an empty config' --- this is a reasonable position; I will test this modification, along with a consistent change to get_config_dict, and we can discuss.

I suggest you ignore T2030 for a moment --- I recall discussing that issue with the author, but I think the code comments are leading you astray ... we can return to that later.

It appears you are sending a redundant 'set' within the path itself.

@fetzerms I believe this is resolved by the fix in T1899, which will be in the next rolling release.

This is a result of a regression in unionfs-fuse, fixed in commit 93c2f9b, but not contained in the debian buster package v1.0. The naming convention for directories was changed from unionfs to unionfs-fuse, and then reverted, forgetting to revert the meta directory name. Consequently, bad things can happen for packages that had defined the meta directory consistent with the pre-v1.0 naming, for example, vyatta-cfg. Two fixes have been tested with initial success: (1) patch unionfs-fuse (2) change the meta directory name in vyatta-cfg. Either fix passes initial tests, although closer scrutiny is warranted, as it is a critical issue. The downside of (1) is the need to maintain an upstream or patched version of unionfs-fuse; the downside of (2) is that we are accommodating a regression, which will break again when/if the upstream package is available. Likely the quickest solution is preferred, as this issue is implicated in several recent bug reports.

Confirmed with the simplest 'login as vyos; set interfaces ... description; login as other; delete interfaces .. description; login as vyos; set ...". T2203 is likely a result of this issue. Will reassign to me.

Same symptoms in T1889.

Reopen if needed, but considered resolved.