The called code can return 3 - in that case in that case _run should return an empty string

I think 'configure; load; commit' is important to make debugging easier and faster. There are issues with the vyos-config-debug method: it needs a full reboot to test every change, which can take minutes (and one may not fix the bug in the first 10 tries even, depending on how sleep deprived one is) and it lacks a easy way to see the scripts stdout/stderr (it is discarded unless we enable airbag's debug log, which is yet another thing to have in mind) as the standard traceback that's logged to /tmp may not be enough to catch the exact error and we need to print some variables to look at them or something like that. But mainly the issue is rebooting is much slower than just doing load/commit.
Can't we just read config.boot into the session config of configtree or config? Wasn't that exactly what was done before? I'm 100% there was a function that read the config.boot file in config.py in case the config system wasn't initialised.

I agree with the downside of vyos-config-debug; it was provided to allow some analysis when other was not available. What was done before was a straight call to the vyatta backend; what is done now is a essentially what you're suggesting, but in a case where the context of config/configtree is not successfully initialized, hence the failure. This is a good point for analysis, however, and I will restore the behaviour, likely by (a) using a straight call to the backend, and meanwhile (b) investigating if we can have partial context for config/configtree in this case.

@jjakob. to clarify two points (for my sake as well), there are cases where config fails to the point where the config session can not be initialized, such that one can not enter a config session: here's an example --- say. during development, someone forgets to import a module in an early conf_mode script; config initialization is completely screwed --- this had occurred ages ago, and the only way I found it was that I had happened to be looking at their code a few moments previously and noticed ... that's what vyos-config-debug is for, and why it is hidden on a boot flag. You are talking about a much more reasonable case, where (I imagine) a specific configuration causes a partial failure, but one can still enter a config session. The checks in subtask T2568 may help restore the ability to debug by config/load/commit. Those checks are needed anyway.