This is extracted into its own task as it does not fit into where it originated: https://vyos.dev/T5083
Demonstration of the issue where API does not return the error message:
curl -k --location --request POST "https://localhost/configure" --form key="localkey" --form data='[{"op":"set","path":["policy", "access-list", "2", "rule", "5", "description", "api-test"]}]' {"success": false, "error": "[[policy]] failed\nCommit failed\n", "data": null}
It does however show up with the CLI:
USER@vyos02# compare
+ policy {
+ access-list 2 {
+ rule 5 {
+ description "cli-test"
+ }
+ }
+ }
[edit]
USER@vyos02# commit
Action must be specified for "access-list 2 rule 5"!
[[policy]] failed
Commit failed
[edit]Error messages when using the API is lost during the chain of child processes being invoked. The reason the CLI can see the error message seems to be a workaround for the lack of a proper error message return in the chain. The workaround is simply spewing the error message into the tty the cli command is running in: https://github.com/thomasfinstad/vyos-1x/blob/eef9874887fbe61006ab7bd861c3aa385fec7f71/src/services/vyos-configd#L130
The chain of execution I believe is as follows:
- API(configure)
- vyos-http-api-server(.py)
- configsession.py
- /opt/vyatta/sbin/my_commit(symlinked to: my_cli_bin)
- sudo is called here, but I am too code blind when it comes to C to figure out where/how it is called
- vyshim (ZMQ client)
- vyos-configd-service (ZMQ server)
- <py service file>
The error message is lost during the MQTT communication as that only returns a single byte integer, and the tty workaround makes sure the CLI shows the error.
(This, and other, internal workflows would be great to find in the docs as I had to stumble my way through several repos to mentally connect the peices)
I have looked at trying to propagate the error message up through the stack back to the API and return it to the user, however I am unable to handle it at this point in time as I am completely inexperienced with C and C++ and the vyatta-cfg repo will not build for me so I can not even test any changes I were to attempt.
For reference this is the build error I get:
$ sudo docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:current ./configure Current UID/GID: 1000/100 ./configure: line 2433: syntax error near unexpected token `gnu' ./configure: line 2433: `AM_INIT_AUTOMAKE(gnu no-dist-gzip dist-bzip2 subdir-objects)'
If I try to use autoconf && automake I get this:
$ sudo docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:current autoconf -i Current UID/GID: 1000/100 configure.ac:26: warning: AC_PROG_LEX without either yywrap or noyywrap is obsolete /usr/share/autoconf/autoconf/programs.m4:716: _AC_PROG_LEX is expanded from... /usr/share/autoconf/autoconf/programs.m4:709: AC_PROG_LEX is expanded from... configure.ac:26: the top level configure.ac:31: warning: The macro `AC_HELP_STRING' is obsolete. configure.ac:31: You should run autoupdate. /usr/share/autoconf/autoconf/general.m4:204: AC_HELP_STRING is expanded from... configure.ac:31: the top level configure.ac:36: warning: The macro `AC_HELP_STRING' is obsolete. configure.ac:36: You should run autoupdate. /usr/share/autoconf/autoconf/general.m4:204: AC_HELP_STRING is expanded from... configure.ac:36: the top level $ sudo docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:current automake --add-missing Current UID/GID: 1000/100 configure.ac:16: error: required directory ./config does not exist configure.ac: error: no proper invocation of AM_INIT_AUTOMAKE was found. configure.ac: You should verify that configure.ac invokes AM_INIT_AUTOMAKE, configure.ac: that aclocal.m4 is present in the top-level directory, configure.ac: and that aclocal.m4 was recently regenerated (using aclocal) Makefile.am:10: error: USE_UNIONFSFUSE does not appear in AM_CONDITIONAL Makefile.am:37: error: Libtool library used but 'LIBTOOL' is undefined Makefile.am:37: The usual way to define 'LIBTOOL' is to add 'LT_INIT' Makefile.am:37: to 'configure.ac' and run 'aclocal' and 'autoconf' again. Makefile.am:37: If 'LT_INIT' is in 'configure.ac', make sure Makefile.am:37: its definition is in aclocal's search path. Makefile.am:46: warning: source file 'src/cli_parse.y' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled automake: warning: possible forward-incompatibility. automake: At least one source file is in a subdirectory, but the 'subdir-objects' automake: automake option hasn't been enabled. For now, the corresponding output automake: object file(s) will be placed in the top-level directory. However, this automake: behavior may change in a future Automake major version, with object automake: files being placed in the same subdirectory as the corresponding sources. automake: You are advised to start using 'subdir-objects' option throughout your automake: project, to avoid future incompatibilities. Makefile.am:46: warning: source file 'src/cli_def.l' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cli_val.l' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cli_new.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cli_path_utils.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/common/unionfs.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cli_val_engine.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cli_objects.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cstore/cstore-c.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cstore/cstore.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cstore/cstore-varref.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cstore/unionfs/cstore-unionfs.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cnode/cnode.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cnode/cnode-algorithm.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cparse/cparse.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/cparse/cparse_lex.c' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:46: warning: source file 'src/commit/commit-algorithm.cpp' is in a subdirectory, Makefile.am:46: but option 'subdir-objects' is disabled Makefile.am:93: warning: source file 'src/dump_session.c' is in a subdirectory, Makefile.am:93: but option 'subdir-objects' is disabled Makefile.am:92: warning: source file 'src/exe_action.c' is in a subdirectory, Makefile.am:92: but option 'subdir-objects' is disabled Makefile.am:94: warning: source file 'src/cli_bin.cpp' is in a subdirectory, Makefile.am:94: but option 'subdir-objects' is disabled Makefile.am:95: warning: source file 'src/cli_shell_api.cpp' is in a subdirectory, Makefile.am:95: but option 'subdir-objects' is disabled Makefile.am:91: warning: source file 'src/priority.c' is in a subdirectory, Makefile.am:91: but option 'subdir-objects' is disabled Makefile.am: error: installing 'config/depcomp'; error while making link: No such file or directory /usr/share/automake-1.16/am/depend2.am: error: am__fastdepCC does not appear in AM_CONDITIONAL /usr/share/automake-1.16/am/depend2.am: The usual way to define 'am__fastdepCC' is to add 'AC_PROG_CC' /usr/share/automake-1.16/am/depend2.am: to 'configure.ac' and run 'aclocal' and 'autoconf' again /usr/share/automake-1.16/am/depend2.am: error: AMDEP does not appear in AM_CONDITIONAL /usr/share/automake-1.16/am/depend2.am: The usual way to define 'AMDEP' is to add one of the compiler tests /usr/share/automake-1.16/am/depend2.am: AC_PROG_CC, AC_PROG_CXX, AC_PROG_OBJC, AC_PROG_OBJCXX, /usr/share/automake-1.16/am/depend2.am: AM_PROG_AS, AM_PROG_GCJ, AM_PROG_UPC /usr/share/automake-1.16/am/depend2.am: to 'configure.ac' and run 'aclocal' and 'autoconf' again /usr/share/automake-1.16/am/depend2.am: error: am__fastdepCXX does not appear in AM_CONDITIONAL /usr/share/automake-1.16/am/depend2.am: The usual way to define 'am__fastdepCXX' is to add 'AC_PROG_CXX' /usr/share/automake-1.16/am/depend2.am: to 'configure.ac' and run 'aclocal' and 'autoconf' again configure.ac: error: installing 'config/ylwrap'; error while making link: No such file or directory
I would be capable of making the changes to the vyos-configd, vyos-http-api-server, and vyshim, but my_commit (and the vyatta-cfg repo in general) is too complex for me.
However it does seem to me it should be possible to cut out the "middle man" here and just go directly from vyos-http-api-server to vyos-configd.
Could someone explain the relationship a little better to me as the doc for development says that the C++ backend stuff is only for CLI, if that is correct I assume it might be legacy and might mean it can be cut out?
One way to give the API a chance to return the real error message would be to write it to a logfile in vyos-configd and read it back out in vyos-http-api-server, but this would be another hacky workaround like the tty printing and I am not sure this would be a good path to continue?