Page MenuHomeVyOS Platform

extend interface schema to include which parameters are required
In progress, NormalPublicENHANCEMENT

Description

Currently the logic to check if a parameter / node is required is done in the python part of the commands, if this was included in the schema definition instead it would allow for much easer development of tooling that can integrate with HTTP API, or via ssh.

For example if the checks to see if port was moved from the bcast_relay.py into the bcast-relay.xml.in it would mean that those that create tooling could more easily include those requirements without relying onthe API to respond with an error, or digging into the source code to understand how the API will behave.

I tried working on a terraform provider based on the schemas, but this issue combined with how the API did not always give the full error message back (missing the reason for the error) made it difficult to continue as the user would not understand why something went wrong as shown below:

cli:

user@vyos02# set service broadcast-relay id 33 description "test"
[edit]
user@vyos02# commit

Port number mandatory for udp broadcast relay "33"

[[service broadcast-relay]] failed
Commit failed
[edit]

api:

user@vyos02:~$ curl -k --location --request POST 'https://localhost/configure' --form data='{"op": "set", "path": ["service", "broadcast-relay", "id", "33", "description", "test"]}' --form key='KEY'
{"success": false, "error": "[[service broadcast-relay]] failed\nCommit failed\n", "data": null}

Details

Version
-
Is it a breaking change?
Unspecified (possibly destroys the router)

Event Timeline

Viacheslav closed this task as Resolved.EditedJan 20 2024, 12:10 PM
Viacheslav claimed this task.
Viacheslav subscribed.

Fixed VyOS 1.5-rolling-202401190024

vyos@r4:~$ curl -k --location --request POST 'https://localhost/configure' --form data='{"op": "set", "path": ["service", "broadcast-relay", "id", "33", "description", "test"]}' --form key='VyOS-key'
{"success": false, "error": "[ service broadcast-relay ]\nPort number is mandatory for UDP broadcast relay \"33\"\n\n[[service broadcast-relay]] failed\nCommit failed\n", "data": null}

This issue seem to be wider than just 1 occurance as I just stumbled into it again on the latest version (1.5-rolling-202402240021)

$ curl -k --location --request POST "https://$VYOS_HOST/configure" --form key="$VYOS_KEY" --form data='[
    {"op":"set","path":["firewall","zone","TF-Examples","intra-zone-filtering","action","accept"]},
    {"op":"set","path":["firewall","zone","TF-Examples","intra-zone-filtering","firewall","name","test"]}
]'
{"success": false, "error": "[[firewall]] failed\nCommit failed\n", "data": null}
USER@vyos02# compare
[firewall zone]
+ CLI-Example {
+     intra-zone-filtering {
+         action "accept"
+         firewall {
+             name "test"
+         }
+     }
+ }

[edit]
USER@vyos02# commit

Zone "CLI-Example" has no interfaces and is not the local zone

[[firewall]] failed
Commit failed
[edit]

The full, and useful part of, description is only showing on the CLI.
I wanted to see if I could figure out a pattern to the issue, but met a dead end when I was unable to track down the change that fixed it.

Viacheslav triaged this task as Normal priority.

Bumped into another instance of this issue:

curl -k --location --request POST "https://$VYOS_HOST/configure" --form key="$VYOS_KEY" --form data='[{"op":"set","path":["policy", "access-list", "2", "rule", "5", "description", "2024-03-16T14:52:44Z"]}]'
{"success": false, "error": "[[policy]] failed\nCommit failed\n", "data": null}
USER@vyos02# compare
+ policy {
+     access-list 2 {
+         rule 5 {
+             description "2024-03-16T14:52:44Z"
+         }
+     }
+ }

[edit]
USER@vyos02# commit

Action must be specified for "access-list 2 rule 5"!

[[policy]] failed
Commit failed
[edit]

Created dedicated task for returning error message via API: https://vyos.dev/T6326

Alternatively I would be willing to look into creating a validator dedicated to Node and TagNode that can be described in the schemas as that is where this task originally started.

A first draft on the structure would be something like:

<?xml version="1.0"?>
<interfaceDefinition>
  <node name="container" owner="${vyos_conf_scripts_dir}/container.py">
    <properties>
      <help>Container applications</help>
      <priority>450</priority>
    </properties>
    <children>
      <tagNode name="name">
        <properties>
          <help>Container name</help>
          <constraint>
            <regex>[-a-zA-Z0-9]+</regex>
          </constraint>
          <constraintErrorMessage>Container name must be alphanumeric and can contain hyphens</constraintErrorMessage>
          <requiredLeafs>
            <must>
              <leaf name="image" \>
            </must>
            <conflicts name="allow_host_networks">
              <leaf name="network" \>
            </conflicts>
            <atLeastOne>
              <leaf name="allow_host_networks" \>
              <leaf name="network" \>
            </atLeastOne>
            <depends name="gid">
              <leaf name="uid" \>
            </depends>
          </requiredLeafs>
        </properties>
        <children>
          ...
        </children>
      </tagNode>
      <tagNode name="network">
        ...
      </tagNode>
      <tagNode name="registry">
        ...
      </tagNode>
    </children>
  </node>
</interfaceDefinition>

In the above suggestion the name requiredLeafs is used to indicate that each Node and TagNode should be responsible for their own LeafNode validation.

The example would replace these checks container.py:

[...]
            if 'image' not in container_config:
                raise ConfigError(f'Container image for "{name}" is mandatory!')
[...]
            # If 'allow-host-networks' or 'network' not set.
            if 'allow_host_networks' not in container_config and 'network' not in container_config:
                raise ConfigError(f'Must either set "network" or "allow-host-networks" for container "{name}"!')

            # Can not set both allow-host-networks and network at the same time
            if {'allow_host_networks', 'network'} <= set(container_config):
                raise ConfigError(f'"allow-host-networks" and "network" for "{name}" cannot be both configured at the same time!')

            # gid cannot be set without uid
            if 'gid' in container_config and 'uid' not in container_config:
                raise ConfigError(f'Cannot set "gid" without "uid" for container')
[...]

This seem the best way to make sure integration tooling based on the schemas can reuse the logic as the constraint + constraintErrorMessage structure don't lend itself well to this work.

It would also allow for incremental migration from the imperative validation to the new declarative way over time, and allowing to clean up the verify() function to be less cluttered when needing to handle more complex checks that can not be done with simple statements in the schema.

Since some of the required children are other node or tagnode types I belive it would be better to change the schema to the following:

<childRequirements>
  <required>
    <child name="image"/>
  </required>
  <conflicts name="allow_host_networks">
    <child name="network"/>
  </conflicts>
  <atLeastOneOf>
    <child name="allow_host_networks"/>
    <child name="network"/>
  </atLeastOneOf>
  <depends name="gid">
    <child name="uid"/>
  </depends>
</childRequirements>

I am looking at the library in vyos1x-config in an attempt to add these features, but progress is/will be slow.