Page MenuHomeVyOS Platform

Rewrite all policy zebra filters to XML/Python style
Closed, ResolvedPublicFEATURE REQUEST


Rewrite zebra policy to XML/Python format

[email protected]# set policy 

+> access-list  
+> access-list6
+> as-path-list
+> community-list
+> extcommunity-list
+> ipv6-route 
+> large-community-list
+> prefix-list  
+> prefix-list6
+> route-map

The second task:
Adding 1000 prefix-lists in the VyOS CLI takes ~3m26s78
Adding 1000 prefix-lists in FRR itself takes ~0m0.424s

Old backend bash checks very slow.
An example


Difficulty level
Unknown (require assessment)
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Internal change (not visible to end users)

Event Timeline

Would love to see this resolved — a large (but reasonable) configuration doing IRR-based filtering from BGP peers took 9 hours to boot up.

I tried to add the prototype policy with XML.
Speed of validators "<validator name="ip-prefix"/>" very slow.

It's due to Python startup time. Validation of every config node requires a full startup of Python. It would be better to if the validators were shell scripts or C programs, as they were before, the load time is very small :) I think this is also a part of the significant difference in boot and commit times between 1.2 and 1.3.

Or if python were invoked only once to validate all the config nodes.

$ time sudo /usr/libexec/vyos/ --exec /usr/libexec/vyos/validators/ip-prefix --value ''

real	0m0.133s
user	0m0.097s
sys	0m0.037s

$ time sudo sh -c ' for ((n=0;n<1000;n++)); do /usr/libexec/vyos/ --exec /usr/libexec/vyos/validators/ip-prefix --value ""; done'

real	1m58.335s
user	1m34.704s
sys	0m24.341s

That's 118s for 1000 config lines (590s for 5000 lines) just for the validator, not counting other execution time. Also note this is a relatively powerful 2.5Ghz dual-core Pentium E5300.

$ cat /usr/libexec/vyos/validators/ip-prefix

ipaddrcheck --is-any-net $1
$ time ipaddrcheck --is-any-net ''

real	0m0.003s
user	0m0.003s
sys	0m0.000s
$ time sudo sh -c ' for ((n=0;n<1000;n++)); do ipaddrcheck --is-any-net ""; done'

real	0m1.985s
user	0m1.235s
sys	0m0.868s

Only 1.9s for 1000 invocations of the same validator directly, without going through the Python invocation.

I think this is a powerful enough argument to rewrite the validators to shell or C (ipaddrcheck is already C). doesn't do anything other than invoke the destination validator or do a simple regex, which can be a sh or C validator too.

Using shell scripts would be a step back. The biggest part of the python script is the parsing of the vyos code. That's why I am suggesting that vyos command should be sent to a python daemon.
We can connect to it using IPC, and this can be done via a small C wrapper (or even directly in the caller's code to not even fork).

[email protected]# time bash -c ""

real	0m0.001s
user	0m0.001s
sys	0m0.000s
[email protected]# time python3 -c ""

real	0m0.019s
user	0m0.011s
sys	0m0.007s
[email protected]# time python3 -c "import vyos.ifconfig"

real	0m0.173s
user	0m0.104s
sys	0m0.031s

For example using the server at:, forking shell, nc, head and using echo:

[email protected]:~$ time bash -c 'echo "test" | nc -U uds_socket | head -1'

real	0m0.004s
user	0m0.003s
sys	0m0.000s

So it is possible to have all the code in a high language and still having good performance that way.

@thomas-mangin if you can demonstrate your idea implementing the actual validator code would be close performance-wise to the old shell/C validators, I would be on board with it. But personally I like shell and C better :)

Edit: since the old config system is C and calls an executable for each validated value, that executable would need to have a very fast load time.

@jjakob I was going to offer to do so this week if people agreed to give me a chance to show good it could be.

Personally I don't understand the insistence on keeping everything Python - in some cases, it is simply much better to use a more suitable language. A simple C program is IMO simpler and more KISS than a python daemon with a listening socket with clients connecting to it via UDP. You're invoking several layers of the OS (networking, process management, sockets) that are completely unnecessary in a simple "is this value OK?" program. I don't like convoluted solutions to solve a simple problem.

Viacheslav renamed this task from Rewrite policy prefix-list to XML/Python style to Rewrite all policy zebra filters to XML/Python style.Feb 11 2021, 12:28 PM
Viacheslav updated the task description. (Show Details)

XML for policy access|access6|prefix|prefix6|community list


[email protected]# set npolicy 
Possible completions:
+> access-list  IP access-list filter
+> access-list6 IPv6 access-list filter
+> as-path-list Border Gateway Protocol (BGP) autonomous system path filter
+> community-list
                Border Gateway Protocol (BGP) autonomous system path filter
+> extcommunity-list
                Border Gateway Protocol (BGP) extended community-list filter
+> large-community-list
                Border Gateway Protocol (BGP) large-community-list filter
+> prefix-list  IP prefix-list filter
+> prefix-list6 IPv6 prefix-list filter
+> route-map    IP route-map
erkin set Issue type to Internal change (not visible to end users).Aug 30 2021, 6:20 AM
erkin removed a subscriber: Active contributors.