Every conf-mode script not handled in-daemon by `vyos-configd`, every op-mode script and every Python validator runs as a fresh interpreter. Almost all of that time goes on importing the `vyos` library and its dependencies, so it is paid again on every set, show and commit. On CPUs with little or no L2/L3 cache it is very noticeable.
Four causes were measured. First, heavy trees were imported at module scope for functions that rarely run:
* `vyos.version` pulled in `requests, urllib3 and`, via vyos.system.image`urllib3` and, jinja2; configsource.py pulled google.protobuf for an often-inactive backend;via `vyos.system.image`
* `jinja2`; opmode.py pulled the whole vyos.config tree for one decorator; ethtool.py pulled pyroute2; and ifconfig/interface.py pulled cryptography for set_eapol() alone. Second, vyos/utils/__init__.py`configsource.py` pulled `google.protobuf` for an often-inactive backend
* `opmode.py` pulled the whole `vyos.config` tree for one decorator
* `ethtool.py` pulled `pyroute2`
* `ifconfig`/`interface.py` pulled `cryptography` for `set_eapol()` alone
* `vyos/utils/__init__.py` eagerly imported all 18 submodules. Third,
* every `ConfigTree` re-opened `libvyosconfig.so` and re-declared 31 ctypes prototypes in `__init__, so one DiffTree performed seven dlopens and ~200 prototype assignments.`, Fourth,so one `DiffTree` performed seven `dlopens` and ~200 prototype assignments
* the package ships 188 .py files and zero .pyc: dh_python3's py3clean lands in prerm, but py3compile is dropped because the postinst carries no debhelper substitution token, so the package removes bytecode nothing creates.
* `vyos/xml_ref/cache.py` is generated by the postinst itself and can never be compiled at build time.
The fix is to move rarely-used imports to their call sites, make `vyos.utils` lazy, cache the FFI (Foreign-Function-Interface) bindings once per process, and call `py3compile` and `compileall` from the postinst.
Measured on a fast x86-64 host, best of five runs, as time and `sys.modules` count:
```
+-----------------------+-----------------+----------------+
| import / operation | before | after |
+-----------------------+-----------------+----------------+
| vyos.opmode | 34.0 ms 206 | 4.6 ms 71 |
| vyos.ifconfig | 205.0 ms 889 | 30.8 ms 193 |
| vyos.config | 129.6 ms 575 | 20.6 ms 130 |
| vyos.qos | 134.1 ms 631 | 12.5 ms 124 |
| vyos/xml_ref/cache.py | 216.1 ms | 32.1 ms |
| ConfigTree() | 72.3 us | 6.3 us |
| DiffTree() | 397.3 us | 34.7 us |
+-----------------------+-----------------+----------------+
```
Also fixes a latent bug: validate_tree_filter declared no restype, so ctypes truncated its returned tree pointer to 32 bits.