Page MenuHomeVyOS Platform

Improve Configuration Load/Commit Speed by moving away from deep-tree flat-file backend
Open, NormalPublicFEATURE REQUEST

Description

Vyos seems to be using a flat-file deep-directory representation of the configuration in /opt/vyatta/config/* to represent the new/old configuration. For even mildly larger router configs, e.g., an IXP connected BGP router with several peers, this means that a relatively large directory tree gets created. Traversing that tree for configurations loading/committing takes significant time which is mostly spend in the config-fs mount.

Significant improvements could be gained by:

  • Moving to a pure memory representation of the configuration throughout the load/commit process, only flushing results to disk
  • Creating an, e.g., nested json representation (which would still be slow, but not that slow)
  • Using a DB or Key-Value store for the config (esp. given that the directory-paths make for unique keys, this might be an option)
  • [...]

Hence, moving config handling away from the current layout would be very much ideal; If done right, it could also come with a final read of the old configuration-style, i.e., could offer seamless migration.

Technically, in the same process, config validation could be touched; At the moment, e.g., bgp-community-regex validation, sequentially, for each instance of a community:

  • spawns /usr/libexec/vyos/validate-value
  • that spawns /bin/sh
  • that spawns python with validator script from /bin/sh

Leading to validation taking a _long_ time on an even mildly large config (~500 communities).

Details

Version
-
Is it a breaking change?
Perfectly compatible
Issue type
Performance optimization

Event Timeline

tfiebig updated the task description. (Show Details)
tfiebig updated the task description. (Show Details)
Viacheslav triaged this task as Normal priority.Apr 7 2024, 3:10 PM
dmbaturin subscribed.