Page MenuHomeVyOS Platform
Create Task
Maniphest T6145

Service config-sync does not rely on priorities but must
Closed, ResolvedPublicBUG
Actions

Description

The service config-sync does not rely on priorities but must.
For example, if we have interface bridge and vrf sections for sync, they will be applied one by one.
The first section interfaces bridge and the second section vrf.
But they must rely on priorities

vyos@r15-left:~$ /opt/vyatta/sbin/priority.pl | match "vrf|bridge"
11 vrf
310 interfaces/bridge

Attaching an interface to the VRF is impossible if the VRF does not exist.

Initial config

set service config-sync mode 'load'
set service config-sync secondary address '192.168.122.11'
set service config-sync secondary key 'foo'
set service config-sync section interfaces bridge
set service config-sync section vrf
commit

Add some config on the primary node (on the secondary node no VRFs and bridge yet):

set interfaces bridge br0 description 'br0-eth1'
set interfaces bridge br0 member interface eth1
set interfaces bridge br0 vrf 'vrf-br'
set vrf name vrf-br table '1010'
commit

commit:

vyos@r15-left# commit
[['interfaces', 'bridge'], ['vrf']]
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.11

The log dump from the secondary node VRF "vrf-br" does not exist

Mar 20 16:08:38 r1-right sudo[5982]:     root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sh -c '/usr/sbin/vyshim VYOS_TAGNODE_VALUE=\'br0\' /usr/libexec/vyos/conf_mode/interfaces_bridge.py'
Mar 20 16:08:38 r1-right sudo[5982]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Mar 20 16:08:38 r1-right vyos-configd[4859]: Received message: {"type": "init"}
Mar 20 16:08:38 r1-right vyos-configd[4859]: config session pid is 3062
Mar 20 16:08:38 r1-right vyos-configd[4859]: Received message: {"type": "node", "data": "VYOS_TAGNODE_VALUE=br0/usr/libexec/vyos/conf_mode/interfaces_bridge.py"}
Mar 20 16:08:38 r1-right vyos-configd[4859]: VRF "vrf-br" does not exist
Mar 20 16:08:38 r1-right vyos-configd[4859]: Sending response 2
Mar 20 16:08:38 r1-right sudo[5982]: pam_unix(sudo:session): session closed for user root
Mar 20 16:08:39 r1-right systemd[1]: opt-vyatta-config-tmp-new_config_3062.mount: Deactivated successfully.
Mar 20 16:08:39 r1-right vyos-http-api[3062]: INFO:      - "POST /configure-section HTTP/1.0" 400 Bad Request
Mar 20 16:08:40 r1-right systemd[1]: opt-vyatta-config-tmp-new_config_3062.mount: Deactivated successfully.
Mar 20 16:08:40 r1-right vyos-http-api[3062]: Configuration modified via HTTP API using key 'KID'
Mar 20 16:08:40 r1-right vyos-http-api[3062]: INFO:      - "POST /configure-section HTTP/1.0" 200 OK
Mar 20 16:08:41 r1-right systemd[1]: opt-vyatta-config-tmp-new_config_3062.mount: Deactivated successfully.
Mar 20 16:08:41 r1-right vyos-http-api[3062]: Configuration modified via HTTP API using key 'KID'
Mar 20 16:08:41 r1-right vyos-http-api[3062]: INFO:      - "POST /configure-section HTTP/1.0" 200 OK
Mar 20 16:08:41 r1-right sudo[6124]:     root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sh -c '/usr/sbin/vyshim /usr/libexec/vyos/conf_mode/vrf.py'
Mar 20 16:08:41 r1-right sudo[6124]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Mar 20 16:08:41 r1-right vyos-configd[4859]: Received message: {"type": "init"}
Mar 20 16:08:41 r1-right vyos-configd[4859]: config session pid is 3062
Mar 20 16:08:41 r1-right vyos-configd[4859]: Received message: {"type": "node", "data": "/usr/libexec/vyos/conf_mode/vrf.py"}

Details

Difficulty level
Normal (likely a few hours)
Version
VyOS 1.5-rolling-202403200018, 1.4-rolling-202403200309
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Related Objects
Search...

Event Timeline

Viacheslav created this task.Mar 20 2024, 2:34 PM
Viacheslav added a parent task: T6121: Extend service config-sync for sections vpn, policy, vrf.Mar 20 2024, 2:50 PM
Viacheslav triaged this task as Normal priority.Mar 20 2024, 2:53 PM
jestabro changed the status of subtask T6146: Add python script to get all priorities of service or section from XML from Open to Backport candidate.Mar 22 2024, 12:56 PM
jestabro claimed this task.Mar 22 2024, 2:50 PM
jestabro added a comment.Mar 22 2024, 2:58 PM

Note that the subtask T6146 is useful in itself, as a modernization of the legacy tool priority.pl, but manual ordering of config-sync sections to address this issue is fragile, given the subtleties (== design constraints) of the legacy commit algorithm (cf. T5492). It is preferable to hand the full commit proposal of the primary node to the secondary node, and allow the underlying configsession to manage priority ordering --- this may be easy to do with a simple refactor in the post-commit hook and (if needed) adjustment in the http-api request handler.

pasik added a subscriber: pasik.Mar 22 2024, 3:14 PM
jestabro added a comment.Mar 23 2024, 2:03 AM

PR:
https://github.com/vyos/vyos-1x/pull/3172

Output for a standard check with section ['nat'] and for the test case above, with debug logging on; changes are correctly mirrored on secondary node:

vyos@vyos# set service config-sync mode 'load'
[edit]
vyos@vyos# set service config-sync secondary address '192.168.122.53'
[edit]
vyos@vyos# set service config-sync secondary key 'baz'
[edit]
vyos@vyos# set service config-sync section nat
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# set nat source rule 100 description 'rule 100'
[edit]
vyos@vyos# set nat source rule 100 outbound-interface name 'eth1'
[edit]
vyos@vyos# set nat source rule 100 translation address 'masquerade'
[edit]
vyos@vyos# commit
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.53
DEBUG:vyos_config_sync:Retrieved config for section '['nat']': {'source': {'rule': {'100': {'description': 'rule 100', 'outbound-interface': {'name': 'eth1'}, 'translation': {'address': 'masquerade'}}}}}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 192.168.122.53:443
DEBUG:urllib3.connectionpool:https://192.168.122.53:443 "POST /configure-section HTTP/1.1" 200 None
DEBUG:vyos_config_sync:Set config for sections '[['nat']]': {'success': True, 'data': None, 'error': None}
[edit]
vyos@vyos# delete nat
[edit]
vyos@vyos# commit
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.53
DEBUG:vyos_config_sync:Retrieved config for section '['nat']': {}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 192.168.122.53:443
DEBUG:urllib3.connectionpool:https://192.168.122.53:443 "POST /configure-section HTTP/1.1" 200 None
DEBUG:vyos_config_sync:Set config for sections '[['nat']]': {'success': True, 'data': None, 'error': None}
[edit]
vyos@vyos# set service config-sync section interfaces bridge
[edit]
vyos@vyos# set service config-sync section vrf
[edit]
vyos@vyos# commit
vyos@vyos# set interfaces bridge br0 description 'br0-eth1'
[edit]
vyos@vyos# set interfaces bridge br0 member interface eth1
[edit]
vyos@vyos# set interfaces bridge br0 vrf 'vrf-br'
[edit]
vyos@vyos# set vrf name vrf-br table '1010'
[edit]
vyos@vyos# commit
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.53
DEBUG:vyos_config_sync:Retrieved config for section '['interfaces', 'bridge']': {'br0': {'description': 'br0-eth1', 'member': {'interface': {'eth1': {}}}, 'vrf': 'vrf-br'}}
DEBUG:vyos_config_sync:Retrieved config for section '['nat']': {}
DEBUG:vyos_config_sync:Retrieved config for section '['vrf']': {'name': {'vrf-br': {'table': '1010'}}}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 192.168.122.53:443
DEBUG:urllib3.connectionpool:https://192.168.122.53:443 "POST /configure-section HTTP/1.1" 200 None
DEBUG:vyos_config_sync:Set config for sections '[['interfaces', 'bridge'], ['nat'], ['vrf']]': {'success': True, 'data': None, 'error': None}
[edit]
vyos@vyos# delete interfaces bridge 
[edit]
vyos@vyos# delete vrf
[edit]
vyos@vyos# commit
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.53
DEBUG:vyos_config_sync:Retrieved config for section '['interfaces', 'bridge']': {}
DEBUG:vyos_config_sync:Retrieved config for section '['nat']': {}
DEBUG:vyos_config_sync:Retrieved config for section '['vrf']': {}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 192.168.122.53:443
DEBUG:urllib3.connectionpool:https://192.168.122.53:443 "POST /configure-section HTTP/1.1" 200 None
DEBUG:vyos_config_sync:Set config for sections '[['interfaces', 'bridge'], ['nat'], ['vrf']]': {'success': True, 'data': None, 'error': None}
jestabro changed Difficulty level from Hard (possibly days) to Normal (likely a few hours).Mar 23 2024, 2:03 AM
jestabro changed the task status from Open to Backport candidate.Mar 23 2024, 4:28 PM
jestabro moved this task from Need Triage to Finished on the VyOS 1.5 Circinus board.
jestabro added a project: VyOS 1.4 Sagitta (1.4.0-epa3).Mar 24 2024, 3:15 PM
jestabro closed this task as Resolved.Mar 25 2024, 12:12 AM
jestabro moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta (1.4.0-epa3) board.