Page MenuHomeVyOS Platform

Config load fails in ConfigTree with ValueError: Failed to parse config: lexing: empty token
Closed, ResolvedPublic

Description

A config file that loads fine in 1.2-rolling-20190623 fails to load after somewhere around 1.2-rolling-201908* with the error:

load /mnt/vyos-config/config.boot-busted
Traceback (most recent call last):
  File "/opt/vyatta/etc/config-migrate/migrate/dns-forwarding/0-to-1", line 34, in <module>
    config = ConfigTree(config_file)
  File "/usr/lib/python3/dist-packages/vyos/configtree.py", line 167, in __init__
    raise ValueError("Failed to parse config: {0}".format(msg))
ValueError: Failed to parse config: lexing: empty token
Loading configuration from '/mnt/vyos-config/config.boot-busted'
Called process error: Command '['/opt/vyatta/etc/config-migrate/migrate/dns-forwarding/0-to-1', '/tmp/tmpuyo4w_hq']' returned non-zero exit status 1..
[edit]

I first encountered this on an upgrade but it also fails when loading the saved config into the latest 1.3-rolling-201910180142.
Looking at the relevant parts of ConfigTree reveals that it's a wrapper around a compiled C library so I'm at a loss at how to debug this.
I can't provide the unredacted config for testing unfortunately so I can't provide a way to reproduce this. I have the redacted config with all IPs changed with x.x.x.x that I can provide but it's quite large so not possible to manually make usable IMO.
I tried to test it via python console but I can't get the module to work (in python3 ran >>> c = vyos.configtree.ConfigTree(""" desc "a->b" """), I get NameError: name 'vyos' is not defined)

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

jjakob created this object in space S1 VyOS Public.

Via the process of elimination I found the culprit - comments with special characters in them, that at some point in the upgrade cycle got converted into backslash-escped decimal:

description "\197\160" # this was once Š
description "\196\140 \196\141" # Č č

\197\160 is C5 A0 which is U+0160, which was the original character.
So old versions allowed these characters in comments and worked fine with them, then at one point an upgrade converted them to backslash decimal UTF-8, which still worked until at some point they started causing the error.

syncer changed the task status from Open to Needs testing.Nov 16 2019, 11:10 PM
syncer assigned this task to zsdc.
syncer triaged this task as Normal priority.
syncer removed a project: VyOS 1.2 Crux.

The replacement of special characters with backslash decimal happened on upgrade from 1.2.0-rc11 to 1.2.0-rolling+201906231514.

The resolution in T1801 will prevent the error in this case.

jjakob moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.
erkin set Issue type to Bug (incorrect behavior).Aug 31 2021, 6:34 PM