IPSec op-mode show sa error
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	Viacheslav
	Apr 14 2021, 4:22 PM

Description

To reproduce this bug we need 2 nodes a 2 vti interfaces.
One of the peers should be configured with a random ESP-group parameters for vti, another with correct.
i.e we should to set working 1 vti interface and 1 not working

Both tunnels in down state (disabled eth1)

vyos@vyos:~$ show vpn ipsec sa
Connection                   State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal
---------------------------  -------  --------  --------------  ----------------  ----------------  -----------  ----------
peer-192.0.2.2-tunnel-vti    down     N/A       N/A             N/A               N/A               N/A          N/A
peer-203.0.113.2-tunnel-vti  down     N/A       N/A             N/A               N/A               N/A          N/A

Enable eth1, so one of the tunnels is "up"

vyos@vyos# del interfaces ethernet eth1 disable
[edit]
vyos@vyos# commit
[edit]

vyos@vyos:~$ show vpn ipsec sa
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/show_ipsec_sa.py", line 111, in <module>
    sa_data = sorted(sa_data, key=lambda peer: peer[0])
TypeError: '<' not supported between instances of 'str' and 'bytes'
vyos@vyos:~$ 
vyos@vyos:~$

https://github.com/vyos/vyos-1x/blob/29d0e4aff88944820aa3f635b1672f03f0e89127/src/op_mode/show_ipsec_sa.py#L77

Left Router config

set interfaces ethernet eth1 address '192.0.2.1/30'
set interfaces ethernet eth1 address '203.0.113.1/30'
set interfaces ethernet eth1 address '100.64.0.1/30'
set interfaces vti vti0 address '10.0.1.1/30'
set interfaces vti vti1 address '10.0.0.1/30'
set vpn ipsec esp-group ESP-GRP-VTI_down compression 'disable'
set vpn ipsec esp-group ESP-GRP-VTI_down lifetime '1800'
set vpn ipsec esp-group ESP-GRP-VTI_down mode 'tunnel'
set vpn ipsec esp-group ESP-GRP-VTI_down pfs 'enable'
set vpn ipsec esp-group ESP-GRP-VTI_down proposal 1 encryption 'aes256'
set vpn ipsec esp-group ESP-GRP-VTI_down proposal 1 hash 'sha1'
set vpn ipsec esp-group ESP-GRP-VTI_up compression 'disable'
set vpn ipsec esp-group ESP-GRP-VTI_up lifetime '1800'
set vpn ipsec esp-group ESP-GRP-VTI_up mode 'tunnel'
set vpn ipsec esp-group ESP-GRP-VTI_up pfs 'enable'
set vpn ipsec esp-group ESP-GRP-VTI_up proposal 1 encryption 'aes256'
set vpn ipsec esp-group ESP-GRP-VTI_up proposal 1 hash 'sha1'
set vpn ipsec ike-group IKE-GRP-VTI_down close-action 'none'
set vpn ipsec ike-group IKE-GRP-VTI_down ikev2-reauth 'no'
set vpn ipsec ike-group IKE-GRP-VTI_down key-exchange 'ikev1'
set vpn ipsec ike-group IKE-GRP-VTI_down lifetime '3600'
set vpn ipsec ike-group IKE-GRP-VTI_down proposal 1 dh-group '2'
set vpn ipsec ike-group IKE-GRP-VTI_down proposal 1 encryption 'aes256'
set vpn ipsec ike-group IKE-GRP-VTI_down proposal 1 hash 'sha1'
set vpn ipsec ike-group IKE-GRP-VTI_up close-action 'none'
set vpn ipsec ike-group IKE-GRP-VTI_up ikev2-reauth 'no'
set vpn ipsec ike-group IKE-GRP-VTI_up key-exchange 'ikev1'
set vpn ipsec ike-group IKE-GRP-VTI_up lifetime '3600'
set vpn ipsec ike-group IKE-GRP-VTI_up proposal 1 dh-group '2'
set vpn ipsec ike-group IKE-GRP-VTI_up proposal 1 encryption 'aes256'
set vpn ipsec ike-group IKE-GRP-VTI_up proposal 1 hash 'sha1'
set vpn ipsec ipsec-interfaces interface 'eth1'
set vpn ipsec site-to-site peer 192.0.2.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 192.0.2.2 authentication pre-shared-secret 'SeCrEt'
set vpn ipsec site-to-site peer 192.0.2.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 192.0.2.2 ike-group 'IKE-GRP-VTI_down'
set vpn ipsec site-to-site peer 192.0.2.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 192.0.2.2 local-address '192.0.2.1'
set vpn ipsec site-to-site peer 192.0.2.2 vti bind 'vti1'
set vpn ipsec site-to-site peer 192.0.2.2 vti esp-group 'ESP-GRP-VTI_down'
set vpn ipsec site-to-site peer 203.0.113.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 203.0.113.2 authentication pre-shared-secret 'SeCrEt'
set vpn ipsec site-to-site peer 203.0.113.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 203.0.113.2 ike-group 'IKE-GRP-VTI_up'
set vpn ipsec site-to-site peer 203.0.113.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 203.0.113.2 local-address '203.0.113.1'
set vpn ipsec site-to-site peer 203.0.113.2 vti bind 'vti0'
set vpn ipsec site-to-site peer 203.0.113.2 vti esp-group 'ESP-GRP-VTI_up'

Right Router Config

set interfaces ethernet eth1 address '192.0.2.2/30'
set interfaces ethernet eth1 address '203.0.113.2/30'
set interfaces ethernet eth1 address '100.64.0.2/30'
set interfaces vti vti0 address '10.0.1.2/30'
set interfaces vti vti1 address '10.0.0.2/30'
set vpn ipsec esp-group ESP-GRP-VTI_down compression 'disable'
set vpn ipsec esp-group ESP-GRP-VTI_down lifetime '1800'
set vpn ipsec esp-group ESP-GRP-VTI_down mode 'tunnel'
set vpn ipsec esp-group ESP-GRP-VTI_down pfs 'disable'
set vpn ipsec esp-group ESP-GRP-VTI_down proposal 1 encryption 'aes256'
set vpn ipsec esp-group ESP-GRP-VTI_down proposal 1 hash 'sha384'
set vpn ipsec esp-group ESP-GRP-VTI_up compression 'disable'
set vpn ipsec esp-group ESP-GRP-VTI_up lifetime '1800'
set vpn ipsec esp-group ESP-GRP-VTI_up mode 'tunnel'
set vpn ipsec esp-group ESP-GRP-VTI_up pfs 'enable'
set vpn ipsec esp-group ESP-GRP-VTI_up proposal 1 encryption 'aes256'
set vpn ipsec esp-group ESP-GRP-VTI_up proposal 1 hash 'sha1'
set vpn ipsec ike-group IKE-GRP-VTI_down ikev2-reauth 'no'
set vpn ipsec ike-group IKE-GRP-VTI_down key-exchange 'ikev1'
set vpn ipsec ike-group IKE-GRP-VTI_down lifetime '3600'
set vpn ipsec ike-group IKE-GRP-VTI_down proposal 1 encryption 'aes256'
set vpn ipsec ike-group IKE-GRP-VTI_down proposal 1 hash 'sha1'
set vpn ipsec ike-group IKE-GRP-VTI_up ikev2-reauth 'no'
set vpn ipsec ike-group IKE-GRP-VTI_up key-exchange 'ikev1'
set vpn ipsec ike-group IKE-GRP-VTI_up lifetime '3600'
set vpn ipsec ike-group IKE-GRP-VTI_up proposal 1 encryption 'aes256'
set vpn ipsec ike-group IKE-GRP-VTI_up proposal 1 hash 'sha1'
set vpn ipsec ipsec-interfaces interface 'eth1'
set vpn ipsec site-to-site peer 192.0.2.1 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 192.0.2.1 authentication pre-shared-secret 'SeCrEt'
set vpn ipsec site-to-site peer 192.0.2.1 ike-group 'IKE-GRP-VTI_down'
set vpn ipsec site-to-site peer 192.0.2.1 local-address '192.0.2.2'
set vpn ipsec site-to-site peer 192.0.2.1 vti bind 'vti1'
set vpn ipsec site-to-site peer 192.0.2.1 vti esp-group 'ESP-GRP-VTI_down'
set vpn ipsec site-to-site peer 203.0.113.1 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 203.0.113.1 authentication pre-shared-secret 'SeCrEt'
set vpn ipsec site-to-site peer 203.0.113.1 ike-group 'IKE-GRP-VTI_up'
set vpn ipsec site-to-site peer 203.0.113.1 local-address '203.0.113.2'
set vpn ipsec site-to-site peer 203.0.113.1 vti bind 'vti0'
set vpn ipsec site-to-site peer 203.0.113.1 vti esp-group 'ESP-GRP-VTI_up'

Details

Difficulty level: Normal (likely a few hours)
Version: VyOS 1.4-rolling-202104132216
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Perfectly compatible

Related Objects

Mentioned In: T3333: "show vpn ipsec sa" reports ESP tunnels to be up when they are not.

Event Timeline

Viacheslav created this task.Apr 14 2021, 4:22 PM

Viacheslav updated the task description. (Show Details)Apr 14 2021, 4:24 PM

Version: VyOS 1.4-rolling-202104151445

Test I:
In single tunnel, if the esp parameters doesn't match then the command output show vpn ipsec sa shows down which is desired result.

Test II:
But when tested with multiple vti tunnels, configured for different interfaces, one vti tunnel has matched esp parameters, other has mismatched parameters, then this error is received which doesn't seem to be right.

vyos@vyos:~$ sh vpn ipsec sa
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/show_ipsec_sa.py", line 110, in <module>
    sa_data = sorted(sa_data, key=lambda peer: peer[0])
TypeError: '<' not supported between instances of 'bytes' and 'str'

If the ike sa is not established, then it does not throw the error:

Peer ID / IP                            Local ID / IP
------------                            -------------
172.26.4.2                              172.26.4.1
    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    down   N/A     n/a      n/a     n/a(n/a)       no     0       n/a
vyos@vyos:~$ sh vpn ipsec sa
Connection                State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal
------------------------  -------  --------  --------------  ----------------  ----------------  -----------  -----------------------------
peer-10.3.0.2-tunnel-vti  up       9m7s      0B/0B           0/0               10.3.0.2          N/A          AES_CBC_256/HMAC_SHA2_512_256

Test II:

If both the vti tunnels are configured in the single interface, one vti tunnel has matched esp parameters, other has mismatched parameters, then it brings both tunnels to down state and also looses the reachability of the interfaces.

R1:

vyos@vyos:~$ sh vpn ipsec sa
Connection                   State    Uptime    Bytes In/Out    Packets In/Out    Rl
---------------------------  -------  --------  --------------  ----------------  --
peer-10.2.0.2-tunnel-vti     down     N/A       N/A             N/A               NA
peer-203.0.113.1-tunnel-vti  down     N/A       N/A             N/A               NA
vyos@vyos:~$ sh vpn ike sa
Peer ID / IP                            Local ID / IP
------------                            -------------
10.2.0.2                                10.3.0.2

    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    down   IKEv1   aes256   sha512_256 14(MODP_2048)  no     0       n/a


Peer ID / IP                            Local ID / IP
------------                            -------------
203.0.113.1                             100.64.0.1

    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    down   IKEv1   n/a      n/a     n/a(n/a)       no     0       n/a

R2:

vyos@vyos:~$ show vpn ipsec sa
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/show_ipsec_sa.py", line 111, in <module>
    sa_data = sorted(sa_data, key=lambda peer: peer[0])
TypeError: '<' not supported between instances of 'bytes' and 'str'
vyos@vyos:~$ sh vpn ike sa
Peer ID / IP                            Local ID / IP
------------                            -------------
10.3.0.2                                10.2.0.2

    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    down   IKEv1   aes256   sha512_256 14(MODP_2048)  no     0       n/a


Peer ID / IP                            Local ID / IP
------------                            -------------
100.64.0.1                              203.0.113.1

    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    down   IKEv1   aes256   sha512_256 14(MODP_2048)  no     0       n/a


Peer ID / IP                            Local ID / IP
------------                            -------------
172.26.4.2                              172.26.4.1

    State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
    -----  ------  -------  ----    ---------      -----  ------  ------
    up     IKEv1   aes256   sha512_256 14(MODP_2048)  no     7200    28800

Configuration:

R1:

set interfaces ethernet eth2 address '10.3.0.2/24'
set interfaces ethernet eth2 address '100.64.0.1/30'
set interfaces vti vti0 address '172.16.0.2/24'
set interfaces vti vti3 address '10.0.1.2/30'
set vpn ipsec esp-group Aesp compression 'disable'
set vpn ipsec esp-group Aesp lifetime '3600'
set vpn ipsec esp-group Aesp mode 'tunnel'
set vpn ipsec esp-group Aesp pfs 'dh-group14'
set vpn ipsec esp-group Aesp proposal 1 encryption 'aes256'
set vpn ipsec esp-group Aesp proposal 1 hash 'sha512'
set vpn ipsec esp-group espA compression 'disable'
set vpn ipsec esp-group espA lifetime '3600'
set vpn ipsec esp-group espA mode 'tunnel'
set vpn ipsec esp-group espA pfs 'disable'
set vpn ipsec esp-group espA proposal 1 encryption 'aes256'
set vpn ipsec esp-group espA proposal 1 hash 'sha512'
set vpn ipsec ike-group ikeA close-action 'none'
set vpn ipsec ike-group ikeA ikev2-reauth 'no'
set vpn ipsec ike-group ikeA key-exchange 'ikev1'
set vpn ipsec ike-group ikeA lifetime '28800'
set vpn ipsec ike-group ikeA proposal 1 dh-group '14'
set vpn ipsec ike-group ikeA proposal 1 encryption 'aes256'
set vpn ipsec ike-group ikeA proposal 1 hash 'sha512'
set vpn ipsec ipsec-interfaces interface 'eth2'
set vpn ipsec site-to-site peer 10.2.0.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 10.2.0.2 authentication pre-shared-secret 'vyos'
set vpn ipsec site-to-site peer 10.2.0.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 10.2.0.2 default-esp-group 'espA'
set vpn ipsec site-to-site peer 10.2.0.2 ike-group 'ikeA'
set vpn ipsec site-to-site peer 10.2.0.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 10.2.0.2 local-address '10.3.0.2'
set vpn ipsec site-to-site peer 10.2.0.2 vti bind 'vti0'
set vpn ipsec site-to-site peer 10.2.0.2 vti esp-group 'espA'
set vpn ipsec site-to-site peer 203.0.113.1 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 203.0.113.1 authentication pre-shared-secret 'vyos'
set vpn ipsec site-to-site peer 203.0.113.1 connection-type 'initiate'
set vpn ipsec site-to-site peer 203.0.113.1 default-esp-group 'Aesp'
set vpn ipsec site-to-site peer 203.0.113.1 ike-group 'ikeA'
set vpn ipsec site-to-site peer 203.0.113.1 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 203.0.113.1 local-address '100.64.0.1'
set vpn ipsec site-to-site peer 203.0.113.1 vti bind 'vti3'
set vpn ipsec site-to-site peer 203.0.113.1 vti esp-group 'Aesp'

R2:

set interfaces ethernet eth1 address '10.2.0.2/24'
set interfaces ethernet eth1 address '203.0.113.1/30'
set interfaces ethernet eth2 address '172.26.4.1/16'
set interfaces vti vti0 address '172.16.0.1/24'
set interfaces vti vti1 address '10.0.0.1/30'
set interfaces vti vti3 address '10.0.1.1/30'
set vpn ipsec esp-group espA compression 'disable'
set vpn ipsec esp-group espA lifetime '3600'
set vpn ipsec esp-group espA mode 'tunnel'
set vpn ipsec esp-group espA pfs 'disable'
set vpn ipsec esp-group espA proposal 1 encryption 'aes256'
set vpn ipsec esp-group espA proposal 1 hash 'sha512'
set vpn ipsec ike-group ikeA close-action 'none'
set vpn ipsec ike-group ikeA ikev2-reauth 'no'
set vpn ipsec ike-group ikeA key-exchange 'ikev1'
set vpn ipsec ike-group ikeA lifetime '28800'
set vpn ipsec ike-group ikeA proposal 1 dh-group '14'
set vpn ipsec ike-group ikeA proposal 1 encryption 'aes256'
set vpn ipsec ike-group ikeA proposal 1 hash 'sha512'
set vpn ipsec ipsec-interfaces interface 'eth1'
set vpn ipsec ipsec-interfaces interface 'eth2'
set vpn ipsec site-to-site peer 10.3.0.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 10.3.0.2 authentication pre-shared-secret 'vyos'
set vpn ipsec site-to-site peer 10.3.0.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 10.3.0.2 default-esp-group 'espA'
set vpn ipsec site-to-site peer 10.3.0.2 ike-group 'ikeA'
set vpn ipsec site-to-site peer 10.3.0.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 10.3.0.2 local-address '10.2.0.2'
set vpn ipsec site-to-site peer 10.3.0.2 vti bind 'vti0'
set vpn ipsec site-to-site peer 10.3.0.2 vti esp-group 'espA'
set vpn ipsec site-to-site peer 100.64.0.1 authentication mode 'pre-shared-secr'
set vpn ipsec site-to-site peer 100.64.0.1 authentication pre-shared-secret 'vy'
set vpn ipsec site-to-site peer 100.64.0.1 connection-type 'initiate'
set vpn ipsec site-to-site peer 100.64.0.1 default-esp-group 'espA'
set vpn ipsec site-to-site peer 100.64.0.1 ike-group 'ikeA'
set vpn ipsec site-to-site peer 100.64.0.1 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 100.64.0.1 local-address '203.0.113.1'
set vpn ipsec site-to-site peer 100.64.0.1 vti bind 'vti3'
set vpn ipsec site-to-site peer 100.64.0.1 vti esp-group 'espA'
set vpn ipsec site-to-site peer 172.26.4.2 authentication mode 'pre-shared-secr'
set vpn ipsec site-to-site peer 172.26.4.2 authentication pre-shared-secret 'vy'
set vpn ipsec site-to-site peer 172.26.4.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 172.26.4.2 default-esp-group 'espA'
set vpn ipsec site-to-site peer 172.26.4.2 ike-group 'ikeA'
set vpn ipsec site-to-site peer 172.26.4.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 172.26.4.2 local-address '172.26.4.1'
set vpn ipsec site-to-site peer 172.26.4.2 vti bind 'vti1'
set vpn ipsec site-to-site peer 172.26.4.2 vti esp-group 'espA'

Observation:
If I correct the config, it does not bring the connection status automatically, even a restart of vpn service did not help, only reboot worked

https://github.com/vyos/vyos-1x/blob/29d0e4aff88944820aa3f635b1672f03f0e89127/src/op_mode/show_ipsec_sa.py#L111

Can someone modify this sentence and add an output statement before it to see what caused the error?

pasik added a subscriber: pasik.Apr 17 2021, 3:33 PM

SrividyaA mentioned this in T3333: "show vpn ipsec sa" reports ESP tunnels to be up when they are not..Apr 20 2021, 1:12 PM

sa_data wrong format

vyos@r6-roll:~$ show vpn ipsec sa
[[b'peer-203.0.113.2-tunnel-vti',
  'up',
  '4m33s',
  '168B/168B',
  '2/2',
  '203.0.113.2',
  'N/A',
  'AES_CBC_256/HMAC_SHA1_96/MODP_1024'],
 ['peer-192.0.2.2-tunnel-vti',
  'down',
  'N/A',
  'N/A',
  'N/A',
  'N/A',
  'N/A',
  'N/A']]
Connection                      State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal
------------------------------  -------  --------  --------------  ----------------  ----------------  -----------  ----------------------------------
b'peer-203.0.113.2-tunnel-vti'  up       4m33s     168B/168B       2/2               203.0.113.2       N/A          AES_CBC_256/HMAC_SHA1_96/MODP_1024
peer-192.0.2.2-tunnel-vti       down     N/A       N/A             N/A               N/A               N/A          N/A
vyos@r6-roll:~$

PR https://github.com/vyos/vyos-1x/pull/820

Viacheslav claimed this task.Apr 27 2021, 3:51 PM

Viacheslav changed the task status from Open to Needs testing.Apr 27 2021, 5:28 PM

Viacheslav closed this task as Resolved.Apr 28 2021, 2:16 PM

this error also occure in VyOS 1.3 RC4.

@rob it fixed in the latest 1.3 with commit https://github.com/vyos/vyos-1x/commit/c7430fbb8738d76e63a6972b7399fa39572e2254
probably just not hit at that time in 1.3-rc4

c-po moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.May 24 2021, 7:39 PM

IPSec op-mode show sa errorClosed, ResolvedPublicBUGActions

Description

Details

Related Objects

Event Timeline

IPSec op-mode show sa error
Closed, ResolvedPublicBUG
Actions