Page MenuHomeVyOS Platform
Create Task
Maniphest T7439

StrongSwan ESP reauthentication causes traffic to be unable to traverse through IPsec Tunnel
Open, Requires assessmentPublicBUG
Actions

Description

We are currently seeing an issue with IPsec tunnels where after the ESP Lifetime expires traffic is unable to route down the tunnel to it's intended destination down the VTI interface. The only way to make traffic traverse down the tunnel is to restart the IPsec process, and this then allows the expected behaviour for the ESP lifetime, then once this expires, traffic stops working again.

We believe this is a VyOS issue as after the ESP time out the IPsec tunnel remains up and traffic is still sent down the tunnel, however this traffic never reaches the other side of the tunnel, and we also do not see any back and forth ESP traffic being sent to the IPsec tunnel's remote gateway after the ESP timeout and can only see this after restarting the IPsec process on the VyOS device.

When we monitor traffic, it still goes out via the vti interfaces that is binded to the site-to-site VPN however when we monitor for ESP packets towards the public endpoint of the VPN tunnel, this stops after the lifetime expires.

We also would expect that the SA should restart after the lifetime (in our case its set to 60 seconds for testing but in production this would be 3600), we just see the VPN SA continue showing as up (longer than 60 seconds) but the ESP traffic stops, we have also set a keep alive on the VPN but with this being IKEv2 I don't believe this feature works as per the documentation.

Packet capture when traffic is working;

vti0:

root@csg19:/home/jack.littlewood# tcpdump -i vti0
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vti0, link-type RAW (Raw IP), snapshot length 262144 bytes
11:56:20.171172 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [S], seq 2994231420, win 28720, options [mss 1300,sackOK,TS val 1174863400 ecr 0,nop,wscale 7], length 0
11:56:20.171360 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49955: Flags [S.], seq 2018934677, ack 2994231421, win 42340, options [mss 1460,nop,nop,sackOK,nop,wscale 12], length 0
11:56:20.182959 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [.], ack 1, win 225, length 0
11:56:20.183110 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [P.], seq 1:93, ack 1, win 225, length 92: HTTP: GET / HTTP/1.1
11:56:20.183204 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49955: Flags [.], ack 93, win 11, length 0
11:56:20.183529 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49955: Flags [P.], seq 1:278, ack 93, win 11, length 277: HTTP: HTTP/1.1 302 Found
11:56:20.195067 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [.], ack 278, win 233, length 0
11:56:20.196704 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [F.], seq 93, ack 278, win 233, length 0
11:56:20.196851 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49955: Flags [F.], seq 278, ack 94, win 11, length 0
11:56:20.208375 IP 172.24.48.128.49955 > page.surfprotect.exa.net.uk.http: Flags [.], ack 279, win 233, length 0
11:56:22.806908 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [S], seq 3151778621, win 28720, options [mss 1300,sackOK,TS val 1174863664 ecr 0,nop,wscale 7], length 0
11:56:22.807083 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.57117: Flags [S.], seq 2313620486, ack 3151778622, win 42340, options [mss 1460,nop,nop,sackOK,nop,wscale 12], length 0
11:56:22.818644 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [.], ack 1, win 225, length 0
11:56:22.818809 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [P.], seq 1:93, ack 1, win 225, length 92: HTTP: GET / HTTP/1.1
11:56:22.818901 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.57117: Flags [.], ack 93, win 11, length 0
11:56:22.819299 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.57117: Flags [P.], seq 1:278, ack 93, win 11, length 277: HTTP: HTTP/1.1 302 Found
11:56:22.830830 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [.], ack 278, win 233, length 0
11:56:22.831222 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [F.], seq 93, ack 278, win 233, length 0
11:56:22.831350 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.57117: Flags [F.], seq 278, ack 94, win 11, length 0
11:56:22.842862 IP 172.24.48.128.57117 > page.surfprotect.exa.net.uk.http: Flags [.], ack 279, win 233, length 0
11:56:23.911252 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [S], seq 2577670235, win 28720, options [mss 1300,sackOK,TS val 1174863774 ecr 0,nop,wscale 7], length 0
11:56:23.911426 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49063: Flags [S.], seq 1832504976, ack 2577670236, win 42340, options [mss 1460,nop,nop,sackOK,nop,wscale 12], length 0
11:56:23.923070 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [.], ack 1, win 225, length 0
11:56:23.923226 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [P.], seq 1:93, ack 1, win 225, length 92: HTTP: GET / HTTP/1.1
11:56:23.923308 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49063: Flags [.], ack 93, win 11, length 0
11:56:23.923758 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49063: Flags [P.], seq 1:278, ack 93, win 11, length 277: HTTP: HTTP/1.1 302 Found
11:56:23.935294 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [.], ack 278, win 233, length 0
11:56:23.935769 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [F.], seq 93, ack 278, win 233, length 0
11:56:23.935910 IP page.surfprotect.exa.net.uk.http > 172.24.48.128.49063: Flags [F.], seq 278, ack 94, win 11, length 0
11:56:23.947449 IP 172.24.48.128.49063 > page.surfprotect.exa.net.uk.http: Flags [.], ack 279, win 233, length 0

eth0/1(WAN):

root@csg19:/home/jack.littlewood# tcpdump -i any 'esp'
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
11:56:20.171194 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x1), length 100
11:56:20.171350 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x1), length 100
11:56:20.182969 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x2), length 84
11:56:20.183115 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x3), length 180
11:56:20.183196 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x2), length 84
11:56:20.183523 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x3), length 356
11:56:20.195080 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x4), length 84
11:56:20.196712 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x5), length 84
11:56:20.196843 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x4), length 84
11:56:20.208382 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x6), length 84
11:56:22.806927 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x7), length 100
11:56:22.807073 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x5), length 100
11:56:22.818650 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x8), length 84
11:56:22.818815 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x9), length 180
11:56:22.818896 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x6), length 84
11:56:22.819295 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x7), length 356
11:56:22.830837 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xa), length 84
11:56:22.831227 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xb), length 84
11:56:22.831343 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x8), length 84
11:56:22.842871 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xc), length 84
11:56:23.911270 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xd), length 100
11:56:23.911416 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0x9), length 100
11:56:23.923077 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xe), length 84
11:56:23.923230 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0xf), length 180
11:56:23.923303 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0xa), length 84
11:56:23.923752 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0xb), length 356
11:56:23.935303 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x10), length 84
11:56:23.935774 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x11), length 84
11:56:23.935900 eth0  In  IP 82.219.26.244 > 82.219.122.120: ESP(spi=0xc349551f,seq=0xc), length 84
11:56:23.947457 eth1  Out IP 82.219.122.120 > 82.219.26.244: ESP(spi=0xc370571a,seq=0x12), length 84

Packet capture when traffic is broken;

vti0:

root@csg19:/home/jack.littlewood# tcpdump -i vti0
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vti0, link-type RAW (Raw IP), snapshot length 262144 bytes
11:54:44.434520 IP 172.24.48.128.42271 > page.surfprotect.exa.net.uk.http: Flags [S], seq 4024688645, win 28720, options [mss 1300,sackOK,TS val 1174853826 ecr 0,nop,wscale 7], length 0
11:54:49.282686 IP 172.24.48.128.59333 > page.surfprotect.exa.net.uk.http: Flags [S], seq 3640303055, win 28720, options [mss 1300,sackOK,TS val 1174854311 ecr 0,nop,wscale 7], length 0
11:54:49.974029 IP 172.24.48.128.48929 > page.surfprotect.exa.net.uk.http: Flags [S], seq 3388062991, win 28720, options [mss 1300,sackOK,TS val 1174854380 ecr 0,nop,wscale 7], length 0
11:54:50.737946 IP 172.24.48.128.33129 > page.surfprotect.exa.net.uk.http: Flags [S], seq 3904427069, win 28720, options [mss 1300,sackOK,TS val 1174854457 ecr 0,nop,wscale 7], length 0
11:54:51.465052 IP 172.24.48.128.53051 > page.surfprotect.exa.net.uk.http: Flags [S], seq 1668219073, win 28720, options [mss 1300,sackOK,TS val 1174854529 ecr 0,nop,wscale 7], length 0
11:54:52.182279 IP 172.24.48.128.44197 > page.surfprotect.exa.net.uk.http: Flags [S], seq 2340681241, win 28720, options [mss 1300,sackOK,TS val 1174854601 ecr 0,nop,wscale 7], length 0
11:54:52.851438 IP 172.24.48.128.35155 > page.surfprotect.exa.net.uk.http: Flags [S], seq 2628254242, win 28720, options [mss 1300,sackOK,TS val 1174854668 ecr 0,nop,wscale 7], length 0
11:54:53.381413 IP 172.24.48.128.52183 > page.surfprotect.exa.net.uk.http: Flags [S], seq 468708520, win 28720, options [mss 1300,sackOK,TS val 1174854721 ecr 0,nop,wscale 7], length 0

eth0/1(WAN):

root@csg19:/home/jack.littlewood# tcpdump -i any 'esp'
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes

Full VPN configuration

daniel.bertram@csg19.tcw.man# edit vpn ipsec
[edit vpn ipsec]
daniel.bertram@csg19.tcw.man# show
 esp-group SPQ {
     lifetime 60
     mode tunnel
     pfs disable
     proposal 1 {
         encryption aes128
         hash sha1
     }
 }
 ike-group SPQ {
     close-action restart
     dead-peer-detection {
         action restart
         interval 5
         timeout 60
     }
     ikev2-reauth
     key-exchange ikev2
     lifetime 60
     proposal 1 {
         dh-group 14
         encryption aes128
         hash sha1
     }
 }
 interface lo
 options {
     disable-route-autoinstall
     interface lo
     virtual-ip
 }
 site-to-site {
     peer OUR-0005 {
         authentication {
             mode x509
             x509 {
                 ca-certificate OUR-0005
                 certificate OUR-0005
             }
         }
         connection-type initiate
         default-esp-group SPQ
         ike-group SPQ
         local-address 82.219.122.120
         remote-address 82.219.26.244
         vti {
             bind vti0
         }
     }
 }
[edit vpn ipsec]
daniel.bertram@csg19.tcw.man#

Can you look into this and advise further?

Details

Version
VyOS 1.5-rolling-202501210804
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

joseph.oshaughnessy created this task.Thu, May 8, 10:59 AM
Viacheslav updated the task description. (Show Details)Thu, May 8, 11:42 AM
pasik subscribed.Thu, May 8, 12:32 PM
Viacheslav closed this task as Invalid.Thu, May 8, 5:37 PM
Viacheslav subscribed.

It looks more like a support request, ask on the forum.

joseph.oshaughnessy reopened this task as Open.Tue, May 13, 10:43 AM

Hi Viacheslav

This is not a support request, this is a bug report as VyOS is not behaving as expected and looks to be an issue with Strong Swan on VyOS based devices.

Viacheslav added a comment.Wed, May 14, 6:34 AM

This way we will expect the set of commands from both sites to reproduce.
If there will be incorrect config settings as in example you provided, the task will be closed as invalid. We do not provide support via forge and will not say you what to change. There is a forum for these questions.