I found another subtle behavior change between the new swanctl setup in sagitta and the older ipsec.conf setup in equuleus. This prevents sagitta clients from initiating ipsec connections to equuleus hosts that are behind NAT (in my test setup, AWS elastic IP 1:1 NAT).
The test configuration is a hub-spoke VPN setup.
Hub: AWS host at 52.xx.xx.xx (private IP is 172.16.5.5), running very old equuleus rolling release (VyOS 1.3-rolling-202010090449—I am still working on getting newer releases to boot on AWS)
Spoke A: 108.xx.xx.xx, running equuleus (VyOS 1.3-rolling-202101052023)
Spoke B : 98.xx.xx.xx, running sagitta (VyOS 1.4-rolling-202109140217)
The hosts connect and authenticate using pre-shared keys. When spoke hosts connect to the hubs, they usually negotiate NAT-T due to the fact AWS always does 1:1 NAT with the public IPs assigned to EC2 hosts.
When the sagitta router restarts, or the ipsec process restarts, it fails to establish a tunnel to the hub router, receiving a failure message in its log:
Sep 15 05:37:31 lcn-router charon: 05[ENC] <peer_52-xx-xx-xx|1> parsed IKE_AUTH response 1 [ N(AUTH_FAILED) ] Sep 15 05:37:31 lcn-router charon: 05[IKE] <peer_52-xx-xx-xx|1> received AUTHENTICATION_FAILED notify error
Because both routers are set to try to initiate, restarting ipsec on the hub router causes it to try to initiate a connection which succeeds without issue.
The equuleus router has no issue establishing a connection in either direction.
In further log analysis, I found the IKE_AUTH messages generated by the spokes running sagitta and equuleus differ slightly:
sagitta: generating IKE_AUTH request 1 [ IDi AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_6_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ] equuleus: generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_6_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
It appears under the old ipsec.conf setup used by equuleus, the IKE daemon sends a remote ID attribute equal to the configured IP of the site-to-site peer automatically, even when one is not explicitly configured using authentication remote-id in the configuration.
Sagitta does not send the remote ID attribute unless I explicitly configure it in the configuration. (Adding set authentication remote-id 52.xx.xx.xx to my config fixed the issue and the connection established reliably).
This may be a fairly quirky use case since the AWS host has to contend to that 1:1 NAT which required some fiddling with the configuration to get the ipsec daemon to understand the IP in auth requests may be different from the IP it listens on (open to advice if there is a better way to configure this). However, there is clearly a subtle behavior change in sagitta we may want to paper over in some way (either via migration or changing how we render swanctl.conf). My understanding of the IKE RFCs is not robust enough to know exactly what the right behavior here should be.
Hub config:
esp-group VyOS-HS { compression disable lifetime 3600 mode tunnel pfs enable proposal 1 { encryption aes256 hash sha256 } } ike-group VyOS-HS { close-action none dead-peer-detection { action clear interval 15 timeout 30 } ikev2-reauth no key-exchange ikev2 lifetime 28800 proposal 1 { dh-group 21 encryption aes256 hash sha256 } } ... site-to-site { peer 98.xx.xx.xx { authentication { id 52.xx.xx.xx mode pre-shared-secret pre-shared-secret REDACTED } connection-type initiate ike-group VyOS-HS ikev2-reauth inherit local-address 172.16.5.5 vti { bind vti0 esp-group VyOS-HS } } peer 108.xx.xx.xx { authentication { id 52.xx.xx.xx mode pre-shared-secret pre-shared-secret REDACTED } connection-type initiate ike-group VyOS-HS ikev2-reauth inherit local-address 172.16.5.5 vti { bind vti1 esp-group VyOS-HS } } }
Spoke config (from sagitta router, other spoke config is identical except for change in local-address):
esp-group VyOS-HS { compression disable lifetime 3600 mode tunnel pfs enable proposal 1 { encryption aes256 hash sha256 } } ike-group VyOS-HS { close-action restart dead-peer-detection { action restart interval 15 timeout 30 } ikev2-reauth no key-exchange ikev2 lifetime 28800 proposal 1 { dh-group 21 encryption aes256 hash sha256 } } site-to-site { peer 52.xx.xx.xx { authentication { mode pre-shared-secret pre-shared-secret REDACTED } connection-type initiate ike-group VyOS-HS ikev2-reauth inherit local-address 98.xx.xx.xx vti { bind vti2 esp-group VyOS-HS } } }