Page MenuHomeVyOS Platform
Create Task
Maniphest T8228

DMVPN IPSec does not initiate if the local NBMA address is found none during the bootup
Closed, ResolvedPublicBUG
Actions

Description

This issue occurs after the device is rebooted when the source IP address is learned via DHCP with a delay. During initialization, the OpenNHRP script attempts to validate both the source and destination NBMA addresses. Because the required information is not available at that time, the script does not proceed with initiating the IKE_SA session and does not recheck or reinitiate it later.

opennhrp-script.py[5947]: Running script with arguments: ['/etc/opennhrp/opennhrp-script.py', 'peer-up'], environment: environ({'NHRP_TYPE': 'static'>
opennhrp-script.py[5947]: Peer UP event for spoke using IKE profile dmvpn-NHRPVPN-tun100
opennhrp-script.py[5947]: Can not get NHRP NBMA addresses: local None, remote 192.168.0.2
staticd[1640]: [S4MGP-4WQTA] route_notify_owner: Route 0.0.0.0/0 failed to install for table: 254
opennhrp-script.py: Can not get NHRP NBMA addresses: local None, remote 192.168.0.2
opennhrp-script.py: Can not get NHRP NBMA addresses: local None, remote 192.168.0.2
staticd[1636]: [S4MGP-4WQTA] route_notify_owner: Route 171.200.0.0/16 failed to install for table: 254

IPSec messages:

systemd[1]: Starting strongSwan IPsec IKEv1/IKEv2 daemon using swanctl...
charon[5408]: 00[DMN] Starting charon-systemd IKE daemon (strongSwan 5.9.11, Linux 6.6.117-amd64-vyos, x86_64)
charon[5408]: 00[CFG] PKCS11 module '<name>' lacks library path
charon[5408]: 00[PTS] TPM 2.0 - could not load "libtss2-tcti-tabrmd.so.0"
charon[5408]: 00[LIB] plugin 'tpm': failed to load - tpm_plugin_create returned NULL
systemd[1]: Started strongSwan IPsec IKEv1/IKEv2 daemon using swanctl.
charon[5408]: 02[KNL] 203.0.113.1 disappeared from eth0
charon[5408]: 08[NET] using forecast interface eth3
charon[5408]: 08[CFG] joining forecast multicast groups: 224.0.0.1,224.0.0.22,224.0.0.251,224.0.0.252,239.255.255.250
haron[5408]: 07[KNL] 203.0.113.1 appeared on eth0
charon[5408]: 03[NET] using forecast interface eth3

Tunnel Status:

$ show nhrp tunnel
Status: ok
Interface    Type    Protocol-Address    Alias-Address    Flags          NBMA-Address

tun100       local   172.16.120.255/32   172.16.120.174   up

Spoke's configuration:

set interfaces ethernet eth0 address dhcp
set interfaces tunnel tun100 address '172.16.120.174/24'
set interfaces tunnel tun100 enable-multicast
set interfaces tunnel tun100 encapsulation 'gre'
set interfaces tunnel tun100 ip adjust-mss '1360'
set interfaces tunnel tun100 mtu '1400'
set interfaces tunnel tun100 parameters ip key '50120'
set interfaces tunnel tun100 source-interface 'eth0'
set protocols nhrp tunnel tun100 holding-time '10'
set protocols nhrp tunnel tun100 map 172.16.120.1/24 nbma-address '192.0.2.2'
set protocols nhrp tunnel tun100 map 172.16.120.1/24 register
set protocols nhrp tunnel tun100 multicast 'nhs'
set protocols nhrp tunnel tun100 redirect
set protocols nhrp tunnel tun100 shortcut
set vpn ipsec esp-group esp-spoke lifetime '1800'
set vpn ipsec esp-group esp-spoke mode 'transport'
set vpn ipsec esp-group esp-spoke pfs 'dh-group19'
set vpn ipsec esp-group esp-spoke proposal 1 encryption 'aes256'
set vpn ipsec esp-group esp-spoke proposal 1 hash 'sha256'
set vpn ipsec ike-group ike-spoke close-action 'start'
set vpn ipsec ike-group ike-spoke dead-peer-detection action 'restart'
set vpn ipsec ike-group ike-spoke key-exchange 'ikev2'
set vpn ipsec ike-group ike-spoke lifetime '1800'
set vpn ipsec ike-group ike-spoke proposal 1 dh-group '19'
set vpn ipsec ike-group ike-spoke proposal 1 encryption 'aes256'
set vpn ipsec ike-group ike-spoke proposal 1 hash 'sha256'
set vpn ipsec ike-group ike-spoke proposal 1 prf 'prfsha256'
set vpn ipsec interface 'eth0'
set vpn ipsec profile nhrvpn authentication mode 'pre-shared-secret'
set vpn ipsec profile nhrvpn authentication pre-shared-secret 'secret'
set vpn ipsec profile nhrvpn bind tunnel 'tun100'
set vpn ipsec profile nhrvpn esp-group 'esp-spoke'
set vpn ipsec profile nhrvpn ike-group 'ike-spoke'

Details

Version
1.4.4
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

SrividyaA created this task.Feb 2 2026, 9:13 AM
pasik subscribed.Feb 2 2026, 11:22 AM
Viacheslav subscribed.Feb 2 2026, 12:47 PM

The original opennhrp-script.py https://github.com/vyos/vyos-1x/blob/c3f688653511881a2270d83cd0f098b45bfa7148/src/etc/opennhrp/opennhrp-script.py#L223C1-L247C1

def peer_up(dmvpn_type: str, conn: str) -> None:
    """Proceed NHRP peer UP event

    Args:
        dmvpn_type (str): a type of peer
        conn (str): an IKE profile name
    """
    logger.info(f'Peer UP event for {dmvpn_type} using IKE profile {conn}')
    src_nbma = os.getenv('NHRP_SRCNBMA')
    dest_nbma = os.getenv('NHRP_DESTNBMA')
    dest_mtu = os.getenv('NHRP_DESTMTU')

    if not src_nbma or not dest_nbma:
        logger.error(
            f'Can not get NHRP NBMA addresses: local {src_nbma}, '
            f'remote {dest_nbma}')
        return

    logger.info(f'NBMA addresses: local {src_nbma}, remote {dest_nbma}')
    if dest_mtu:
        add_peer_route(src_nbma, dest_nbma, dest_mtu)
    if conn and dmvpn_type == 'spoke' and process_named_running('charon'):
        vici_terminate(conn, src_nbma, dest_nbma)
        vici_initiate(conn, 'dmvpn', src_nbma, dest_nbma)

We can try to add some attempts (for example):

import time

def peer_up(dmvpn_type: str, conn: str) -> None:
    """Proceed NHRP peer UP event

    Args:
        dmvpn_type (str): a type of peer
        conn (str): an IKE profile name
    """
    logger.info(f'Peer UP event for {dmvpn_type} using IKE profile {conn}')

    max_attempts = 10
    delay_sec = 3

    src_nbma = None
    dest_nbma = None
    dest_mtu = None

    for attempt in range(1, max_attempts + 1):
        src_nbma = os.getenv('NHRP_SRCNBMA')
        dest_nbma = os.getenv('NHRP_DESTNBMA')
        dest_mtu = os.getenv('NHRP_DESTMTU')

        if src_nbma and dest_nbma:
            logger.info(
                f'NBMA addresses ready after {attempt} attempts: '
                f'local {src_nbma}, remote {dest_nbma}'
            )
            break

        logger.warning(
            f'NBMA not ready (attempt {attempt}/{max_attempts}): '
            f'local={src_nbma}, remote={dest_nbma}'
        )
        time.sleep(delay_sec)

    if not src_nbma or not dest_nbma:
        logger.error(
            f'Can not get NHRP NBMA addresses after {max_attempts} attempts: '
            f'local {src_nbma}, remote {dest_nbma}'
        )
        return

    if dest_mtu:
        add_peer_route(src_nbma, dest_nbma, dest_mtu)

    if conn and dmvpn_type == 'spoke' and process_named_running('charon'):
        vici_terminate(conn, src_nbma, dest_nbma)
        vici_initiate(conn, 'dmvpn', src_nbma, dest_nbma)
Viacheslav triaged this task as High priority.Feb 2 2026, 1:01 PM
SrividyaA renamed this task from DMVPN IPSec does not initiate if the local NMBA address is found none during the bootup to DMVPN IPSec does not initiate if the local NBMA address is found none during the bootup.Feb 4 2026, 7:41 AM
SrividyaA added a comment.Feb 4 2026, 11:34 AM

The suggested changes does not seem to fix the issue, the ipsec tunnel does not initiate after reboot (opennhrp service brings the tunnel up) The dhcp ip address is intact for the interface, no loss of IP was noticed.

Feb 04 09:50:00 spoke03 opennhrp[3357]: opennhrp[3357]: Adding static 172.19.200.1/24 nbma 100.10.1.2 dev tun100
Feb 04 09:50:00 spoke03 opennhrp[3357]: Adding static 172.19.200.1/24 nbma 100.10.1.2 dev tun100
Feb 04 09:50:04 spoke03 opennhrp[3373]: Removing static 172.19.200.1/24 nbma 100.10.1.2 dev tun100
Feb 04 09:50:04 spoke03 opennhrp[3726]: opennhrp[3726]: Adding static 172.19.200.1/24 nbma 100.10.1.2 dev tun100
Feb 04 09:50:04 spoke03 opennhrp[3726]: Adding static 172.19.200.1/24 nbma 100.10.1.2 dev tun100
Feb 04 09:50:09 spoke03 opennhrp-script.py[4050]: Running script with arguments: ['/etc/opennhrp/opennhrp-script.py', 'peer-up'], environment: environ({'NHRP_TYPE': 'static', 'NHRP_SRCADDR': '172.19.200.3', 'NHRP_DESTADDR': '172.19.200.1', 'NHRP_DESTPREFIX': '24', 'NHRP_PEER_DOWN_REASON': 'user-request', 'NHRP_DESTNBMA': '100.10.1.2', 'NHRP_INTERFACE': 'tun100', 'NHRP_GRE_KEY': '50120', 'LC_CTYPE': 'C.UTF-8'})
Feb 04 09:50:09 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 1/10): local=None, remote=100.10.1.2
Feb 04 09:50:12 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 2/10): local=None, remote=100.10.1.2
Feb 04 09:50:15 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 3/10): local=None, remote=100.10.1.2
Feb 04 09:50:18 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 4/10): local=None, remote=100.10.1.2
Feb 04 09:50:21 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 5/10): local=None, remote=100.10.1.2
Feb 04 09:50:24 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 6/10): local=None, remote=100.10.1.2
Feb 04 09:50:27 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 7/10): local=None, remote=100.10.1.2
Feb 04 09:50:30 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 8/10): local=None, remote=100.10.1.2
Feb 04 09:50:33 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 9/10): local=None, remote=100.10.1.2
Feb 04 09:50:36 spoke03 opennhrp-script.py[4050]: NBMA not ready (attempt 10/10): local=None, remote=100.10.1.2
Feb 04 09:50:39 spoke03 opennhrp-script.py[4050]: Can not get NHRP NBMA addresses after 10 attempts: local None, remote 100.10.1.2
Feb 04 09:50:39 spoke03 opennhrp[3742]: Sending packet 3, from: 172.19.200.3 (nbma (unspecified)), to: 172.19.200.1 (nbma 100.10.1.2)
Feb 04 09:50:40 spoke03 opennhrp[3742]: Packet type 4 from nbma src 10.51.1.2, proto src 172.19.200.3, proto dst 172.19.200.1 dropped: no matching request
Feb 04 09:50:40 spoke03 opennhrp[3742]: Sending packet 7, from: 172.19.200.3 (nbma (unspecified)), to: 172.19.200.1 (nbma 100.10.1.2)
o.kuchmystyi changed the task status from Open to In progress.Apr 6 2026, 8:10 AM
o.kuchmystyi claimed this task.
o.kuchmystyi added a comment.EditedApr 9 2026, 1:19 PM

PR: https://github.com/VyOS-Networks/vyos-1x/pull/2005

Viacheslav closed this task as Resolved.Apr 16 2026, 3:34 PM
Viacheslav added a project: VyOS 1.4 Sagitta (1.4.5).