Page MenuHomeVyOS Platform
Create Task
Maniphest T2969

OpenVPN: command_set on interface is not applied, if interface doesn't come up in commit
Closed, ResolvedPublicBUG
Actions

Description

Long title - this is what I mean:
When doing a commit, any property of an interface, that gets set with the '_set_command' in vyos/ifconfig/control.py only gets applied to the interface if it's up at the time of the commit. If the interface comes up later the property doesn't get applied to the interface.
To give an example:

set interfaces openvpn vtun10 mode client
...
set interfaces openvpn vtun10 vrf MY-VRF

If the server we're trying to connect to isn't available at commit vtun10 doesn't come up and therefor it doesn't get added to MY-VRF.
Some time later, we can connect to the server and the vtun10 comes up. But it won't be part of MY-VRF.
This is a special problem with OpenVPN client interfaces on reboot. Even if the server is available, the vtun interface doesn't come up immidiatly and won't join the VRF.
This problem is not specific to OpenVPN and VRF, but applies to all properties set with the '_set_command' on all interfaces that use it.

I don't have a solution for this, since it can be solved in several ways and the core maintainers should decide how to do it.

My suggestions are to use the if-up script calls from Debian:

  1. Make a script file on commit, that contains all the properties for the interface and put it in if-up.d/
  2. Put a permanent script file if-up.d/ that makes a callback to a python file, passing the interface as a parameter. The python file could then run the '_set_command's as they are done now. Instead of running them on commit.

Personally I think 2) is better.

Details

Difficulty level
Easy (less than an hour)
Version
1.3
Why the issue appeared?
Other
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Related Objects
Search...

StatusSubtypeAssignedTask
 In progressFEATURE REQUESTNoneT3355 Remove all remaining legacy Vyatta code
 ResolvedFEATURE REQUESTNoneT1579 Rewrite all interface types in new XML/Python style
 ResolvedFEATURE REQUESTc-poT2653 "set interfaces" Python handler code improvements - next iteration
 Resolvedc-poT2644 Bonding interfaces cannot be disabled
 ResolvedBUGc-poT2106 Wrong interface states after reboot
 ResolvedFEATURE REQUESTc-poT2733 Support MTU configuration on pseudo ethernet devices
 ResolvedFEATURE REQUESTc-poT2734 WireGuard: fwmark CLI definition is inconsistent
 ResolvedFEATURE REQUESTc-poT2746 IPv6 link-local addresses not configured
 ResolvedFEATURE REQUESTc-poT2903 Q-in-Q (802.1.ad) ethertype should be defined explicitly and not via its raw value
 ResolvedFEATURE REQUESTc-poT2904 802.1ad / Q-in-Q ethertype default not utilized
 ResolvedFEATURE REQUESTc-poT2905 Sync CLI nodes between PPPoE and WWAN interface
 ResolvedFEATURE REQUESTc-poT2912 When setting MTU check for hardware maximum supported MTU size
 ResolvedBUGc-poT2928 MTU less then 1280 bytes and IPv6 will raise FileNotFoundError
 ResolvedFEATURE REQUESTc-poT2930 Support configuration of MAC address for VXLAN and GENEVE tunnel
 ResolvedFEATURE REQUESTc-poT2985 Add glue code to create bridge interface on demand
 ResolvedFEATURE REQUESTc-poT2994 Migrate OpenVPN interfaces to get_config_dict() syntax
 ResolvedBUGc-poT2969 OpenVPN: command_set on interface is not applied, if interface doesn't come up in commit
 ResolvedFEATURE REQUESTc-poT3043 Wireless: Refactor CLI
 ResolvedFEATURE REQUESTc-poT3034 Add WiFi WPA 3 support
 ResolvedFEATURE REQUESTc-poT3072 Migrate tunnel interfaces to new get_config_dict() approach
 ResolvedFEATURE REQUESTc-poT2966 tunnel: add new encapsulation types ip6tnl and ip6gretap
 ResolvedENHANCEMENTc-poT3068 Automatic generation of IPv6 link local addresses for tunnel interfaces
 ResolvedFEATURE REQUESTc-poT3364 tunnel: cleanup/rename CLI nodes
 ResolvedFEATURE REQUESTc-poT3077 WireGuard: automatically create link-local IPv6 adresses
 ResolvedFEATURE REQUESTc-poT2259 Support for bind vif-c interfaces into VRFs
 Resolvedc-poT2177 Commit fails on adding disabled interface to bridge
 ResolvedFEATURE REQUESTc-poT3112 PPPoE IPv6: remove "enable" node

Event Timeline

Magnum created this task.Oct 7 2020, 4:56 PM
pasik added a subscriber: pasik.Oct 7 2020, 6:01 PM
c-po added a subscriber: c-po.Oct 11 2020, 4:27 PM

I can feel that pain! When looking at the source from VyOS 1.2 (crux) it looks like it always behaved in this way.

If I recall Debian and ifupdown the helper scripts from /etc/if-up.d are only used when ifupdown is used to bring up an interface - thus this approach won't work. It may be the first use for some asynchronous halpers of vyos-configd.

Magnum added a comment.Oct 13 2020, 10:29 AM

You're right, if-up.d scripts only get run for the interfaces defined in /etc/network/interfaces.

Mabe we can use the iproute2 tools?
Like 'ip monitor' and 'rtmon' - I'm not an expert in these, but I think they could trigger a script?

Magnum added a comment.Oct 13 2020, 11:35 AM

This bug seems to be worse than I thought.
Here's an example:
On reboot an openvpn client inteface will come up outside the vrf. Any routes that get pushed by the server will not get added to the client because it's wants to add the routes inside the vrf of the vtun interface - but the vtun isn't a member.
Heres a log snippet:

Oct 13 13:09:22 cpe2-2 openvpn-vtun11[2665]: /usr/libexec/vyos/system/unpriv-ip addr add dev vtun11 10.112.19.5/24 broadcast 10.112.19.255
Oct 13 13:09:22 cpe2-2 charon: 08[KNL] 10.112.19.5 appeared on vtun11
Oct 13 13:09:22 cpe2-2 netplugd[777]: vtun11: ignoring event
Oct 13 13:09:22 cpe2-2 openvpn-vtun11[2665]: /usr/libexec/vyos/system/unpriv-ip route add 192.168.112.0/24 via 10.112.19.1
Oct 13 13:09:22 cpe2-2 netplugd[777]: vtun11: ignoring event
Oct 13 13:09:22 cpe2-2 netplugd[777]: vtun11: ignoring event
Oct 13 13:09:22 cpe2-2 bfdd[950]: [EC 100663301] INTERFACE_VRF_UPDATE: Cannot find IF vtun11 in VRF 0
Oct 13 13:09:22 cpe2-2 charon: 07[KNL] interface vtun11 deactivated
Oct 13 13:09:22 cpe2-2 netplugd[777]: vtun11: ignoring event
Oct 13 13:09:22 cpe2-2 charon: 08[KNL] interface vtun11 activated
Oct 13 13:09:22 cpe2-2 openvpn-vtun11[2665]: ERROR: Linux route add command failed: external program exited with error status: 2
c-po added a comment.Oct 14 2020, 4:58 AM

Please share your OpenVPN config

Magnum added a comment.Oct 14 2020, 7:01 AM

Client:

interfaces {
    ethernet eth2 {
        address 10.201.1.2/30
        description WAN
        hw-id 0c:6b:af:b0:4f:02
    }
    openvpn vtun11 {
        description "CPE MGMT"
        device-type tun
        encryption {
            cipher aes256
        }
        hash sha1
        mode client
        persistent-tunnel
        protocol udp
        remote-host 10.200.200.11
        remote-port 1194
        tls {
            auth-file /config/auth/shared.key
            ca-cert-file /config/auth/ca.crt
            cert-file /config/auth/cpe1-1.crt
            key-file /config/auth/cpe1-1.key
        }
        vrf CPE-MGMT
    }
}
protocols {
    static {
        route 0.0.0.0/0 {
            next-hop 10.201.1.1 {
            }
        }
    }
}
vrf {
    name CPE-MGMT {
        description "CPE MGMT"
        table 112
    }
}

Server:

interfaces {
    ethernet eth2 {
        address 10.200.200.11/26
        description WAN
        hw-id 0c:6b:af:12:34:02
    }
    ethernet eth3 {
        address 192.168.112.19/24
        description "CPE MGMT"
        hw-id 0c:6b:af:12:34:03
        vrf CPE-MGMT
    }
    openvpn vtun11 {
        description "CPE MGMT"
        device-type tun
        encryption {
            cipher aes256
            disable-ncp
        }
        hash sha1
        local-host 10.200.200.11
        local-port 1194
        mode server
        persistent-tunnel
        protocol udp
        server {
            push-route 192.168.112.0/24
            subnet 10.112.19.0/24
            topology subnet
        }
        tls {
            auth-file /config/auth/shared.key
            ca-cert-file /config/auth/ca.crt
            cert-file /config/auth/central.crt
            dh-file /config/auth/dh.pem
            key-file /config/auth/central.key
        }
        vrf CPE-MGMT
    }
}
protocols {
    static {
        route 0.0.0.0/0 {
            next-hop 10.200.200.1 {
            }
        }
    }
}
vrf {
    name CPE-MGMT {
        description "CPE MGMT"
        table 112
    }
}
c-po added a comment.Oct 18 2020, 10:16 AM

The root cause of this problem is that OpenVPN when the deamon is started and in tries to connect to the server, yet did not create the vtun11 interface on the system. Thus all calls to the ifconfig python library will fail big time.

Are you aware of any OpenVPN configuration parameter that the interface should be created no matter if the connection has been successful or not - that would be the simplest and most preferred solution as we then can reuse all of our codebase.

Magnum added a comment.Oct 18 2020, 3:13 PM

I know, that my specific problem is related to OpenVPN, but are you saying, that this is only relevant for OpenVPN and it's not going to impact other interfaces?

In regards to OpenVPN, I don't know of any way to force a client interface up.
You can check for messages of interface up/down - but that's basically the same as using iproute2

I think there is support for VRF on Linux in OpenVPN 2.4.3 now - but that only solveves half the problem:
https://github.com/OpenVPN/openvpn/pull/65

c-po changed the task status from Open to In progress.Oct 18 2020, 3:58 PM
c-po claimed this task.
c-po added a comment.EditedOct 18 2020, 4:03 PM

It seems that calling openvpn --mktun is what we need. Please try the next rolling ISO which will contain a fix for this.

Nevertheless this code should be migrated to get_config_dict() to utilize the full power of the new backend.

c-po changed the task status from In progress to Needs testing.Oct 18 2020, 4:16 PM
c-po triaged this task as Normal priority.
c-po changed Difficulty level from Unknown (require assessment) to Easy (less than an hour).
c-po changed Why the issue appeared? from Will be filled on close to Other.
c-po changed Is it a breaking change? from Behavior change to Perfectly compatible.
c-po added a parent task: T1579: Rewrite all interface types in new XML/Python style.Oct 18 2020, 4:37 PM
Magnum added a comment.Oct 18 2020, 6:32 PM

Well spotted - i hadn't seen that option before.
I'll give it a go and see how it runs.

Magnum added a comment.Oct 19 2020, 8:57 AM

Tested in my lab and it works both during creation and reboot.

On creation of vtun interface on client if no server is available:
The interface is created and put in VRF and later remains there, when server is reachable.

On reboot:
The interface is created and put int the VRF and then it joins the server.

Routes from the server are pushed to the correct VRF in both cases.

c-po renamed this task from command_set on interface is not applied, if interface doesn't come up in commit to OpenVPN: command_set on interface is not applied, if interface doesn't come up in commit.Oct 19 2020, 9:01 AM
c-po closed this task as Resolved.
Magnum reopened this task as In progress.Oct 19 2020, 10:15 AM

I spoke to soon.
The interfaces are very persistant now - when you delete the vtun interface it doesn't get taken down!

c-po added a comment.Oct 19 2020, 10:39 AM

There have been some deletion errors yesterday - are you running the latest rolling release? They should have been fixed in there. If noe please provide me some CLI samples to reproduce the issue.

Magnum added a comment.Oct 19 2020, 10:50 AM

No, I'm running this commit:
https://github.com/vyos/vyos-1x/commit/029f9839c21317ec5959b331eee25da472d08dc1

I can se you made some changes - I'll compile a new image and test.

Magnum added a comment.Oct 19 2020, 12:28 PM

Just tested with:
https://github.com/vyos/vyos-1x/commit/85cc735b05be109e6daa5403efa4122b8b6e79d2

The interfaces come up and connect to the server, when it becomes available and are in the correct VRF, but they don't disapear, when the interface is deleted.

To reproduce:
Commit config below on client (don't need server).
Check that vtun11 has been created (this one is connected to server):

vyos@vyos:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun11           10.112.19.2/24                    u/u  CPE MGMT

Delete the interface:

delete interface openvpn vtun11

check interfaces again - it's still there (not connected to server):

vyos@vyos:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun11           -                     u/u  CPE MGMT

My client config:

set interfaces ethernet eth0 address '10.100.1.1/24'
set interfaces ethernet eth0 description 'LAN'
set interfaces ethernet eth2 address '10.201.1.2/30'
set interfaces ethernet eth2 description 'WAN'
set interfaces loopback lo address '10.0.0.1/32'
set interfaces openvpn vtun11 description 'CPE MGMT'
set interfaces openvpn vtun11 device-type 'tun'
set interfaces openvpn vtun11 encryption cipher 'aes256'
set interfaces openvpn vtun11 hash 'sha1'
set interfaces openvpn vtun11 mode 'client'
set interfaces openvpn vtun11 persistent-tunnel
set interfaces openvpn vtun11 protocol 'udp'
set interfaces openvpn vtun11 remote-host '10.200.200.11'
set interfaces openvpn vtun11 remote-port '1194'
set interfaces openvpn vtun11 tls auth-file '/config/auth/shared.key'
set interfaces openvpn vtun11 tls ca-cert-file '/config/auth/ca.crt'
set interfaces openvpn vtun11 tls cert-file '/config/auth/cpe1-1.crt'
set interfaces openvpn vtun11 tls key-file '/config/auth/cpe1-1.key'
set interfaces openvpn vtun11 vrf 'CPE-MGMT'
set protocols static route 0.0.0.0/0 next-hop 10.201.1.1
set vrf name CPE-MGMT description 'CPE MGMT'
set vrf name CPE-MGMT table '112'
c-po added a comment.Oct 19 2020, 6:52 PM

Unfortunately I can not reproduce this issue on my test system and also our smoketests (https://github.com/vyos/vyos-1x/blob/current/smoketest/scripts/cli/test_interfaces_openvpn.py) do not trigger the bug when run locally on the VyOS device by calling:

vyos@vyos:~$ /usr/libexec/vyos/tests/smoke/cli/test_interfaces_openvpn.py
...
----------------------------------------------------------------------
Ran 3 tests in 62.111s

OK

Can you please create the debug file as described in https://docs.vyos.io/en/latest/contributing/debugging.html#live-system by running
$ touch /tmp/vyos.ifconfig.debug
And then run your commands to trigger the issue again, and share the result?

Magnum added a comment.Oct 21 2020, 1:40 PM

I can see why the smoketests pass. They only check if the interface has been removed from the config - not the system.

The debug deosn't seem to have any effect?

Welcome to VyOS - cpe1 ttyS0

cpe1 login: vyo
Password: 
Welcome to VyOS - cpe1 ttyS0

cpe1 login: vyos
Password: 
Linux cpe1 4.19.152-amd64-vyos #1 SMP Sat Oct 17 21:32:49 UTC 2020 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
vyos@cpe1:~$ 
vyos@cpe1:~$ 
vyos@cpe1:~$ sho conf
interfaces {
    ethernet eth0 {
        address 10.100.1.1/24
        description LAN
        hw-id 0c:6b:af:94:e5:00
    }
    ethernet eth1 {
        address dhcp
        description MGMT
        hw-id 0c:6b:af:94:e5:01
    }
    ethernet eth2 {
        address 10.201.1.2/30
        description WAN
        hw-id 0c:6b:af:94:e5:02
    }
    loopback lo {
        address 10.0.0.1/32
    }
    openvpn vtun10 {
        description "Core OVPN"
        device-type tun
        hash sha1
        mode client
        persistent-tunnel
        protocol udp
        remote-host 10.200.200.10
        remote-port 1194
        tls {
            auth-file /config/auth/shared.key
            ca-cert-file /config/auth/ca.crt
            cert-file /config/auth/cpe1-1.crt
            key-file /config/auth/cpe1-1.key
        }
    }
    openvpn vtun11 {
        description "CPE MGMT"
        device-type tun
        encryption {
            cipher aes256
        }
        hash sha1
        mode client
        persistent-tunnel
        protocol udp
        remote-host 10.200.200.11
vyos@cpe1:~$ conf
[edit]
vyos@cpe1# del int o vtun10
[edit]
vyos@cpe1# commit
[edit]
vyos@cpe1# exit
Warning: configuration changes have not been saved.
exit
vyos@cpe1:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun10           -                                 A/D  Core OVPN
vtun11           -                                 A/D  CPE MGMT
vyos@cpe1:~$ conf
[edit]
vyos@cpe1# save
Saving configuration to '/config/config.boot'...
Done
[edit]
vyos@cpe1# touch /tmp/vyos.ifconfig.debug
[edit]
vyos@cpe1# ed int o vtun12
[edit interfaces openvpn vtun12]
vyos@cpe1# top show
[edit]
vyos@cpe1# show int o
 openvpn vtun11 {
     description "CPE MGMT"
     device-type tun
     encryption {
         cipher aes256
     }
     hash sha1
     mode client
     persistent-tunnel
     protocol udp
     remote-host 10.200.200.11
     remote-port 1194
     tls {
         auth-file /config/auth/shared.key
         ca-cert-file /config/auth/ca.crt
         cert-file /config/auth/cpe1-1.crt
         key-file /config/auth/cpe1-1.key
     }
     vrf CPE-MGMT
 }
+openvpn vtun12 {
+}
[edit]
vyos@cpe1# ed int o vtun12
[edit interfaces openvpn vtun12]
vyos@cpe1# set description test
[edit interfaces openvpn vtun12]
vyos@cpe1# set device-type tun
[edit interfaces openvpn vtun12]
vyos@cpe1# set encryption cipher aes2
aes256     aes256gcm  
[edit interfaces openvpn vtun12]
vyos@cpe1# set encryption cipher aes256
[edit interfaces openvpn vtun12]
vyos@cpe1# set hash sha1
[edit interfaces openvpn vtun12]
vyos@cpe1# set mode client
[edit interfaces openvpn vtun12]
vyos@cpe1# set persistent-tunnel 
[edit interfaces openvpn vtun12]
vyos@cpe1# set prot udp
[edit interfaces openvpn vtun12]
vyos@cpe1# set remote-host 10.100.100.1
[edit interfaces openvpn vtun12]
vyos@cpe1# set remote-port 1194
[edit interfaces openvpn vtun12]
vyos@cpe1# set tls auth-file /config/auth/shared.key
[edit interfaces openvpn vtun12]
vyos@cpe1# ed tls
[edit interfaces openvpn vtun12 tls]
vyos@cpe1# set ca-cert-file /config/auth/ca.cr
Possible completions:
   file         File in /config/auth directory
                

      
[edit interfaces openvpn vtun12 tls]
vyos@cpe1# set ca-cert-file /config/auth/ca.crt
[edit interfaces openvpn vtun12 tls]
vyos@cpe1# set cert-file /config/auth/cpe1-1.crt
[edit interfaces openvpn vtun12 tls]
vyos@cpe1# set key-file /config/aut/cpe1-1.key

  WARNING: File '/config/aut/cpe1-1.key' not found

[edit interfaces openvpn vtun12 tls]
vyos@cpe1# set key-file /config/auth/cpe1-1.key
[edit interfaces openvpn vtun12 tls]
vyos@cpe1# top
[edit]
vyos@cpe1# commit
[edit]
vyos@cpe1# cat /tmp/
.font-unix/
.ICE-unix/
systemd-private-bbd627ef261741ee832dbc885a5e5329-haveged.service-CGyXzt/
systemd-private-bbd627ef261741ee832dbc885a5e5329-ntp.service-KrFKgv/
systemd-private-bbd627ef261741ee832dbc885a5e5329-openvpn@vtun11.service-oCPPJU/
systemd-private-bbd627ef261741ee832dbc885a5e5329-openvpn@vtun12.service-R8Dqh3/
.Test-unix/
vyos-config-status
vyos.ifconfig.debug
.X11-unix/
.XIM-unix/
[edit]
vyos@cpe1# cat /tmp/vy
vyos-config-status   vyos.ifconfig.debug  
[edit]
vyos@cpe1# cat /tmp/vyos.ifconfig.debug 
[edit]
vyos@cpe1# exit
Warning: configuration changes have not been saved.
exit
vyos@cpe1:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun10           -                                 A/D  Core OVPN
vtun11           -                                 A/D  CPE MGMT
vtun12           -                                 A/D  test
vyos@cpe1:~$ conf
[edit]
vyos@cpe1# delete interfaces o vtun12
[edit]
vyos@cpe1# commit
[edit]
vyos@cpe1# show int o
 openvpn vtun11 {
     description "CPE MGMT"
     device-type tun
     encryption {
         cipher aes256
     }
     hash sha1
     mode client
     persistent-tunnel
     protocol udp
     remote-host 10.200.200.11
     remote-port 1194
     tls {
         auth-file /config/auth/shared.key
         ca-cert-file /config/auth/ca.crt
         cert-file /config/auth/cpe1-1.crt
         key-file /config/auth/cpe1-1.key
     }
     vrf CPE-MGMT
 }
[edit]
vyos@cpe1# exit
Warning: configuration changes have not been saved.
exit
vyos@cpe1:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun10           -                                 A/D  Core OVPN
vtun11           -                                 A/D  CPE MGMT
vtun12           -                                 A/D  test
vyos@cpe1:~$ cat /tmp/vyo
vyos-config-status   vyos.ifconfig.debug  
vyos@cpe1:~$ cat /tmp/vyos.ifconfig.debug 
vyos@cpe1:~$
Magnum added a comment.Oct 21 2020, 2:02 PM

Also, while setting up test for T2906:

vyos@cpe1:~$ conf 
[edit]
vyos@cpe1# show int o vtun11
 description "CPE MGMT"
 device-type tun
 encryption {
     cipher aes256
 }
 hash sha1
 mode client
 persistent-tunnel
 protocol udp
 remote-host 10.200.200.11
 remote-port 1194
 tls {
     auth-file /config/auth/shared.key
     ca-cert-file /config/auth/ca.crt
     cert-file /config/auth/cpe1-1.crt
     key-file /config/auth/cpe1-1.key
 }
 vrf CPE-MGMT
[edit]
vyos@cpe1# set int o vtun13 device-type tun
[edit]
vyos@cpe1# edit int o vtun13 
[edit interfaces openvpn vtun13]
vyos@cpe1# set encryption disable-ncp 
[edit interfaces openvpn vtun13]
vyos@cpe1# set hash sha1
[edit interfaces openvpn vtun13]
vyos@cpe1# set local
local-address  local-host     local-port     
[edit interfaces openvpn vtun13]
vyos@cpe1# set local-host 10.201.1.2
[edit interfaces openvpn vtun13]
vyos@cpe1# set local-port 1194
[edit interfaces openvpn vtun13]
vyos@cpe1# set persistent-tunnel 
[edit interfaces openvpn vtun13]
vyos@cpe1# set protocol udp 
[edit interfaces openvpn vtun13]
vyos@cpe1# set server subnet 172.16.11.0/24
[edit interfaces openvpn vtun13]
vyos@cpe1# set server topology subnet 
[edit interfaces openvpn vtun13]
vyos@cpe1# commit
[ interfaces openvpn vtun13 ]
Must specify OpenVPN operation mode

[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1# show
+device-type tun
+encryption {
+    disable-ncp
+}
+hash sha1
+local-host 10.201.1.2
+local-port 1194
+persistent-tunnel
+protocol udp
+server {
+    subnet 172.16.11.0/24
+    topology subnet
+}
[edit interfaces openvpn vtun13]
vyos@cpe1# set mode server 
[edit interfaces openvpn vtun13]
vyos@cpe1# commit
Must specify "tls dh-file" when not using EC keys in server mode
[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1# set tls ca-cert-file /config/auth/ca.crt
[edit interfaces openvpn vtun13]
vyos@cpe1# set tls cert-file /config/auth/cpe1-1.crt
[edit interfaces openvpn vtun13]
vyos@cpe1# set tls key-file /config/auth/cpe1-1.key
[edit interfaces openvpn vtun13]
vyos@cpe1# set tls auth-file /config/auth/shared.key
[edit interfaces openvpn vtun13]
vyos@cpe1# commit
Must specify "tls dh-file" when not using EC keys in server mode
[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1# set tls dh-file /config/auth/dh.pem

  WARNING: File '/config/auth/dh.pem' not found

[edit interfaces openvpn vtun13]
vyos@cpe1# commit
[ interfaces openvpn vtun13 tls dh-file /config/auth/dh.pem ]
WARNING: File '/config/auth/dh.pem' not found

Specified dh-file "/config/auth/dh.pem" is not valid
[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1# ruun show int

  Invalid command: [ruun]

[edit interfaces openvpn vtun13]
vyos@cpe1# run show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun10           -                                 A/D  Core OVPN
vtun11           -                                 A/D  CPE MGMT
vtun12           -                                 A/D  test
[edit interfaces openvpn vtun13]

Copy dh.pem file to device

vyos@cpe1# commit
[ interfaces openvpn vtun13 ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Make sure you are running the latest stable version of VyOS
  the code is available at https://downloads.vyos.io/?dir=release/current
- Contact us using the online help desk
  https://support.vyos.io/
- Join our community on slack where our users exchange help and advice
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your 
  business policy requires it)
- and include all the information presented below

Report Time:      2020-10-21 15:56:28
Image Version:    VyOS 1.3-zt-20201019
Release Train:    equuleus

Built by:         [email protected]
Built on:         Mon 19 Oct 2020 11:00 UTC
Build UUID:       661a3bf2-dfb4-43c9-b1cc-77a8bc274e71
Build Commit ID:  7de6c3bb71d40e

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:     
Hardware UUID:    b8895c0e-850e-4405-add7-0b82d99294e5

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1110, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1063, in apply
    cmd(f'sudo openvpn --mktun --dev-type {dev_type} --dev {interface}')
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 179, in cmd
    raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: sudo openvpn --mktun --dev-type tun --dev vtun13
returned: Wed Oct 21 15:56:27 2020 ERROR: Cannot ioctl TUNSETIFF vtun13: Device or resource busy (errno=16)
Wed Oct 21 15:56:27 2020 Exiting due to fatal error
exit code: 1



[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1# ls /run/op
openvpn/        openvpn-client/ openvpn-server/ 
[edit interfaces openvpn vtun13]
vyos@cpe1# ls /run/openvpn
ccd                vtun10.status  vtun11.status  vtun13.pid
openvpn-mgmt-intf  vtun11.conf    vtun12.status  vtun13.status
status             vtun11.pid     vtun13.conf
[edit interfaces openvpn vtun13]
vyos@cpe1# run show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             10.100.1.1/24                     u/u  LAN
eth1             192.168.125.205/24                u/u  MGMT
eth2             10.201.1.2/30                     u/u  WAN
lo               127.0.0.1/8                       u/u  
                 10.0.0.1/32                            
                 ::1/128                                
vtun10           -                                 A/D  Core OVPN
vtun11           -                                 A/D  CPE MGMT
vtun12           -                                 A/D  test
vtun13           172.16.11.1/24                    u/u  
[edit interfaces openvpn vtun13]
vyos@cpe1# cat /var/run/ope
openvpn/        openvpn-client/ openvpn-server/ 
[edit interfaces openvpn vtun13]
vyos@cpe1# cat /var/run/openvpn/vtun
vtun10.status  vtun11.pid     vtun12.status  vtun13.pid     
vtun11.conf    vtun11.status  vtun13.conf    vtun13.status  
[edit interfaces openvpn vtun13]
vyos@cpe1# cat /var/run/openvpn/vtun13.conf 
### Autogenerated by interfaces-openvpn.py ###
#
# See https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage
# for individual keyword definition

verb 3

user openvpn
group openvpn

dev-type tun
dev vtun13
persist-key
iproute /usr/libexec/vyos/system/unpriv-ip

proto udp
local 10.201.1.2
lport 1194


persist-tun

#
# OpenVPN Server mode
#
topology subnet
server 172.16.11.0 255.255.255.0 nopool
ifconfig-pool 172.16.11.2 172.16.11.253 255.255.255.0

keepalive 10 60
management /run/openvpn/openvpn-mgmt-intf unix



# TLS options
ca /config/auth/ca.crt
cert /config/auth/cpe1-1.crt
key /config/auth/cpe1-1.key
dh /config/auth/dh.pem
tls-auth /config/auth/shared.key 0

# Encryption options
ncp-disable

auth sha1

# DEPRECATED This option will be removed in OpenVPN 2.5
# Until OpenVPN v2.3 the format of the X.509 Subject fields was formatted like this:
# /C=US/L=Somewhere/CN=John Doe/[email protected] In addition the old
# behaviour was to remap any character other than alphanumeric, underscore ('_'),
# dash ('-'), dot ('.'), and slash ('/') to underscore ('_'). The X.509 Subject
# string as returned by the tls_id environmental variable, could additionally
# contain colon (':') or equal ('='). When using the --compat-names option, this
# old formatting and remapping will be re-enabled again. This is purely implemented
# for compatibility reasons when using older plug-ins or scripts which does not
# handle the new formatting or UTF-8 characters.
#
# See https://phabricator.vyos.net/T1512
compat-names

[edit interfaces openvpn vtun13]
vyos@cpe1# commit
[ interfaces openvpn vtun13 ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Make sure you are running the latest stable version of VyOS
  the code is available at https://downloads.vyos.io/?dir=release/current
- Contact us using the online help desk
  https://support.vyos.io/
- Join our community on slack where our users exchange help and advice
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your 
  business policy requires it)
- and include all the information presented below

Report Time:      2020-10-21 15:57:59
Image Version:    VyOS 1.3-zt-20201019
Release Train:    equuleus

Built by:         [email protected]
Built on:         Mon 19 Oct 2020 11:00 UTC
Build UUID:       661a3bf2-dfb4-43c9-b1cc-77a8bc274e71
Build Commit ID:  7de6c3bb71d40e

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:     
Hardware UUID:    b8895c0e-850e-4405-add7-0b82d99294e5

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1110, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1063, in apply
    cmd(f'sudo openvpn --mktun --dev-type {dev_type} --dev {interface}')
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 179, in cmd
    raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: sudo openvpn --mktun --dev-type tun --dev vtun13
returned: Wed Oct 21 15:57:59 2020 ERROR: Cannot ioctl TUNSETIFF vtun13: Device or resource busy (errno=16)
Wed Oct 21 15:57:59 2020 Exiting due to fatal error
exit code: 1



[[interfaces openvpn vtun13]] failed
Commit failed
[edit interfaces openvpn vtun13]
vyos@cpe1#
c-po added a comment.Oct 25 2020, 5:40 PM

Thanks - hopefully T2994 will catch all this stuff! Please be patient, I'm currently in the testing phase.

c-po added a parent task: T2994: Migrate OpenVPN interfaces to get_config_dict() syntax.Oct 25 2020, 5:51 PM
c-po added a comment.Oct 30 2020, 6:03 PM

Closing in favor of T2994 - please try tomorrows rolling release. If there are new bugs (which might always happen on rewrites) please file a new Bug report and I try to fix them ASAP.

c-po closed this task as Resolved.Oct 30 2020, 6:03 PM
c-po moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.Nov 23 2020, 4:01 PM
erkin set Issue type to Bug (incorrect behavior).Aug 29 2021, 12:40 PM
erkin removed a subscriber: Active contributors.
syncer edited projects, added VyOS 1.3 Equuleus (1.3.0); removed VyOS 1.3 Equuleus.Aug 29 2022, 12:26 PM
syncer moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus (1.3.0) board.