Page MenuHomeVyOS Platform

Removing tunnel deletes all tunnels?
Closed, ResolvedPublicBUG

Description

vyos@ROUTER:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description


br33 - A/D
br62 - u/u
br72 - u/u
eth0 8.4.1.170/29 u/u

8.4.1.171/29
8.4.1.172/29
8.4.1.173/29

eth1 - A/D
lo 127.0.0.1/8 u/u

10.255.3.102/32
::1/128

tun33 - A/D
tun36 - u/u
tun37 - u/u
tun326 - u/u tunnel to EU for US
tun327 - u/u tunnel to CA for NY
vyos@ROUTER:~$ config
[edit]
vyos@ROUTER# delete interfaces tunnel tun37
[edit]
vyos@ROUTER# delete interfaces tunnel tun327
[edit]
vyos@ROUTER# commit
[ interfaces tunnel tun327 description tunnel to CA for NY ]
sh: /sys/class/net/tun327/ifalias: No such file or directory

delete [ interfaces tunnel tun327 ] failed
Commit failed
[edit]
vyos@ROUTER# run show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description


br33 - A/D
br62 - u/D
br72 - u/D
eth0 8.4.1.170/29 u/u

8.4.1.171/29
8.4.1.172/29
8.4.1.173/29

eth1 - A/D
lo 127.0.0.1/8 u/u

10.255.3.102/32
::1/128

[edit]
vyos@ROUTER#

show version
Version: VyOS 1.2.0-rolling+201811170337
Built by: [email protected]
Built on: Sat 17 Nov 2018 03:37 UTC
Build ID: b60ecc7a-3335-40e3-917a-80018241178c

Architecture: x86_64
Boot via: installed image
System type: bare metal

Hardware vendor: Supermicro
Hardware model: Super Server
Hardware S/N: Unknown
Hardware UUID: Unknown

Copyright: VyOS maintainers and contributors

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.2.0-rolling+201811170337
Why the issue appeared?
Will be filled on close

Event Timeline

syncer triaged this task as Normal priority.
syncer edited projects, added VyOS 1.2 Crux (VyOS 1.2.0-rc10); removed VyOS 1.2 Crux.

I could not reproduce it, in its simplest form:

vyos@vyos-test-1# run show interfaces tunnel 
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
tun0             10.255.0.100/29                   u/u  
tun2             -                                 u/u  
[edit]
vyos@vyos-test-1# delete interfaces tunnel tun2
[edit]
vyos@vyos-test-1# commit
[edit]
vyos@vyos-test-1# run show interfaces tunnel 
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
tun0             10.255.0.100/29                   u/u

It doesn't mean the bug doesn't exist of course, but we need to narrow down the reproducing procedure. @Barrysdca could you post the tunnels config?

@Barrysdca Can you please share your config? At least the tunnel parts.

below is a crude paste with IP's changed. If you want the original config then I will share it privately. just can't have it out in the open. tx

firewall {

all-ping enable
broadcast-ping disable
config-trap disable
group {
    address-group RouterPubInterfaceIPs {
        address ...

}

address-group trustedIPs {
    address ...

}

}
ipv6-receive-redirects disable
ipv6-src-route disable
ip-src-route disable
log-martians enable
name ProtectP2PRouter {
    default-action drop
    rule 10 {
        action accept
        protocol all
        source {
            group {
                address-group trustedIPs
            }
        }
    }
    rule 20 {
        action accept
        description "allow access to router pub ip ports from authorized ip addresses"
        protocol all
        source {
            group {
                address-group RouterPubInterfaceIPs
            }
        }
    }
    rule 90 {
        action accept
        description Ping
        log enable
        protocol icmp
    }
    rule 1000 {
        action drop
        protocol all
    }
}
receive-redirects disable
send-redirects enable
source-validation disable
syn-cookies enable
twa-hazards-protection disable

}
interfaces {

 bridge br33 {
     aging 300
     description "L2 VPN Bridge to OTHER NY P2P"
     hello-time 2
     max-age 20
     priority 0
     stp false
}
 bridge br62 {
     aging 300
     description "L2 VPN Bridge NUM2 to UK P2P"
     hello-time 2
     max-age 20
     priority 0
     stp false
 }
 bridge br72 {
     aging 300
     description "L2 VPN Bridge NUM2 to NYC"
     hello-time 2
     max-age 20
     priority 0
     stp false
 }
 ethernet eth0 {
     address 1.2.3.4.170/29
     address 1.2.3.4.171/29
     address 1.2.3.4.172/29
     address 1.2.3.4.173/29
     duplex auto
     firewall {
         in {
             name ProtectP2PRouter
         }
         local {
             name ProtectP2PRouter
         }
     }
     hw-id ac:1f:6b:80:be:06
     mtu 4000
     offload-options {
         generic-receive on
         generic-segmentation on
         scatter-gather on
         tcp-segmentation on
     }
     smp-affinity auto
     speed auto
 }
 loopback lo {
     address 10.255.3.102/32
 }
 tunnel tun33 {
     description "GRE bridge to NY P2P router"
     encapsulation gre-bridge
     local-ip 1.2.3.4.171
     multicast enable
     parameters {
         ip {
             bridge-group {
                 bridge br33
             }
         }
     }
     policy {
         route MSS-CLAMP-P2P
     }
     remote-ip 6.7.8.130
 }
 tunnel tun36 {
     description "GRE bridge to NYC2 for UK"
     encapsulation gre-bridge
     local-ip 1.2.3.4.171
     multicast enable
     parameters {
         ip {
             bridge-group {
                 bridge br62
             }
         }
     }
     policy {
         route MSS-CLAMP-P2P
     }
     remote-ip 2.3.4.90
 }
 tunnel tun37 {
     description "GRE bridge to NYC2 Router via DECIX for LA"
     encapsulation gre-bridge
     local-ip 1.2.3.4.172
     multicast enable
     parameters {
         ip {
             bridge-group {
                 bridge br72
             }
         }
     }
     policy {
         route MSS-CLAMP-P2P
     }
     remote-ip 2.3.4.90
 }
 tunnel tun326 {
     description "GRE bridge to UK-P2P FOR NYC2"
     encapsulation gre-bridge
     local-ip 1.2.3.4.171
     multicast enable
     parameters {
         ip {
             bridge-group {
                 bridge br62
             }
         }
     }
     policy {
         route MSS-CLAMP-P2P
     }
     remote-ip 11.12.13.146
 }
 tunnel tun327 {
     description "GRE bridge to LA1 for NYC"
     encapsulation gre-bridge
     local-ip 1.2.3.4.172
     multicast enable
     parameters {
         ip {
             bridge-group {
                 bridge br72
             }
         }
     }
     policy {
         route MSS-CLAMP-P2P
     }
     remote-ip 15.16.17.5
 }

}
policy {

route MSS-CLAMP-P2P {
    rule 10 {
        protocol tcp
        set {
            tcp-mss 1460
        }
        tcp {
            flags SYN
        }
    }
}

}
protocols {

static {
    route 0.0.0.0/0 {
        next-hop 1.2.3.4.169 {
        }
    }
}

}
service {

ssh {
}

}
system {

config-management {
    commit-revisions 100
}
console {
    device ttyS0 {
        speed 9600
    }
}
host-name NY2-P2P
login {
    user vyos {
        authentication {
            encrypted-password ****************
            plaintext-password ****************
        }
        level admin
    }
}
name-server 8.8.8.8
name-server 8.8.4.4
ntp {
    server 0.pool.ntp.org {
    }
    server 1.pool.ntp.org {
    }
    server 2.pool.ntp.org {
    }
}
syslog {
        set {
            tcp-mss 1460
        }
        tcp {
            flags SYN
        }
    }
}

}
protocols {

static {
    route 0.0.0.0/0 {
        next-hop 1.2.3.4.169 {
        }
    }
}

}
service {

ssh {
}

}
system {

config-management {
    commit-revisions 100
}
console {
    device ttyS0 {
        speed 9600
    }
}
host-name NY2-P2P
login {
    user vyos {
        authentication {
            encrypted-password ****************
            plaintext-password ****************
        }
        level admin
    }
}
name-server 8.8.8.8
name-server 8.8.4.4
ntp {
    server 0.pool.ntp.org {
    }
    server 1.pool.ntp.org {
    }
    server 2.pool.ntp.org {
    }
}
syslog {
    global {
        facility all {
            level notice
        }
        facility protocols {
            level debug
        }
    }
}
time-zone UTC

}

Could you please do it via: 'show config command'. thx.

I can test on Saturdays when our maintenance window is open btw.

set interfaces bridge br33 aging '300'
set interfaces bridge br33 description 'L2 VPN Bridge to OTHER NY P2P'
set interfaces bridge br33 hello-time '2'
set interfaces bridge br33 max-age '20'
set interfaces bridge br33 priority '0'
set interfaces bridge br33 stp 'false'
set interfaces bridge br62 aging '300'
set interfaces bridge br62 description 'L2 VPN Bridge NUM2 to UK P2P'
set interfaces bridge br62 hello-time '2'
set interfaces bridge br62 max-age '20'
set interfaces bridge br62 priority '0'
set interfaces bridge br62 stp 'false'
set interfaces bridge br72 aging '300'
set interfaces bridge br72 description 'L2 VPN Bridge NUM2 to NYC'
set interfaces bridge br72 hello-time '2'
set interfaces bridge br72 max-age '20'
set interfaces bridge br72 priority '0'
set interfaces bridge br72 stp 'false'
set interfaces ethernet eth0 address '1.2.3.170/29'
set interfaces ethernet eth0 address '1.2.3.171/29'
set interfaces ethernet eth0 address '1.2.3.172/29'
set interfaces ethernet eth0 address '1.2.3.173/29'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 firewall in name 'ProtectP2PRouter'
set interfaces ethernet eth0 firewall local name 'ProtectP2PRouter'
set interfaces ethernet eth0 hw-id 'ac:1f:6b:80:be:06'
set interfaces ethernet eth0 mtu '4000'
set interfaces ethernet eth0 offload-options generic-receive 'on'
set interfaces ethernet eth0 offload-options generic-segmentation 'on'
set interfaces ethernet eth0 offload-options scatter-gather 'on'
set interfaces ethernet eth0 offload-options tcp-segmentation 'on'
set interfaces ethernet eth0 smp-affinity 'auto'
set interfaces ethernet eth0 speed 'auto'
set interfaces loopback lo address '10.255.3.102/32'
set interfaces tunnel tun33 description 'GRE bridge to NY P2P router'
set interfaces tunnel tun33 encapsulation 'gre-bridge'
set interfaces tunnel tun33 local-ip '1.2.3.171'
set interfaces tunnel tun33 multicast 'enable'
set interfaces tunnel tun33 parameters ip bridge-group bridge 'br33'
set interfaces tunnel tun33 policy route 'MSS-CLAMP-P2P'
set interfaces tunnel tun33 remote-ip '4.5.6.130'
set interfaces tunnel tun36 description 'GRE bridge to NYC2 for UK'
set interfaces tunnel tun36 encapsulation 'gre-bridge'
set interfaces tunnel tun36 local-ip '1.2.3.171'
set interfaces tunnel tun36 multicast 'enable'
set interfaces tunnel tun36 parameters ip bridge-group bridge 'br62'
set interfaces tunnel tun36 policy route 'MSS-CLAMP-P2P'
set interfaces tunnel tun36 remote-ip '7.8.9.90'
set interfaces tunnel tun37 description 'GRE bridge to NYC2 Router via DECIX for LA'
set interfaces tunnel tun37 encapsulation 'gre-bridge'
set interfaces tunnel tun37 local-ip '1.2.3.172'
set interfaces tunnel tun37 multicast 'enable'
set interfaces tunnel tun37 parameters ip bridge-group bridge 'br72'
set interfaces tunnel tun37 policy route 'MSS-CLAMP-P2P'
set interfaces tunnel tun37 remote-ip '7.8.9.90'
set interfaces tunnel tun326 description 'GRE bridge to UK-P2P FOR NYC2'
set interfaces tunnel tun326 encapsulation 'gre-bridge'
set interfaces tunnel tun326 local-ip '1.2.3.171'
set interfaces tunnel tun326 multicast 'enable'
set interfaces tunnel tun326 parameters ip bridge-group bridge 'br62'
set interfaces tunnel tun326 policy route 'MSS-CLAMP-P2P'
set interfaces tunnel tun326 remote-ip '11.12.13.146'
set interfaces tunnel tun327 description 'GRE bridge to LA1 for NYC'
set interfaces tunnel tun327 encapsulation 'gre-bridge'
set interfaces tunnel tun327 local-ip '1.2.3.172'
set interfaces tunnel tun327 multicast 'enable'
set interfaces tunnel tun327 parameters ip bridge-group bridge 'br72'
set interfaces tunnel tun327 policy route 'MSS-CLAMP-P2P'
set interfaces tunnel tun327 remote-ip '15.16.17.5'
set policy route MSS-CLAMP-P2P rule 10 protocol 'tcp'
set policy route MSS-CLAMP-P2P rule 10 set tcp-mss '1460'
set policy route MSS-CLAMP-P2P rule 10 tcp flags 'SYN'

Bug confirmed, I can reproduce it. I can't tell you how long it takes to fix it, it's all old code base.

hagbard changed the task status from Open to In progress.Dec 12 2018, 7:59 PM

Hi @Barrysdca , can you please test if the issue persists with https://downloads.vyos.io/rolling/current/amd64/vyos-1.2.0-rolling%2B201812120337-amd64.iso
I tested it on the image and it appears that I can't reproduce it anymore.

@Barrysdca Can you please test with the latest rolling release, please?

hagbard changed the task status from In progress to On hold.Dec 18 2018, 6:06 PM

I can test this on Saturday. Sorry was sick last few days, couldn't test anything.

Hi @Barrysdca did you have a chance to test again?

tested with most recent rolling version. problem still persists but it's not throwing errors.

tun33 - u/u GRE bridge to NY P2P router
tun36 - u/u GRE bridge to NYC2 for UK
tun37 - u/u GRE bridge to NYC2 Router via DECIX for LA
tun326 - u/u GRE bridge to UK-P2P FOR NYC2
tun327 - u/u GRE bridge to LA1 for NYC
vyos@NY2-P2P:~$ config
[edit]
vyos@NY2-P2P# delete interfaces tunnel tun37
[edit]
vyos@NY2-P2P# commit
[edit]
vyos@NY2-P2P# exit
Warning: configuration changes have not been saved.
exit
vyos@NY2-P2P:~$ show int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description


br33 - u/D L2 VPN Bridge to OTHER NY P2P
br62 - u/D L2 VPN Bridge NUM2 to UK P2P
br72 - u/D L2 VPN Bridge NUM2 to NYC
eth0 ..../29 u/u

eth1 - u/D
lo 127.0.0.1/8 u/u

10.255.3.102/32
::1/128

deleted all tunnels

$ show version
Version: VyOS 1.2.0-rolling+201812300337
Built by: [email protected]
Built on: Sun 30 Dec 2018 03:37 UTC
Build ID: ab13c0ce-78da-49ff-aa63-d96c379c15af

Architecture: x86_64
Boot via: installed image
System type: bare metal

Hardware vendor: Supermicro
Hardware model: Super Server
Hardware S/N: Unknown
Hardware UUID: Unknown

Copyright: VyOS maintainers and contributors

hagbard changed the task status from On hold to In progress.Jan 7 2019, 10:29 PM

Sorry for the delay @Barrysdca , please test the rolling release January 8th. or alternativly you can install http://dev.packages.vyos.net/repositories/current/vyos/pool/main/v/vyatta-cfg-system/vyatta-cfg-system_0.20.44+vyos2+current17_amd64.deb as well, which should fix the issue.
Please provide feedback as soon as you can, I tested the config you have posted above and everything appears to be working well now with the new package.

hagbard changed the task status from In progress to Needs testing.Jan 7 2019, 11:21 PM

np. I can test this on Saturday. Let me know anything else needing testing please. My window to test in this live environment is very narrow. I'll download the rolling release just prior.

Do you know when/where I receive my access to download the release? I thought people who contribute are to get a login, but I didn't see mine. maybe blocked by spam filter. not sure, nor do I know where to ask... thanks

That's all to test. I did test it based on the config you provide above, I just want to see if there are any corner case I did not consider.