Page MenuHomeVyOS Platform
Create Task
Maniphest T5236

show wan-load-balance not working after reboot
Closed, InvalidPublicBUG
Actions

Description

Configured load-balancing:

set load-balancing wan flush-connections
set load-balancing wan interface-health eth0 failure-count '3'
set load-balancing wan interface-health eth0 nexthop '10.88.89.254'
set load-balancing wan interface-health eth0 success-count '5'
set load-balancing wan interface-health eth0 test 1 target '1.1.1.1'
set load-balancing wan interface-health eth0 test 2 target '8.8.8.8'
set load-balancing wan interface-health eth3 failure-count '10'
set load-balancing wan interface-health eth3 nexthop 'dhcp'
set load-balancing wan interface-health eth3 success-count '10'
set load-balancing wan interface-health eth3 test 1 target '1.1.1.1'
set load-balancing wan interface-health eth3 test 2 target '8.8.8.8'
set load-balancing wan rule 1 failover
set load-balancing wan rule 1 inbound-interface 'eth1.10'
set load-balancing wan rule 1 interface eth0 weight '100'
set load-balancing wan rule 1 interface eth3 weight '10'

After commit:

danhusan@vyos:~$ show wan-load-balance 
Interface:  eth0
  Status:  active
  Last Status Change:  Mon May 22 16:01:35 2023
  +Test:  ping  Target: 1.1.1.1
   Test:  
    Last Interface Success:  0s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0

Interface:  eth3
  Status:  active
  Last Status Change:  Mon May 22 16:01:35 2023
  +Test:  ping  Target: 1.1.1.1
   Test:  
    Last Interface Success:  0s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0

After rebooting show command stops working - however failover still occurs as it should.

danhusan@vyos:~$ show wan-load-balance 
WAN load balancing is not configured

danhusan@vyos:~$ ps ax|grep lb
   3500 ?        Ss     0:00 /opt/vyatta/sbin/wan_lb -f /run/load-balance/wlb.conf -d -i /var/run/vyatta/wlb.pid
   3501 ?        S      0:00 /opt/vyatta/sbin/wan_lb -f /run/load-balance/wlb.conf -d -i /var/run/vyatta/wlb.pid
   3655 pts/0    S+     0:00 grep lb

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202305220317
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

danhusan created this task.May 22 2023, 4:06 PM
pasik added a subscriber: pasik.May 22 2023, 5:32 PM
danhusan added a comment.May 22 2023, 7:42 PM

Further testing reveals that re-connecting the interfaces after reboot solves the issue:

danhusan@vyos:~$ show wan-load-balance 
WAN load balancing is not configured

<<unplugging and reconnecting eth0 and eth3 >>

danhusan@vyos:~$ show wan-load-balance 
Interface:  eth0
  Status:  active
  Last Status Change:  Mon May 22 19:41:26 2023
  -Test:  ping  Target: 1.1.1.1
  +Test:  ping  Target: 8.8.8.8
    Last Interface Success:  0s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0

Interface:  eth3
  Status:  active
  Last Status Change:  Mon May 22 19:41:26 2023
  -Test:  ping  Target: 1.1.1.1
  +Test:  ping  Target: 8.8.8.8
    Last Interface Success:  0s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0
Viacheslav added a subscriber: Viacheslav.May 23 2023, 9:26 AM

@danhusan Could you check if the file /var/run/load-balance/wlb.out exists?
https://github.com/vyos/vyatta-wanloadbalance/blob/5a3ab6c426928644dc9ad9a70296263781523919/templates-op/show/wan-load-balance/node.def#L2C15-L5

danhusan added a comment.May 23 2023, 12:47 PM
danhusan@vyos-1:~$ cat /var/run/load-balance/wlb.out
cat: /var/run/load-balance/wlb.out: No such file or directory
Viacheslav changed the task status from Open to In progress.May 25 2023, 8:39 AM
Viacheslav claimed this task.
Viacheslav added a project: VyOS 1.4 Sagitta.
Viacheslav added a comment.May 25 2023, 8:59 AM

I can't reproduce it VyOS 1.4-rolling-202305250317

set load-balancing wan flush-connections
set load-balancing wan interface-health eth0 failure-count '3'
set load-balancing wan interface-health eth0 nexthop '192.168.122.1'
set load-balancing wan interface-health eth0 success-count '5'
set load-balancing wan interface-health eth0 test 1 target '1.1.1.1'
set load-balancing wan interface-health eth0 test 2 target '8.8.8.8'
set load-balancing wan interface-health eth2 failure-count '10'
set load-balancing wan interface-health eth2 nexthop 'dhcp'
set load-balancing wan interface-health eth2 success-count '10'
set load-balancing wan interface-health eth2 test 1 target '1.1.1.1'
set load-balancing wan interface-health eth2 test 2 target '8.8.8.8'
set load-balancing wan rule 1 failover
set load-balancing wan rule 1 inbound-interface 'eth1.10'
set load-balancing wan rule 1 interface eth0 weight '100'
set load-balancing wan rule 1 interface eth2 weight '10'

After rebooting the output works, but maybe some timeout is required, after reboot ~10-15sec until file is created.

Output:

vyos@r14:~$ 
vyos@r14:~$ uptime
 11:57:33 up 3 min,  1 user,  load average: 0.13, 0.30, 0.15
vyos@r14:~$ 
vyos@r14:~$ 
vyos@r14:~$ show wan-load-balance 
Interface:  eth0
  Status:  active
  Last Status Change:  Thu May 25 11:55:12 2023
  +Test:  ping  Target: 1.1.1.1
   Test:  
    Last Interface Success:  5s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0

Interface:  eth2
  Status:  failed
  Last Status Change:  Thu May 25 11:57:38 2023
  -Test:  ping  Target: 1.1.1.1
  -Test:  ping  Target: 8.8.8.8
    Last Interface Success:  n/a                
    Last Interface Failure:  0s 
    # Interface Failure(s):  10

vyos@r14:~$ 
vyos@r14:~$ ls -la /var/run/lo
load-balance/ lock/         log/          
vyos@r14:~$ ls -la /var/run/load-balance/
total 8
drwxr-xr-x  2 root root        80 May 25 11:55 .
drwxr-xr-x 42 root root      1260 May 25 11:54 ..
-rw-r--r--  1 root root       963 May 25 11:54 wlb.conf
-rw-r--r--  1 root vyattacfg  471 May 25 11:57 wlb.out
vyos@r14:~$

@danhusan Could you check which files in this directory?

ls -la /var/run/load-balance/
danhusan added a comment.May 25 2023, 4:22 PM

I could have sworn it took quite a while for the wlb.out to be populated when testing earlier on. However as I´ve built out the config a bit more I am now only able to reproduce it a few seconds after reboot - so your theory holds. Let's close out the bug report for now and I´ll reopen if something changes down the line.

Thanks for helping.

danhusan closed this task as Invalid.May 25 2023, 4:30 PM