Page MenuHomeVyOS Platform

Throughput anomalies
Closed, ResolvedPublicBUG

Description

I have found some weird behaviors at VyOS when doing simple routing.

In a GNS3 lab running on a local computer, there are clearly two different levels of throughput as shown in my attached tests. They also vary depending on the VyOS version.

In an EVE-NG lab running in a Sentrium server, throughput is very different depending on the VyOS version.

I attach a document with all the tests I have done so far, which shows some patterns of those irregular behaviors.

.

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.2.4, VyOS 1.2.5epa1, VyOS 1.3 Rolling
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Unknown Object (User) created this task.Feb 19 2020, 1:34 PM
Unknown Object (User) added a comment.EditedFeb 19 2020, 10:22 PM

It would be useful if someone could do similar tests using GNS3 in a powerful computer.

Doing the tests using EVE-NG in a low-resource computer, would be helpful too.

The test is just to run several times iperf between every possible pair of spokes and take note of the average throughput or throughputs (sometimes there are clearly two different averages depending on the pair of spokes used). That should be tested for both Rolling and LTS.

Please find below the only config needed:

---HUB---
set int eth eth0 addr 192.168.0.1/24
set int eth eth1 addr 192.168.1.1/24
set int eth eth2 addr 192.168.2.1/24
set int eth eth3 addr 192.168.3.1/24
set prot ospf area 0 network 192.168.0.0/24
set prot ospf area 0 network 192.168.1.0/24
set prot ospf area 0 network 192.168.2.0/24
set prot ospf area 0 network 192.168.3.0/24
---SPOKE0---
set int eth eth0 addr 192.168.0.10/24
set prot ospf area 0 network 192.168.0.0/24
---SPOKE1---
set int eth eth0 addr 192.168.1.10/24
set prot ospf area 0 network 192.168.1.0/24
---SPOKE2---
set int eth eth0 addr 192.168.2.10/24
set prot ospf area 0 network 192.168.2.0/24
---SPOKE3---
set int eth eth0 addr 192.168.3.10/24
set prot ospf area 0 network 192.168.3.0/24

Please check the attached .odt document or ask me if there is any doubt. Thanks.

Santi

Unknown Object (User) added a comment.Feb 24 2020, 11:43 PM

Robert did new tests on his GNS3 lab (results attached).

He did a similar test to mine.
With Crux he got the same wired pattern I got on Test9 of my tests.
With Rolling he got no weird pattern, but very bad performance.

But then he did a second test changing all NIC drivers to virtio-net-pci. And then both weird pattern and bad performance were gone.

However, even with the virtio-net-pci driver, the Rolling version keeps giving worse throughput results than Crux, same as I had in my test6 and test7.

I saw that EVE-NG was using virtio-net-pci too, so I tested GNS3 with different drivers and EVE-NG with different drivers: The patterns showing different throughput seem to happen only when using the Intel e1000 driver.

So now there are two things to investigate:

The problem with the Intel e1000 driver (the cause of the patterns of different throughputs).

The throughput difference between crux and rolling.

erkin set Issue type to Bug (incorrect behavior).Aug 31 2021, 5:37 PM
syncer reassigned this task from Unknown Object (User) to Viacheslav.Jul 16 2023, 9:29 PM
syncer added a subscriber: Unknown Object (User).

Fixed with enabling offloads https://vyos.dev/T3619

vyos@r14# set interfaces ethernet eth0 offload 
Possible completions:
   gro                  Enable Generic Receive Offload
   gso                  Enable Generic Segmentation Offload
   lro                  Enable Large Receive Offload
   rfs                  Enable Receive Flow Steering
   rps                  Enable Receive Packet Steering
   sg                   Enable Scatter-Gather
   tso                  Enable TCP Segmentation Offloading