That's odd, I really wonder what the difference between our environments is. I'm running all tests inside the build container directly calling "sudo make <target>" one after another for the targets:
- Feed Queries
- All Stories
- Search
- Feed Search
- Transactions
- Transaction Logs
Feed Search
Jul 11 2025
Jul 11 2025
In our environment, all tests passed as during build, as installed on the test VM
DEBUG - Running Testcase: /usr/libexec/vyos/tests/smoke/cli/test_vrf.py DEBUG - test_vrf_assign_interface (__main__.VRFTest.test_vrf_assign_interface) ... ok DEBUG - test_vrf_bind_all (__main__.VRFTest.test_vrf_bind_all) ... ok DEBUG - test_vrf_conntrack (__main__.VRFTest.test_vrf_conntrack) ... ok DEBUG - test_vrf_disable_forwarding (__main__.VRFTest.test_vrf_disable_forwarding) ... ok DEBUG - test_vrf_ip_ipv6_nht (__main__.VRFTest.test_vrf_ip_ipv6_nht) ... ok DEBUG - test_vrf_ip_ipv6_protocol_non_existing_route_map (__main__.VRFTest.test_vrf_ip_ipv6_protocol_non_existing_route_map) ... ok DEBUG - test_vrf_ip_protocol_route_map (__main__.VRFTest.test_vrf_ip_protocol_route_map) ... ok DEBUG - test_vrf_ipv6_protocol_route_map (__main__.VRFTest.test_vrf_ipv6_protocol_route_map) ... ok DEBUG - test_vrf_link_local_ip_addresses (__main__.VRFTest.test_vrf_link_local_ip_addresses) ... ok DEBUG - test_vrf_loopbacks_ips (__main__.VRFTest.test_vrf_loopbacks_ips) ... ok DEBUG - test_vrf_static_route (__main__.VRFTest.test_vrf_static_route) ... ok DEBUG - test_vrf_table_id_is_unalterable (__main__.VRFTest.test_vrf_table_id_is_unalterable) ... ok DEBUG - test_vrf_vni_add_change_remove (__main__.VRFTest.test_vrf_vni_add_change_remove) ... ok DEBUG - test_vrf_vni_and_table_id (__main__.VRFTest.test_vrf_vni_and_table_id) ... ok DEBUG - test_vrf_vni_duplicates (__main__.VRFTest.test_vrf_vni_duplicates) ... ok DEBUG - DEBUG - ---------------------------------------------------------------------- DEBUG - Ran 15 tests in 308.260s DEBUG -
For example, the latest rolling:
vyos@r14:~$ /usr/libexec/vyos/tests/smoke/cli/test_vrf.py test_vrf_assign_interface (__main__.VRFTest.test_vrf_assign_interface) ... ok test_vrf_bind_all (__main__.VRFTest.test_vrf_bind_all) ... ok test_vrf_conntrack (__main__.VRFTest.test_vrf_conntrack) ... ok test_vrf_disable_forwarding (__main__.VRFTest.test_vrf_disable_forwarding) ... ok test_vrf_ip_ipv6_nht (__main__.VRFTest.test_vrf_ip_ipv6_nht) ... ok test_vrf_ip_ipv6_protocol_non_existing_route_map (__main__.VRFTest.test_vrf_ip_ipv6_protocol_non_existing_route_map) ... ok test_vrf_ip_protocol_route_map (__main__.VRFTest.test_vrf_ip_protocol_route_map) ... ok test_vrf_ipv6_protocol_route_map (__main__.VRFTest.test_vrf_ipv6_protocol_route_map) ... ok test_vrf_link_local_ip_addresses (__main__.VRFTest.test_vrf_link_local_ip_addresses) ... ok test_vrf_loopbacks_ips (__main__.VRFTest.test_vrf_loopbacks_ips) ... ok test_vrf_static_route (__main__.VRFTest.test_vrf_static_route) ... ok test_vrf_table_id_is_unalterable (__main__.VRFTest.test_vrf_table_id_is_unalterable) ... ok test_vrf_vni_add_change_remove (__main__.VRFTest.test_vrf_vni_add_change_remove) ... ok test_vrf_vni_and_table_id (__main__.VRFTest.test_vrf_vni_and_table_id) ... ok test_vrf_vni_duplicates (__main__.VRFTest.test_vrf_vni_duplicates) ... ok
Any updates on this? Just re-ran the tests to discover there are quite a lot of other/new failures. I also noticed the nightly builds workflow to be disabled since 4 months. Still there are recent nightly releases available in that repository. Did you internally switch to some other workflow that's not public (yet)? Otherwise I'd not assume you've quit running the tests yourself.
Seems fixed
<pre>vyos@r16# set load-balancing wan interface-health eth1 nexthop 192.0.2.1 [edit] vyos@r16# commit [edit] vyos@r16# exit Warning: configuration changes have not been saved. exit vyos@r16:~$ vyos@r16:~$ show wan-load-balance Interface: eth1 Status: active Last Status Change: N/A Last Interface Success: N/A Last Interface Failure: N/A Interface Failures: 0
Fix of commit-config after changes: https://github.com/vyos/vyatta-cfg/pull/107
Jul 10 2025
Jul 10 2025
dmbaturin edited projects for T7544: Trying to create a VRF named "vni" leads to an unhandled exception, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus.
jestabro added a project to T7524: The aws-gwlbtun service cannot start: VyOS 1.5 Circinus (1.5-stream-2025-Q3).
Jul 9 2025
Jul 9 2025
jestabro moved T5400: Move libvyosconfig build out of the Docker image from Backlog - Feature Requests to Completed on the VyOS Rolling board.
jestabro moved T5400: Move libvyosconfig build out of the Docker image from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T6736: Make REST api and GraphQL api distinct nodes from Need Triage to Completed on the VyOS Rolling board.
jestabro moved T6736: Make REST api and GraphQL api distinct nodes from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T6968: Issue with Configuration Migration from VyOS 1.3.8 to 1.4.1 from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T7119: Fix misleading debug message from vyos-configd from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T7289: Add libvyosconfig as git submodule of vyos-1x for stream builds from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T7294: image upgrade will replace symlinks with a copy from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
jestabro moved T7294: image upgrade will replace symlinks with a copy from Backlog to Finished on the VyOS 1.4 Sagitta (1.4.2) board.
Unknown Object (User) triaged T7612: Flowspec `local-install` CLI command is dysfunctional as Normal priority.
jestabro moved T7359: Image update: confirm image name is available before accepting from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
natali-rs1985 added a comment to T6909: Move "show monitoring" and "show zebra" to "show monitoring frr".
jestabro closed T7499: Update config merge tool, a subtask of T7427: Replace legacy commit algorithm and data structures with vyconfd, as Resolved.
jestabro closed T7499: Update config merge tool, a subtask of T7498: Extend HTTP API load/merge request to allow passing config in body, as Resolved.
natali-rs1985 moved T6852: Add op mode command "show dhcpv6 server statistics" from Backport Candidates to Completed on the VyOS Rolling board.
dmbaturin moved T7430: Add root-guard and bpdu-guard option to bridge interfaces from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6144: Updating the system image without enough space for the files can break the system from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
Jul 9 2025, 1:30 PM · VyOS 1.5 Circinus (1.5-stream-2025-Q2), VyOS 1.4 Sagitta (1.4.3), Restricted Project, VyOS Rolling
dmbaturin renamed T7430: Add root-guard and bpdu-guard option to bridge interfaces from Add root-guard and bpdu-guard to Bridge implemenation to Add root-guard and bpdu-guard option to bridge interfaces.
dmbaturin moved T7466: Bonding interface mode allows malformed variations of 802.3ad from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7510: OSPF NSSA translation error from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7512: Zone-based firewalls fails on VLAN aware bridges from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7538: Delete the "run show login level" command from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin edited projects for T7358: firewall: Allow offload in global options., added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin edited projects for T7410: The tech support script fails on machines without a USB controller, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin moved T7412: containers: Allow privileged option from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin edited projects for T7386: Remote-Groups for IPv6 CIDRs, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin edited projects for T7450: Use PCRE2 in validation utilities, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin moved T7492: Modem connection code doesn't work from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin edited projects for T7484: ARM64 config fails to commit due to ttyS0 console, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin moved T7511: Unable to apply OpenConnect RADIUS accounting settings from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6679: Add support for NAT66 destination groups from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7539: wwan: extend smoketests to cover WWAN driver option and hwsim from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin edited projects for T7540: Move "clear connection" to "reset connection", added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin edited projects for T7545: Command 'show vpn debug peer <peer_name>' does not work correctly, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin edited projects for T7558: Allow for specification of the `dont-query` parameter for PowerDNS, added: VyOS Rolling, VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin edited projects for T7562: Command 'set vpn ipsec disable-uniqreqids' does nothing, added: VyOS 1.5 Circinus (1.5-stream-2025-Q3); removed VyOS 1.5 Circinus (1.5-stream-2025-Q2).
dmbaturin moved T4627: Ability to set host part IPv6 address via interface IP token from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T5735: Add CLI and configuration scripts for stunnel from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6031: static-route option breaks DHCP server from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6430: Allow larger table ids in policy route from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6490: Allow creation of wireguard interfaces without requiring peers from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6620: Allow using list_nodes on paths that do not exist from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T6918: PPPoE traffic is classified as invalid from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin renamed T6918: PPPoE traffic is classified as invalid from pppoe traffic is classified as invalid to PPPoE traffic is classified as invalid.
dmbaturin moved T6949: blackbox_exporter for probing endpoints from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7049: Boot failure after installing on RAID1 from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7159: Display the non-production banner depending on the build type from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin moved T7270: syslog: typos in rsyslog.conf from Open to Finished on the VyOS 1.5 Circinus (1.5-stream-2025-Q2) board.
dmbaturin renamed T7567: WAN load balancing information is not shown immediately after startup from WAN Load Balancing show command broken to WAN load balancing information is not shown immediately after startup.
dmbaturin changed Is it a breaking change? from none to compatible on T7573: pki: TypeError: argument of type 'NoneType' is not iterable when HAProxy is not in use.
dmbaturin changed Is it a breaking change? from none to compatible on T7531: Add FRR no-ipv6-auto-ra option.
dmbaturin edited projects for T6144: Updating the system image without enough space for the files can break the system, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus, Bugs.
Jul 9 2025, 1:12 PM · VyOS 1.5 Circinus (1.5-stream-2025-Q2), VyOS 1.4 Sagitta (1.4.3), Restricted Project, VyOS Rolling
dmbaturin edited projects for T6951: Add a configuration command for ethertypes that bridge firewalls should always accept, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus.
dmbaturin edited projects for T6968: Issue with Configuration Migration from VyOS 1.3.8 to 1.4.1, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus.
dmbaturin renamed T6927: Allow setting name server for containers from allow setting name server for container to Allow setting name server for containers.
dmbaturin edited projects for T7350: Do not allow deleting interfaces referenced in flowtables, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus.
dmbaturin renamed T5636: Add GeoIP matching support for policy route from Need geoip option for policy route to Add GeoIP matching support for policy route.
dmbaturin edited projects for T7056: OpenVPN error : Unable to bind the tunnel interface to bridge if misconfigured first, added: VyOS 1.5 Circinus (1.5-stream-2025-Q2); removed VyOS 1.5 Circinus.
dmbaturin changed Is it a breaking change? from none to compatible on T7523: PPPoE Discovery and Wake-on-LAN Traffic marked invalid on bridge interfaces.
dmbaturin changed Is it a breaking change? from none to compatible on T7492: Modem connection code doesn't work.
dmbaturin renamed T7510: OSPF NSSA translation error from ospf nssa translation error to OSPF NSSA translation error.
dmbaturin renamed T6679: Add support for NAT66 destination groups from NAT66 destination group to Add support for NAT66 destination groups.
dmbaturin changed Is it a breaking change? from none to compatible on T7514: Smoketest QoS limiter fails.
dmbaturin changed Is it a breaking change? from none to compatible on T7511: Unable to apply OpenConnect RADIUS accounting settings.