Page MenuHomeVyOS Platform

Every log message is recorded twice: rsyslog loads imuxsock and imjournal while journald forwards to syslog
In progress, NormalPublicBUG

Description

Summary

VyOS ships two independent paths from journald into rsyslog, so every log entry is stored twice locally and transmitted twice to any remote syslog target.

data/templates/rsyslog/rsyslog.conf.j2 loads both input modules:

module(load="imuxsock")
module(load="imjournal" StateFile="/var/spool/rsyslog/imjournal.state" ...)

/etc/systemd/journald.conf ships with ForwardToSyslog=yes. This is a modification to the Debian conffile, which ships with every option commented out:

$ sudo dpkg --verify systemd | grep journald
??5?????? c /etc/systemd/journald.conf

The file's mtime is the image build date (2026-03-19 07:03:08, zero nanoseconds), not a date from the running system, so it arrived with the image rather than being edited locally.

journald therefore forwards each entry to /run/systemd/journal/syslog, where imuxsock collects it, while imjournal independently reads the same entry from the journal.

Reproduce

No configuration change required. Default install, local logging only:

$ logger -t duptest "single message reproducer"
$ sudo grep duptest /var/log/messages
Aug 9 18:38:05 vyos duptest: single message reproducer
Aug 9 18:38:05 vyos duptest[24467]: single message reproducer

Actual: two entries per event. Expected: one entry per event.

logger emits exactly one message, so this is not an application logging twice. The two lines differ only in the tag — one carries the PID, one does not — which identifies the mechanism: imjournal renders SYSLOG_PID from the journal's structured fields, while the socket-forwarded copy arrives without it. A single message written by two rsyslog actions would be byte-identical.

Not caused by having both local and remote targets

Both copies appear in the same destination (/var/log/messages), and the duplication is present with only local logging configured — the reproducer above requires no system syslog remote. On a system that does have a remote target, the remote receives two copies as well.

Impact

Doubles local log storage.
Doubles bytes sent to every remote syslog target, and doubles storage consumed on the collector. Where the collector has a size or retention cap, effective retention is halved.
Any downstream analysis that counts events — rate limiting, alerting on repetition, SIEM correlation — sees each event twice.

Suggested fix

Either is sufficient.

(a) Ship ForwardToSyslog=no, or a drop-in under /etc/systemd/journald.conf.d/, leaving imjournal as the single source. This is the smaller change and does not affect any other consumer.

(b) Do not load imuxsock. Under systemd, /dev/log is journald's socket, so programs logging via syslog(3) reach rsyslog through imjournal regardless. Here imuxsock binds /run/systemd/journal/syslog and receives only what journald forwards — i.e. the duplicate.

Caveat on (b): /etc/rsyslog.d/49-haproxy.conf uses $AddUnixListenSocket for its own socket, which is an imuxsock directive. If that file is shipped by VyOS, removing the module would need that handled. This has not been verified, which is why (a) is offered first.

Workaround

sudo mkdir -p /etc/systemd/journald.conf.d
printf '[Journal]\nForwardToSyslog=no\n' | \

sudo tee /etc/systemd/journald.conf.d/10-no-syslog-forward.conf

sudo systemctl restart systemd-journald rsyslog

This does not survive add system image: a new image starts from a fresh squashfs, so the duplication returns silently after an upgrade.

Details

Version
2026.3
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

Viacheslav triaged this task as Normal priority.Aug 10 2026, 3:07 PM

I hit this on my own fleet and put up a PR for fix (a): https://github.com/vyos/vyos-build/pull/1304. It sets ForwardToSyslog=no in data/live-build-config/includes.chroot/etc/systemd/journald.conf, which is where the image gets its journald.conf. That line came in with vyos-build bc0406ce3 (T552, 2018), back when the rsyslog config only loaded imuxsock and imklog. imjournal showed up later, in vyos-1x e143e496e (T6989, 2025-01-13).

I see it on all five of my routers on 2026.09.11-1436-rolling. Every service_name except kernel ships at exactly 2x, and that holds for the 30 days of logs I can query. My busiest router wastes about 30,000 lines a day on the duplicate, 42% of its syslog volume.

I tested the fix as a journald drop-in on one router, with a second router left alone as a control. One logger -t duptest line gave me 1 copy from the patched router and 2 from the control. Nothing else stopped arriving, and rsyslog did not need a restart. Short test, not a long soak.

On the haproxy caveat in the report, for fix (b): /etc/rsyslog.d/49-haproxy.conf does ship, from Debian's haproxy package rather than VyOS, and it does use $AddUnixListenSocket. Take out the imuxsock load and rsyslogd -N1 fails with invalid or yet-unknown config file command 'AddUnixListenSocket' - have you forgotten to load a module?. VyOS runs haproxy chrooted at /var/lib/haproxy with log /dev/log, so that socket is the only way its messages get out. I did not watch haproxy lose logs, since it is off on my routers, but the chroot and the socket ownership say it would. So (a) looks like the only safe option to me.

Heads up on one side effect: with the forward off, journald's per-unit rate limit applies to what rsyslog gets. Today the forwarded copy skips it.

Viacheslav changed the task status from Open to In progress.Wed, Sep 16, 7:52 AM
Viacheslav assigned this task to cr0ntab.