Systemd 246 compatibility

[kh34]

We ran into an issue after upgrading to systemd 246 (Flatcar Linux 2605.10.0) where no logs were returned. There was no error messages in the logs. We were able to get it to work again, but only with 2 workarounds:

1. Add Compress=no to /etc/systemd/journald.conf
2. Set environment variable SYSTEMD_JOURNAL_KEYED_HASH=0 for systemd-journal service.

We used the following example code to reproduce the issue:

#!/usr/bin/env ruby

gem "systemd-journal", ">= 1.4.2"
require 'systemd/journal'

if ARGV.length == 0
  puts "usage: ./#{File.basename(__FILE__)} /var/log/journal/{machine-id}"
  exit(1)
end

begin
  j = Systemd::Journal.new(path: ARGV[0])

  j.each do |entry|
    puts entry

Inside the following docker container:

docker pull rubylang/ruby:2.6.5-bionic
docker run --rm --privileged -it -v /var/log/journal:/var/log/journal rubylang/ruby:2.6.5-bionic /bin/sh
apt update && apt install -y vim libsystemd0 bash
gem install systemd-journal -v 1.4.2
ruby ./test.rb /var/log/journal

I would have thought the newer libsystemd0 in bionic would have helped but it did not. Is there anything that can be done in this codebase to allow systemd 246 to work in the default configuration?

[ledbettj]

Hi, just to confirm:

• Outside the container you have systemd ~245~ 246 in flatcar linux
• Inside the container you have libsystemd0 which is tagged as 237-3ubuntu10.43

The result is that you are trying to read journal files created under 246 with a libsystemd that is version 237. If this is correct, you can find the problem by installing systemd and running journalctl -D /var/log/journal/{machine-id}:

Journal file [...] uses an unsupported feature, ignoring file.
Use SYSTEMD_LOG_LEVEL=debug journalctl --file=[...] to see the details.

I think this is the same issue as #71, which unfortunately I don't think is fixable beyond what you have done, which is to configure the new version to not use features the old version can't handle.

An old version of the library can't read a new version of the file, and the fact that the file is unreadable is not, or was not at the time I last checked, even exposed to us :(

[kh34]

That's correct. Sorry, I was under the impression that libsystemd was newer in that image. I'll try upgrading that. Thanks for the response.

[ledbettj]

Yes, unfortunately I think that's your best bet if you don't want to disable those features :frowning: It's a bummer that this error isn't exposed via the sd_journal_* API.