libdbus-sys: consider a Rust-native implementation

[lucab]

This is a feature request to eventually get rid of the C libdbus part and make this a native Rust crate.

Context and usecase for this is that I have some portable/containerized applications which need to talk over dbus. The goal would be for those applications to be static binaries only depending on rust/musl toolchain to build, and not depending on external libs to run. However this currently requires libdbus library at build- and run-time, which in turn depends on a longer list of other C libraries dependencies (see ldd libdbus.so).

[diwic]

I consider it regurlarly, i e, every time I get frustrated over DBus library's internals ;-)

Last time it was discussed was here: https://github.com/diwic/dbus-rs/issues/85#issuecomment-307561563

[lucab]

@diwic ah, good to know. Perhaps you may want to write down a short doc about what's the design you would expect for such a library, or perhaps stub down an interface, either here or in some experimental repo ad-hoc. I don't have enough dbus-internals knowledge to design/architect it myself, but perhaps I can contribute to implementation. Not sure if @albel727 is interested too.

[diwic]

@lucab You might want to look at https://crates.io/crates/dbus-bytestream

[Arnavion]

I started https://github.com/Arnavion/dbus-pure for my own projects. It has the D-Bus type system, and a raw Connection type with send(header, body) and recv() -> (header, body) that implements the binary protocol using serde.

Let me know if you want to use it.

[diwic]

@Arnavion cool!

As for serde, last time I checked, I could not use it because it couldn't handle empty arrays correctly - i e, when you serialize an empty array, there is no way to figure out what type that array should be. Is that not a problem with recent serde versions?

[Arnavion]

https://github.com/Arnavion/dbus-pure/blob/395d1d6332ad5883a4be9be0c3dc25d5b46cfd62/src/types/variant.rs#L7-L14

Enforcing that invariant is the caller's responsibility.

[Arnavion]

To clarify, when I say that dbus-pure uses serde, I don't mean that it exposes a Deserializer which the caller can use to deserialize message bodies as arbitrary structs. Similarly, the crate does not expose a Serializer that the caller can use to serialize arbitrary structs as message bodies.

The crate internally uses a serde::Deserializer / serde::Serializer, but the public Connection::send and Connection::recv API are in terms of the MessageHeader type for the message header, and the Variant type for the message body. To send an arbitrary struct as a message body, the user will have to first convert it to Variant (*). Similarly, when they receive a message, they'll have to convert the Variant into their arbitrary struct (**). So it's not a problem if the user has a Vec<T> that they want to serialize but the Vec is empty - they have to convert it to a Variant::Array anyway, and at that point they have to set what the element's signature is.

Eg https://github.com/Arnavion/dbus-pure/blob/395d1d6332ad5883a4be9be0c3dc25d5b46cfd62/examples/mpris_playback_status.rs#L55-L71

Of course, there can be a wrapper on top of raw Variant API where the user sees normal Rust code like:

trait MprisPlayer {
    fn playback_status(&mut self) -> String;
}

Then you can have a code generator macro that emits something like:

struct MprisPlayer<'a> {
    client: &'a Client,
    destination: String,
    object_path: String,
}

impl MprisPlayer<'_> {
    fn playback_status(&mut self) -> Result<String> {
        let (mut request_header, request_body) = /* Build a Variant::Tuple for the message body, like in that example */;
        let (response_header, response_body) = self.client.method_call(&mut request_header, &request_body)?;
        response_body.into_string() // Fallibly parse the response Variant as a Variant::String and return the inner String
    }
}

Similarly for a &[&str] parameter, the impl code would have the compile-time type information to know that it has to construct a Variant::Array { element_signature: Signature::String, elements: ... } regardless of whether the slice is empty at runtime or not.

But this kind of generated code is at a higher level of abstraction than Connection and Variant. What dbus_pure provides are the building blocks that such a macro would expand to.

---

(*) : Variant uses Cow for everything, so it can be usually be created using borrows of the original Rust type instead of copying.

(**) : I'll probably implement serde::Deserializer on Variant so that it's easier to parse into an arbitrary struct, just like serde_json::Value does.

[diwic]

@Arnavion Thanks for the explanation!

So I had some more look at your code. I'm open for optionally depending on dbus-pure, here are my current hesitations/thoughts:

• dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?
• There are functions in the C library that you currently don't implement (I think?), e g checking if a string is a valid object path or getting the machine UUID.
• My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

[Arnavion]

• dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?

I need a D-Bus client that doesn't depend on the C library for my own use. So I'm going to develop and maintain it for that purpose, including publishing to crates.io when I've ironed out the obvious bugs - serialization of empty arrays was broken until last night due to missing pre-element padding, connecting to the system bus only got added last night, env var parsing is still just slicing the string instead of properly parsing as a URL, etc.

Currently I'm using the raw Connection interface that I wrote about above, not this crate, and that will definitely hold as long as this crate has a hard dependency on the C library.

• There are functions in the C library that you currently don't implement (I think?),

Almost certainly. I have no idea what the C library offers. I want to make a clean-room implementation just from the spec so that it doesn't have to be GPL, so I've only looked at the spec, not the C library. So the minimum I will guarantee is that the type system and binary protocol are implemented correctly, and anything else mentioned in the spec.

e g checking if a string is a valid object path

I'm curious why a client needs that. That sounds like something a message bus implementation itself would need. Or is this just for convenience in the ObjectPath newtype's constructor?

Anyway, yes, some validation is missing that I plan to add. The more important one is for validity of signatures and containers. Currently it's possible to construct invalid Variants, like DictEntrys with non-simple keys like Array, and anything involving Tuple that isn't at the top-level like Array(Tuple(...)).

or getting the machine UUID.

That is obtained by calling the org.freedesktop.DBus.Peer.GetMachineId method, so it doesn't look like it needs to be a dedicated function in the library.

• My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

I haven't looked at this crate at all yet - after I saw it depended on the C library I stopped looking at it. I'll go over this crate's docs over the weekend.

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

That is likely to be impossible / more effort than I care to make, but I'll know for sure after the weekend.

I want to emphasize that my personal need is already satisfied by dbus-pure's level of abstraction, so making this crate use dbus-pure as a backend is not a priority for me. For example, from a quick glance the only thing that this crate offers which I could want is the Get trait for deserializing message bodies to Rust types, but as I wrote in the last comment I'm planning to solve that by impling serde::Deserializer on Variant instead.

[diwic]

• dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?

I need a D-Bus client that doesn't depend on the C library for my own use. So I'm going to develop and maintain it for that purpose, including publishing to crates.io when I've ironed out the obvious bugs - serialization of empty arrays was broken until last night due to missing pre-element padding, connecting to the system bus only got added last night, env var parsing is still just slicing the string instead of properly parsing as a URL, etc.

Currently I'm using the raw Connection interface that I wrote about above, not this crate, and that will definitely hold as long as this crate has a hard dependency on the C library.

• There are functions in the C library that you currently don't implement (I think?),

Almost certainly. I have no idea what the C library offers. I want to make a clean-room implementation just from the spec so that it doesn't have to be GPL, so I've only looked at the spec, not the C library. So the minimum I will guarantee is that the type system and binary protocol are implemented correctly, and anything else mentioned in the spec.

The C library is dual licensed under GPL and AFL, both allow you to study the source code.

e g checking if a string is a valid object path

I'm curious why a client needs that. That sounds like something a message bus implementation itself would need. Or is this just for convenience in the ObjectPath newtype's constructor?

The latter, mostly. It's a matter of being a well behaved client (or server) - I don't want the users of my dbus crate to be able to send invalid ObjectPaths on the bus.

Anyway, yes, some validation is missing that I plan to add. The more important one is for validity of signatures and containers. Currently it's possible to construct invalid Variants, like DictEntrys with non-simple keys like Array, and anything involving Tuple that isn't at the top-level like Array(Tuple(...)).

or getting the machine UUID.

That is obtained by calling the org.freedesktop.DBus.Peer.GetMachineId method, so it doesn't look like it needs to be a dedicated function in the library.

A well behaved server should implement org.freedesktop.DBus.Peer on all its object paths. That's how I understand the spec, at least. And my dbus library supports both clients and servers.

• My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

I haven't looked at this crate at all yet - after I saw it depended on the C library I stopped looking at it. I'll go over this crate's docs over the weekend.

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

That is likely to be impossible / more effort than I care to make, but I'll know for sure after the weekend.

I want to emphasize that my personal need is already satisfied by dbus-pure's level of abstraction, so making this crate use dbus-pure as a backend is not a priority for me. For example, from a quick glance the only thing that this crate offers which I could want is the Get trait for deserializing message bodies to Rust types, but as I wrote in the last comment I'm planning to solve that by impling serde::Deserializer on Variant instead.

Okay, thanks for clarifying. Maybe someone else will carry on the torch and try to write some glue between dbus-pure and the dbus crate, if neither us have the time or effort necessary to complete it right now.

[Arnavion]

The C library is dual licensed under GPL and AFL, both allow you to study the source code.

I'm aware. This is not acceptable reasoning for me. It's non-trivial to argue that writing code based on reading GPL'd code is not a derived work of said GPL'd code. A clean-room implementation is safe(r).

A well behaved server should implement org.freedesktop.DBus.Peer on all its object paths. That's how I understand the spec, at least.

Sure, and it can be implemented by the layer on top of dbus-pure, sourced from the bus's response to org.freedesktop.DBus.Peer.GetMachineId or from another place of the caller's choosing. I don't think it belongs at the dbus-pure layer.

And my dbus library supports both clients and servers.

To be clear, so does dbus-pure, since it does not different between them.

[mathstuf]

On January 3, 2020 4:45:59 PM EST, Arnav Singh [email protected] wrote:

A well behaved server should implement org.freedesktop.DBus.Peer on all its object paths. That's how I understand the spec, at least.

Sure, and it can be implemented by the layer on top of dbus-pure, sourced from the bus's response to org.freedesktop.DBus.Peer.GetMachineId or from another place of the caller's choosing. I don't think it belongs at the dbus-pure layer.

I was working on a dbus crate a few years ago (rust-bus). It's not the best Rust code, but I've learned a bunch since then and have ideas for it. One of the things I did for it was write the machine-id crate for it to use. It needs work on macOS and Windows, but I have a PR for Windows at least (needs CI for it).

[diwic]

GetMachineId is not a big deal, since it just reads /etc/machine-id anyway. And given that it seems possible to get something up and running in just a few weeks, maybe I would be better off writing something myself? That way I get something that's specifically tailored to the needs of the dbus crate. What do you think?

[mathstuf]

That assumes it's using /etc/machine-id (have the non-systemd distros started providing it?). I'd also really like to not have it be Linux-only.

[mathstuf]

Also, if you have two crates wanting to work with machine-id numbers and one has to be crafted, it'd be nice if all the internals would agree on the same one at least (even if other processes don't necessarily agree).

[diwic]

Oh, and suddenly they just keep popping up like mushrooms after a rainy day!

https://github.com/KillingSpark/rustbus https://gitlab.freedesktop.org/zeenix/zbus/ https://github.com/Arnavion/dbus-pure https://github.com/srwalter/dbus-bytestream

...and did I mention I started writing something myself yesterday...

[mathstuf]

dbus-bytestream is considerably older than the rest and is just a marshaller for the bytestream of the protocol itself; it doesn't implement any actual communication logic. I recommend it to any pure Rust impl of D-Bus as a place to start.

[KillingSpark]

Oh, and suddenly they just keep popping up like mushrooms after a rainy day!

https://github.com/KillingSpark/rustbus https://gitlab.freedesktop.org/zeenix/zbus/ https://github.com/Arnavion/dbus-pure https://github.com/srwalter/dbus-bytestream

I started a benchmarking repo for those here

[landhb]

It isn't as ideal as a pure-rust implementation, but I've started a draft PR to add a vendoring option, which will reduce the build time dependencies.

https://github.com/diwic/dbus-rs/pull/408