How should dynamic linking work in Rust?

[jsgf]

If I set crate-type=["dylib"] in Cargo.toml, I would expect it to not only compile the top-level crate as dynamic, but also all its transitive dependencies, and link with them dynamically (ie, with -C prefer-dynamic semantics).

Right now if you try this, it builds all the dependencies as static, and statically links them into the dynamic crate it builds. If this dynamic crate is subsequently used to build an executable with a diamond dependency graph (ie two of its dependencies have the same transitive dependency), rustc fails because of the crate that's duplicated by being linked into two other dynamic crates. This happens most often with libstd, but it can happen with any shared dependency. This suggests that the default for any dynamic crate should be "prefer dynamic" for its dependencies, since the current default is only useful in very niche cases.

If you specify -C prefer-dynamic as a build flag, then it will not include the dependent crates in the dynamic output - but it also won't build the dependencies at all.

Further, if you specify crate-type=["dylib", "rlib"], I would like it to build the crate twice as static and dynamic (this does work currently), and all its dependencies (this does not).

Related: #3408

[alexcrichton]

Note that the current behavior exists for backwards compatibility at this point. Long ago a cdylib didn't exist so the only way to create a dynamic library was through the dylib crate type, so Cargo must preserve the current behavior today. That diamond dependencies don't work is a bug (https://github.com/rust-lang/rust/issues/34909) and it should be fixable at any time really.

The use case for dynamic libraries in Rust has basically never been gamed out. As a result a bug like this is likely much broader in terms of "how should dynamic linking work in Rust" at all. I don't think that just building a bunch of dylibs will solve the real problem at hand, so this seems unlikely to be a quick bug fix in Cargo.

[zicklag]

@alexcrichton, you said in your other comment:

note that the crate type "cdylib" is most appropriate for creating a Rust DSO. If you're trying to use a dylib linking to other Rust dylibs it'll be a nonstop world of pain right now unfortunately, but cdylib should be quite smooth.

I'm sorry if this is the wrong spot to ask and I'll be happy to move this discussion somewhere else if necessary, but if you use cdylib for a crate, the only way to link to it from another Rust crate is to use extern "C" blocks and functions that must be defined manually, right? It would be a pretty big pain to have to manually bind your API to a C interface even if you were only intending on linking to it from another Rust crate.

The problem with dylibs in Rust, on the other hand, is that the Rust ABI is unstable and a dylib built with one version of Rust will be incompatible with a dylib built with another version of Rust.

What if Rust provided a way to automatically create direct "C" bindings for a Rust crate, not with the intention of having C programs link to it, but with the purpose of giving Rust a stable ABI over which to dynamically link other Rust crates.

For example, there could be:

• A new crate type that you could build crates as such as rdylib.
• A new way to import extern crates such as extern dynamic crate crate_name ( This could maybe be done away with if it could be automatically detected that the library is an rdylib )

If you build a crate as an rdylib it would be very similar to building a dylib except that it would automatically create extern "C" functions ( either literally, or just do the equivalent under the hood ) for the entire public Rust API for that crate. When including a crate using the extern dynamic crate syntax, it would essentially automatically create extern "C" blocks for the rust crate so that you could call its functions. You would probably have to build derive macros as rdylib's as well so that they could be linked into rustc while compiling the dependent crate.

As far as the developer is concerned, they don't have to manually create any bindings, and they can now safely dynamically link to other Rust libraries.

The motivation for this is the desire to be able to create a Rust plugin interface. I've attempted a tutorial which is based on using dylib's but I think that has problems in a larger context. I think I've experienced the problem where dependent crates are not built like @jsgf did. To solve that issue in the context of my idea you might be able to build all of the dependencies as rdylib's or maybe statically link them into the single generated rdylib ( if that doesn't cause issues with diamond dependencies ).

Anyway, I would appreciate your thoughts on the subject as I think that dynamic linking would be really helpful to me if there was a way to get it to work. If there is already a way to do this without changes to Cargo or Rust that I'm not aware of, that would be great. :slightly_smiling_face:

[zicklag]

Another thought is that you might be able to implement this without modifying rustc or Cargo by creating a macro that would create the extern functions automatically by reading a library's rlib and put them in a Rust file that could be compiled to create the shared library. All of which could probably be automated by a build script.

In a similar manner, you could create a macro for importing the crate that would expand to all of the extern blocks necessary to link to it.

That would be a good way to prove out the concept.

[alexcrichton]

For designing a system of official supported dynamic libraries in Rust I'd recommend the internals/users forum rather than an old issue on the Cargo issue tracker. It's likely to definitely need rustc changes and require an RFC eventually.

[clouds56]

How should I do with cargo if I really want to build every dependencies with dylib? I've tried to use RUSTFLAGS="-C prefer-dynamic" cargo build --release and cargo rustc --release -- -C prefer-dynamic but without luck.

[zicklag]

I don't think you can have it build every crate as a dylib. I think that will just dynamically link the standard library with -C prefer-dynamic. Also, you probably already know, but dylib is really not stable so to speak and creates libs that are specific to rustc version.

Here is some perspective that might help a little:

https://users.rust-lang.org/t/what-is-the-difference-between-dylib-and-cdylib/28847/3?u=zicklag

[clouds56]

@zicklag thanks for kindly reply. I do know that the dylib is not compatible between rustc versions. But I still want this feature since I'm creating a game. It took long time building even if I modified a little in the engine or other subsystems. The binary is now ~200MB and would grow as more features added. Almost no game is delivered as a single file, I could tolerated that lots of dll in the same folder, but it's hard to deliver update if the single bin file is such large.

Maybe now cargo doesn't support make every crate as dylib, could I specify RUSTFLAGS for some specific crates (of which is used in several crates that rustc complains)

[zicklag]

Ah, yes, those are pretty much the reasons that I tried to figure out how to use dylib. I create a tutorial based on my attempts to get that working and you're welcome to check it out, but I pretty much decided it probably wasn't going to work well enough. I'm not sure if it will end up helping you or not.

I know that the bevy engine had dynamic plugin loading at one point, but I don't know how it worked.

Also take a look a ABI Stable Crates which might be your best bet.

[zyansheep]

I have an idea for how dynamic linking should work but I have no idea if it is even feasible because I have no idea how dynamic linking works. My idea is as follows: each dependency of a program is compiled separately and named with the library name, version, and sha256 hash. i.e. rand-0.8.0-5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03 When the main program is compiled, it would have in the binary a list of sha256 hashes corresponding to different dependencies. These files would be able to be served by crates.io or some other distribution system, or downloaded with the executable. When the program runs, it looks in some predefined place for dependencies (maybe looking for paths from environment variable?) and once it finds them all, it runs. This system would also allow for lots of flexibility when managing and selecting dependencies compared to C where your program might error if you load the wrong version.

The ultimate version of this might look like downloading a super small executable which when run looks for specific dependencies on the system and downloads them if they don't exist.

[cdecompilador]

It would be great to have at least for debug builds an option to compile all the dependencies as dynamic libs, when you have a lot of dependencies the linking time is horrible for the minimal change

[epage]

Going to snapshot some notes that I have on this topic:

Opaque dependencies

Cargo's dependencies todays are transparent. You see and control the entire dependency tree and how they are built.

This causes limitations when evaluating build-time performance improvements including

• Caching: differences in dependency versions cause a unique instance of everything built above it
• Pre-built binaries: similar to the above, there are too many combinations to account for. You could fake it and say "pick one and deal with it" but that runs counter to the design of these dependencies
• MIR-only or cross-crate lazy building: Every one of your dependencies will be too unique to reuse between binaries within your workspace or cached between projects. Similarly, Every one of the binaries within your workspace will have to do codegen for your dependencies. If we could have a hybrid model where small, lower down dependencies are built lazily but certain heavy-weight dependencies get built and code-genned once, it might strike the right balance.
• Optimizing the build for dependencies you are unlikely to debug into.

In reflecting on this, this feels similar to the divide between C++ headers-only libraries and dynamic libraries. As a consumer, you don't care how the sausage is made for a dynamic library. You don't care what version of an inner dependency is used. You can swap in or out a debug build, a release build, change various compiler flags, etc and it doesn't affect you.

What if we had an opaque dependency with:

• Its own Cargo.lock
• Its own RUSTFLAGS
• Its own profile
• Can more easily be switched between rlib and rdylib (reduce link times)

Examples of what this might apply to include:

• std
• bevy
• gitoxide
• tauri
• iced
• axum

std? Yes, part of the inspiration for this is the build-std effort where you can choose to rebuild std.

• std and build-std have their own Cargo.lock
• At least std has its own RUSTFLAGS, profiles, etc

Open question:

• Should cargo clean skip cleaning opaque dependencies? Or should we leave that to #5931.

Opportunistic reuse

If a dependency happens to be shareable between an opaque dependency and the caller, should it?

• With the same -Cmetadata, you could accidentally be using a copy of your dependency where you should be using a re-export from the opaque dependency which could break you on upgrade
• Depending on how we handle the MIR-only part of this, there might be fewer opportunities for reuse
• Cargo's existing work for reusing between host/target is complicated and doesn't get as much reuse as one would like
  • However, depending on the caching setup, we might get this "for free" if the -Cmetadata can be the same

Open question:

• Should we include the identity of the opaque dependency in all -Cmetadata hashes for members of it to avoid accidental API breakages?

Library packs

Something that std calls attention to that could be easily overlooked in this initial plan is you can choose to depend on core, alloc, std, proc_macro, etc which all come from sysroot.

TODO how are these linked? Are they separate rlibs but built as a single set? If so, then we may need "library packs" to allow more than one top-level package. This might also work well for loosening of the orphan rules.

Singletons

How do we deal with singletons?

• log
• colorchoice
• rayon-core

Prior art

What lessons can we learn from C++ application development?

• On how to configure this and tie it together
• In what situatiouns may someone want a static lib vs a dynamic lib?
• If profiles are decoupled, how we decide what provide to build within an opaque dependency?
• How do we know when to switch profiles of an opaque dependency?