sui icon indicating copy to clipboard operation
sui copied to clipboard

Impossible to package for NixOS due to duplicate dependencies in Cargo.lock

Open shackra opened this issue 5 months ago • 12 comments

In release 1.49.0 there's a duplication with sui-graphql-client-build

https://github.com/MystenLabs/sui/blob/62b9f371e37e618b7048417e2d232d6cd678c967/Cargo.lock#L14121

In the main branch there is also a duplication, but with tokio-macros

https://github.com/MystenLabs/sui/blob/c746e35a09584b63b3ed92ce81f45a9047f9c1e1/Cargo.lock#L17226

This kind of problems in Cargo.lock makes it difficult to package Sui in NixOS.

I suggest cleaning the Cargo.lock with, maybe, something like https://crates.io/crates/cargo-lock and introducing checks in the CI/CD pipeline to prevent this sort of things happening in the future.

shackra avatar Jun 11 '25 23:06 shackra

Thank you for opening this issue, a team member will review it shortly. Until then, please do not interact with any users that claim to be from Sui support and do not click on any links!

github-actions[bot] avatar Jun 11 '25 23:06 github-actions[bot]

Some corrections:

  • cargo-lock does nothing like deduplication
  • the Sui version with the duplicated sui-graphql-client-build is 1.49.2 and the reason is crates/sui-replay-2 has a different revision of sui-graphql-client-build set.
  • I don't know Rust or Cargo, but the easiest way to figure out dependency duplication is running cargo vendor and see it fail.

shackra avatar Jun 11 '25 23:06 shackra

that code has been removed and later released and main do not have that any longer. Sorry for the issue. If it is critical for you to package Sui you can remove sui-replay-2 from your build until you move to a newer version. Would that work?

dariorussi avatar Jun 12 '25 12:06 dariorussi

Not knowing exactly what you're trying to do, would suiup be of any help here? https://github.com/mystenLabs/suiup

stefan-mysten avatar Jun 12 '25 18:06 stefan-mysten

that code has been removed and later released and main do not have that any longer. Sorry for the issue. If it is critical for you to package Sui you can remove sui-replay-2 from your build until you move to a newer version. Would that work?

I could give it a try by patching the Cargo.lock file, sure.

Not knowing exactly what you're trying to do, would suiup be of any help here? mystenLabs/suiup

Basically, I'm trying to package Sui from source. In NixOS, to package a Rust application, how it is done is that the system fetches the dependencies of the project as they appear in the Cargo.lock file, sort of like when cargo vendor is used and the dependencies are recorded in Git along with the source code that use them. When the same version of the same dependency appears twice in Cargo.lock, that phase of the packaging fails (see https://github.com/NixOS/nixpkgs/issues/359340)

shackra avatar Jun 12 '25 22:06 shackra

Having commit c7ec9546978b3c52b0c57bbdb9693f5068dd3383 as base, I dived into patching all Cargo.lock files as needed and ended with this (see attach). However, during compilation, h2 failed to compile:

   Compiling rand_xorshift v0.3.0
   Compiling atomic-waker v1.1.2
   Compiling zerocopy v0.7.35
   Compiling block-padding v0.2.1
   Compiling h2 v0.4.5
   Compiling block-buffer v0.9.0
   Compiling spki v0.7.3
   Compiling keccak v0.1.3
error[E0599]: the method `poll_next` exists for struct `Pin<&mut FramedRead<T, LengthDelimitedCodec>>`,
   --> /build/sui-unstable-2025-06-16-vendor/h2-0.4.5/src/codec/framed_read.rs:384:64
    |
384 |               let bytes = match ready!(Pin::new(&mut self.inner).poll_next(cx)) {
    |                                                                  ^^^^^^^^^ method cannot be called
    |
   ::: /build/sui-unstable-2025-06-16-vendor/tokio-util-0.7.13/src/codec/framed_read.rs:14:1
    |
14  | / pin_project! {
15  | |     /// A [`Stream`] of messages decoded from an [`AsyncRead`].
16  | |     ///
17  | |     /// For examples of how to use `FramedRead` with a codec, see the
...   |
33  | | }
    | |_- doesn't satisfy `_: Stream`
    |
    = note: the following trait bounds were not satisfied:
            `T: real_tokio::io::async_read::AsyncRead`
            which is required by `tokio_util::codec::FramedRead<T, LengthDelimitedCodec>: futures_core::

error[E0277]: the trait bound `T: real_tokio::io::async_write::AsyncWrite` is not satisfied
   --> /build/sui-unstable-2025-06-16-vendor/h2-0.4.5/src/codec/framed_write.rs:140:47
    |
140 |                         ready!(poll_write_buf(Pin::new(&mut self.inner), cx, &mut buf))?
    |                                -------------- ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `real_tokio::io:
    |                                |
    |                                required by a bound introduced by this call
    |
note: required by a bound in `poll_write_buf`
   --> /build/sui-unstable-2025-06-16-vendor/tokio-util-0.7.13/src/util/poll_buf.rs:122:26
    |
122 | pub fn poll_write_buf<T: AsyncWrite + ?Sized, B: Buf>(
    |                          ^^^^^^^^^^ required by this bound in `poll_write_buf`
help: consider further restricting type parameter `T` with trait `AsyncWrite`
    |
81  |     T: AsyncWrite + Unpin + real_tokio::io::async_write::AsyncWrite,
    |                           +++++++++++++++++++++++++++++++++++++++++

error[E0277]: the trait bound `T: real_tokio::io::async_write::AsyncWrite` is not satisfied
   --> /build/sui-unstable-2025-06-16-vendor/h2-0.4.5/src/codec/framed_write.rs:145:29
    |
144 |                         ready!(poll_write_buf(
    |                                -------------- required by a bound introduced by this call
145 |                             Pin::new(&mut self.inner),
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `real_tokio::io::async_write::Asyn
    |
note: required by a bound in `poll_write_buf`
   --> /build/sui-unstable-2025-06-16-vendor/tokio-util-0.7.13/src/util/poll_buf.rs:122:26
    |
122 | pub fn poll_write_buf<T: AsyncWrite + ?Sized, B: Buf>(
    |                          ^^^^^^^^^^ required by this bound in `poll_write_buf`
help: consider further restricting type parameter `T` with trait `AsyncWrite`
    |
81  |     T: AsyncWrite + Unpin + real_tokio::io::async_write::AsyncWrite,
    |                           +++++++++++++++++++++++++++++++++++++++++

error[E0277]: the trait bound `framed_write::FramedWrite<T, _>: real_tokio::io::async_read::AsyncRead` i
   --> /build/sui-unstable-2025-06-16-vendor/h2-0.4.5/src/codec/mod.rs:50:23
    |
50  |             .new_read(framed_write);
    |              -------- ^^^^^^^^^^^^ the trait `real_tokio::io::async_read::AsyncRead` is not implem
    |              |
    |              required by a bound introduced by this call
    |
    = help: the following other types implement trait `real_tokio::io::async_read::AsyncRead`:
              &[u8]
              &mut T
              Box<T>
              Compat<T>
              Either<L, R>
              InspectReader<R, F>
              InspectWriter<W, F>
              Pin<P>
            and 3 others
note: required by a bound in `tokio_util::codec::length_delimited::Builder::new_read`
   --> /build/sui-unstable-2025-06-16-vendor/tokio-util-0.7.13/src/codec/length_delimited.rs:993:12
    |
991 |     pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
    |            -------- required by a bound in this associated function
992 |     where
993 |         T: AsyncRead,
    |            ^^^^^^^^^ required by this bound in `Builder::new_read`

   Compiling pkcs8 v0.10.2
   Compiling group v0.13.0
   Compiling blst v0.3.11
   Compiling sha2 v0.9.9
   Compiling prost-derive v0.13.3
Some errors have detailed explanations: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `h2` (lib) due to 4 previous errors
warning: build failed, waiting for other jobs to finish...
error: Cannot build '/nix/store/wy1i1m4hs3hg94rb4giywggspzb6bizw-sui-unstable-2025-06-16.drv'.
       Reason: builder failed with exit code 101.
       Output paths:
         /nix/store/ba7r97ywrw3prr9pzqk3xqncg2g9j11n-sui-unstable-2025-06-16
       Last 25 log lines:
       >               Box<T>
       >               Compat<T>
       >               Either<L, R>
       >               InspectReader<R, F>
       >               InspectWriter<W, F>
       >               Pin<P>
       >             and 3 others
       > note: required by a bound in `tokio_util::codec::length_delimited::Builder::new_read`
       >    --> /build/sui-unstable-2025-06-16-vendor/tokio-util-0.7.13/src/codec/length_delimited.rs:993:12
       >     |
       > 991 |     pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
       >     |            -------- required by a bound in this associated function
       > 992 |     where
       > 993 |         T: AsyncRead,
       >     |            ^^^^^^^^^ required by this bound in `Builder::new_read`
       >
       >    Compiling pkcs8 v0.10.2
       >    Compiling group v0.13.0
       >    Compiling blst v0.3.11
       >    Compiling sha2 v0.9.9
       >    Compiling prost-derive v0.13.3
       > Some errors have detailed explanations: E0277, E0599.
       > For more information about an error, try `rustc --explain E0277`.
       > error: could not compile `h2` (lib) due to 4 previous errors
       > warning: build failed, waiting for other jobs to finish...
       For full logs, run:
         nix log /nix/store/wy1i1m4hs3hg94rb4giywggspzb6bizw-sui-unstable-2025-06-16.drv

I'm not sure where to go from here because I'm not versed in Rust or if this was introduced by the patching of Cargo.lock files. If anyone can orient me, I appreciate it a lot.

shackra avatar Jun 17 '25 03:06 shackra

Is the goal to support release 1.49, or all releases? If not, then latest release v1.50.1 is available for testnet and mainnet, so you shouldn't have any issues building that. This is the sha for this release: 197ebfbc1ff03d83f908e23281e7269b65cf350d.

Now, I don't know NixOS or how its pkg mgmt system works. Why does the lockfile need to be patched when building from https://github.com/MystenLabs/sui/commit/c7ec9546978b3c52b0c57bbdb9693f5068dd3383?

stefan-mysten avatar Jun 17 '25 19:06 stefan-mysten

Now, I don't know NixOS or how its pkg mgmt system works. Why does the lockfile need to be patched when building from https://github.com/MystenLabs/sui/commit/c7ec9546978b3c52b0c57bbdb9693f5068dd3383?

because otherwise it will fail copying the dependencies, just like if you ran cargo vendor on that commit. See https://github.com/NixOS/nixpkgs/issues/359340

Is the goal to support release 1.49, or all releases?

only the latest, but I can't keep up with the updates :)

shackra avatar Jun 17 '25 20:06 shackra

Hmm, no, tokio-macros is a duplicate dependency still:

Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0
Copying to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0
Traceback (most recent call last):
  File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 349, in <module>
    main()
  File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 345, in main
    subcommand_func()
  File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 337, in <lambda>
    "create-vendor": lambda: create_vendor(vendor_staging_dir=Path(sys.argv[2]), out_dir=Path(sys.argv[3]))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 294, in create_vendor
    copy_and_patch_git_crate_subtree(git_tree, pkg["name"], crate_out_dir)
  File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 233, in copy_and_patch_git_crate_subtree
    shutil.copytree(crate_tree, crate_out_dir, ignore=ignore_func)
  File "/nix/store/8w718rm43x7z73xhw9d6vh8s4snrq67h-python3-3.12.10/lib/python3.12/shutil.py", line 600, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/8w718rm43x7z73xhw9d6vh8s4snrq67h-python3-3.12.10/lib/python3.12/shutil.py", line 498, in _copytree
    os.makedirs(dst, exist_ok=dirs_exist_ok)
  File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0'
error: Cannot build '/nix/store/fw99ra0yic35hh6753hhx9rsb1jxx579-sui-1.50.1-vendor.drv'.
       Reason: builder failed with exit code 1.
       Output paths:
         /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor
       Last 25 log lines:
       > Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tinyvec_macros-0.1.0
       > Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-1.44.2
       > Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-cron-scheduler-0.10.0
       > Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-io-timeout-1.2.0
       > Unpacking to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0
       > Copying to /nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0
       > Traceback (most recent call last):
       >   File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 349, in <module>
       >     main()
       >   File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 345, in main
       >     subcommand_func()
       >   File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 337, in <lambda>
       >     "create-vendor": lambda: create_vendor(vendor_staging_dir=Path(sys.argv[2]), out_dir=Path(sys.argv[3]))
       >                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       >   File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 294, in create_vendor
       >     copy_and_patch_git_crate_subtree(git_tree, pkg["name"], crate_out_dir)
       >   File "/nix/store/7f3a269xl53xbywqpwwihipjv3iagqya-fetch-cargo-vendor-util/bin/fetch-cargo-vendor-util", line 233, in copy_and_patch_git_crate_subtree
       >     shutil.copytree(crate_tree, crate_out_dir, ignore=ignore_func)
       >   File "/nix/store/8w718rm43x7z73xhw9d6vh8s4snrq67h-python3-3.12.10/lib/python3.12/shutil.py", line 600, in copytree
       >     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
       >            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       >   File "/nix/store/8w718rm43x7z73xhw9d6vh8s4snrq67h-python3-3.12.10/lib/python3.12/shutil.py", line 498, in _copytree
       >     os.makedirs(dst, exist_ok=dirs_exist_ok)
       >   File "<frozen os>", line 225, in makedirs
       > FileExistsError: [Errno 17] File exists: '/nix/store/zm1wm44pyq2q9gj4j5kmvmii7a3gzk2h-sui-1.50.1-vendor/tokio-macros-2.5.0'
       For full logs, run:
         nix log /nix/store/fw99ra0yic35hh6753hhx9rsb1jxx579-sui-1.50.1-vendor.drv
error: Cannot build '/nix/store/2hs187mi3i2dqfnamwra0x9rypd35xpc-sui-1.50.1.drv'.
       Reason: 1 dependency failed.
       Output paths:
         /nix/store/i4ryfa4fpx1khy4h3b5hmgd7srksxkcl-sui-1.50.1

shackra avatar Jun 17 '25 21:06 shackra

@shackra thanks for that. I have a Mac M1 on which I cannot reproduce the error after a cargo clean Here's what I ran:

➜  sui git:(197ebfbc1f) cargo build --release --bin sui --features tracing
....
   Finished `release` profile [optimized + debuginfo] target(s) in 10m 34s

So I am starting to think that there might be some library dependency issue going on?

On another machine with archOS I am getting build errors around librocksdb, so maybe I have a missing / dated dependency there. I will try to get back to you if I can get it to compile on my archOS machine, that's probably closer to your NixOS. Note that I checked out this sha: 197ebfbc1f.

stefan-mysten avatar Jun 18 '25 00:06 stefan-mysten

@stefan-mysten run cargo vendor and let me know

shackra avatar Jun 18 '25 01:06 shackra

So I am starting to think that there might be some library dependency issue going on?

it is, but what you think, you think is some sort of compatibility issue, but is not. Just run cargo vendor and you will understand.

shackra avatar Jun 18 '25 02:06 shackra

@shackra - let me also ask a few colleagues internally, but here's what might be the problem. We have a fork of tokio for a specific thing. It errors because it does not know which crate to use because there are two sources.

It might be possible to have a workaround by adding this to the Cargo.toml, at the end, in the [patch.crates-io] section.

tokio-macros = { git = "https://github.com/MystenLabs/tokio-msim-fork.git", package = "tokio-macros", rev = "7329bff6ee996d8df6cf810a9c2e59631ad5a2fb" }
tokio-util = { git = "https://github.com/MystenLabs/tokio-msim-fork.git", package = "tokio-util", rev = "7329bff6ee996d8df6cf810a9c2e59631ad5a2fb" }

After this, I was able to run cargo vendor

To use vendored sources, add this to your .cargo/config.toml for this project:

[source.crates-io]
replace-with = "vendored-sources"

[source."git+https://github.com/MystenLabs/fastcrypto?rev=69d496c71fb37e3d22fe85e5bbfd4256d61422b9"]
git = "https://github.com/MystenLabs/fastcrypto"
rev = "69d496c71fb37e3d22fe85e5bbfd4256d61422b9"
replace-with = "vendored-sources"

[source."git+https://github.com/MystenLabs/mysten-sim.git?rev=cad62679fd180a48c665f842cb80045f8fcfdee9"]
git = "https://github.com/MystenLabs/mysten-sim.git"
rev = "cad62679fd180a48c665f842cb80045f8fcfdee9"
replace-with = "vendored-sources"

[source."git+https://github.com/MystenLabs/sui-rust-sdk.git?rev=b64827452de2e3f9e62ee8f3cb456a93c2b66b53"]
git = "https://github.com/MystenLabs/sui-rust-sdk.git"
rev = "b64827452de2e3f9e62ee8f3cb456a93c2b66b53"
replace-with = "vendored-sources"

[source."git+https://github.com/MystenLabs/tokio-msim-fork.git?rev=7329bff6ee996d8df6cf810a9c2e59631ad5a2fb"]
git = "https://github.com/MystenLabs/tokio-msim-fork.git"
rev = "7329bff6ee996d8df6cf810a9c2e59631ad5a2fb"
replace-with = "vendored-sources"

[source."git+https://github.com/amnn/async-graphql?branch=v7.0.1-react-18"]
git = "https://github.com/amnn/async-graphql"
branch = "v7.0.1-react-18"
replace-with = "vendored-sources"

[source."git+https://github.com/andll/tidehunter.git?rev=dd686f055375aa8fa2145618301bdfd5170a2a6b"]
git = "https://github.com/andll/tidehunter.git"
rev = "dd686f055375aa8fa2145618301bdfd5170a2a6b"
replace-with = "vendored-sources"

[source."git+https://github.com/asonnino/prometheus-parser.git?rev=75334db"]
git = "https://github.com/asonnino/prometheus-parser.git"
rev = "75334db"
replace-with = "vendored-sources"

[source."git+https://github.com/bmwill/axum-server.git?rev=f44323e271afdd1365fd0c8b0a4c0bbdf4956cb7"]
git = "https://github.com/bmwill/axum-server.git"
rev = "f44323e271afdd1365fd0c8b0a4c0bbdf4956cb7"
replace-with = "vendored-sources"

[source."git+https://github.com/mystenlabs/anemo.git?rev=9c52c3c7946532163a79129db15180cdb984bab4"]
git = "https://github.com/mystenlabs/anemo.git"
rev = "9c52c3c7946532163a79129db15180cdb984bab4"
replace-with = "vendored-sources"

[source."git+https://github.com/mystenmark/async-task?rev=4e45b26e11126b191701b9b2ce5e2346b8d7682f"]
git = "https://github.com/mystenmark/async-task"
rev = "4e45b26e11126b191701b9b2ce5e2346b8d7682f"
replace-with = "vendored-sources"

[source."git+https://github.com/nextest-rs/datatest-stable.git?rev=72db7f6d1bbe36a5407e96b9488a581f763e106f"]
git = "https://github.com/nextest-rs/datatest-stable.git"
rev = "72db7f6d1bbe36a5407e96b9488a581f763e106f"
replace-with = "vendored-sources"

[source."git+https://github.com/nextest-rs/nexlint.git?rev=7ce56bd591242a57660ed05f14ca2483c37d895b"]
git = "https://github.com/nextest-rs/nexlint.git"
rev = "7ce56bd591242a57660ed05f14ca2483c37d895b"
replace-with = "vendored-sources"

[source."git+https://github.com/zhiburt/tabled/?rev=e449317a1c02eb6b29e409ad6617e5d9eb7b3bd4"]
git = "https://github.com/zhiburt/tabled/"
rev = "e449317a1c02eb6b29e409ad6617e5d9eb7b3bd4"
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "vendor"

stefan-mysten avatar Jun 23 '25 16:06 stefan-mysten

thanks, I'll give it a try in my end. Ideally, cryptocurrency projects should have its dependencies vendored

shackra avatar Jun 23 '25 18:06 shackra

yep, that fixed it and the nix builder was able to start compiling. However, I got this error during compilation, any idea what it is and how to fix it?

   Compiling h2 v0.4.5
   Compiling group v0.13.0
   Compiling sha2 v0.9.9
error[E0599]: the method `poll_next` exists for struct `Pin<&mut FramedRead<T, LengthDelimitedCodec>>`, but its trait bounds were not satisfied
   --> /build/sui-1.50.1-unstable-vendor/h2-0.4.5/src/codec/framed_read.rs:384:64
    |
384 |               let bytes = match ready!(Pin::new(&mut self.inner).poll_next(cx)) {
    |                                                                  ^^^^^^^^^ method cannot be called on `Pin<&mut FramedRead<T, LengthDelimitedCodec>>` due to unsatisfied trait bounds
    |
   ::: /build/sui-1.50.1-unstable-vendor/tokio-util-0.7.13/src/codec/framed_read.rs:14:1
    |
14  | / pin_project! {
15  | |     /// A [`Stream`] of messages decoded from an [`AsyncRead`].
16  | |     ///
17  | |     /// For examples of how to use `FramedRead` with a codec, see the
...   |
33  | | }
    | |_- doesn't satisfy `_: Stream`
    |
    = note: the following trait bounds were not satisfied:
            `T: real_tokio::io::async_read::AsyncRead`
            which is required by `tokio_util::codec::FramedRead<T, LengthDelimitedCodec>: futures_core::Stream`

error[E0277]: the trait bound `T: real_tokio::io::async_write::AsyncWrite` is not satisfied
   --> /build/sui-1.50.1-unstable-vendor/h2-0.4.5/src/codec/framed_write.rs:140:47
    |
140 |                         ready!(poll_write_buf(Pin::new(&mut self.inner), cx, &mut buf))?
    |                                -------------- ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `real_tokio::io::async_write::AsyncWrite` is not implemented for `T`
    |                                |
    |                                required by a bound introduced by this call
    |
note: required by a bound in `poll_write_buf`
   --> /build/sui-1.50.1-unstable-vendor/tokio-util-0.7.13/src/util/poll_buf.rs:122:26
    |
122 | pub fn poll_write_buf<T: AsyncWrite + ?Sized, B: Buf>(
    |                          ^^^^^^^^^^ required by this bound in `poll_write_buf`
help: consider further restricting type parameter `T` with trait `AsyncWrite`
    |
81  |     T: AsyncWrite + Unpin + real_tokio::io::async_write::AsyncWrite,
    |                           +++++++++++++++++++++++++++++++++++++++++

error[E0277]: the trait bound `T: real_tokio::io::async_write::AsyncWrite` is not satisfied
   --> /build/sui-1.50.1-unstable-vendor/h2-0.4.5/src/codec/framed_write.rs:145:29
    |
144 |                         ready!(poll_write_buf(
    |                                -------------- required by a bound introduced by this call
145 |                             Pin::new(&mut self.inner),
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `real_tokio::io::async_write::AsyncWrite` is not implemented for `T`
    |
note: required by a bound in `poll_write_buf`
   --> /build/sui-1.50.1-unstable-vendor/tokio-util-0.7.13/src/util/poll_buf.rs:122:26
    |
122 | pub fn poll_write_buf<T: AsyncWrite + ?Sized, B: Buf>(
    |                          ^^^^^^^^^^ required by this bound in `poll_write_buf`
help: consider further restricting type parameter `T` with trait `AsyncWrite`
    |
81  |     T: AsyncWrite + Unpin + real_tokio::io::async_write::AsyncWrite,
    |                           +++++++++++++++++++++++++++++++++++++++++

error[E0277]: the trait bound `framed_write::FramedWrite<T, _>: real_tokio::io::async_read::AsyncRead` is not satisfied
   --> /build/sui-1.50.1-unstable-vendor/h2-0.4.5/src/codec/mod.rs:50:23
    |
50  |             .new_read(framed_write);
    |              -------- ^^^^^^^^^^^^ the trait `real_tokio::io::async_read::AsyncRead` is not implemented for `framed_write::FramedWrite<T, _>`
    |              |
    |              required by a bound introduced by this call
    |
    = help: the following other types implement trait `real_tokio::io::async_read::AsyncRead`:
              &[u8]
              &mut T
              Box<T>
              Compat<T>
              Either<L, R>
              InspectReader<R, F>
              InspectWriter<W, F>
              Pin<P>
            and 3 others
note: required by a bound in `tokio_util::codec::length_delimited::Builder::new_read`
   --> /build/sui-1.50.1-unstable-vendor/tokio-util-0.7.13/src/codec/length_delimited.rs:993:12
    |
991 |     pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
    |            -------- required by a bound in this associated function
992 |     where
993 |         T: AsyncRead,
    |            ^^^^^^^^^ required by this bound in `Builder::new_read`

   Compiling spki v0.7.3
   Compiling bitcoin-private v0.1.0
   Compiling pkcs8 v0.10.2
   Compiling prost-derive v0.13.3
   Compiling serde_urlencoded v0.7.1
Some errors have detailed explanations: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `h2` (lib) due to 4 previous errors
warning: build failed, waiting for other jobs to finish...
error: Cannot build '/nix/store/fzynq8jvrpvczmzkc7vy14vkcvdqxxyw-sui-1.50.1-unstable.drv'.
       Reason: builder failed with exit code 101.
       Output paths:
         /nix/store/g3n0ghsqi561wmmxn38m0xzmk5n00wfs-sui-1.50.1-unstable
       Last 25 log lines:
       >               Box<T>
       >               Compat<T>
       >               Either<L, R>
       >               InspectReader<R, F>
       >               InspectWriter<W, F>
       >               Pin<P>
       >             and 3 others
       > note: required by a bound in `tokio_util::codec::length_delimited::Builder::new_read`
       >    --> /build/sui-1.50.1-unstable-vendor/tokio-util-0.7.13/src/codec/length_delimited.rs:993:12
       >     |
       > 991 |     pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
       >     |            -------- required by a bound in this associated function
       > 992 |     where
       > 993 |         T: AsyncRead,
       >     |            ^^^^^^^^^ required by this bound in `Builder::new_read`
       >
       >    Compiling spki v0.7.3
       >    Compiling bitcoin-private v0.1.0
       >    Compiling pkcs8 v0.10.2
       >    Compiling prost-derive v0.13.3
       >    Compiling serde_urlencoded v0.7.1
       > Some errors have detailed explanations: E0277, E0599.
       > For more information about an error, try `rustc --explain E0277`.
       > error: could not compile `h2` (lib) due to 4 previous errors
       > warning: build failed, waiting for other jobs to finish...
       For full logs, run:
         nix log /nix/store/fzynq8jvrpvczmzkc7vy14vkcvdqxxyw-sui-1.50.1-unstable.drv

shackra avatar Jun 23 '25 20:06 shackra

Well, just a thought based on my intuition. There's the h2 crate that requires to use tokio util, maybe a specific version. Now that we forced it to use the one from the other crate, it might crash.

Try to set the tokio-macros and tokio-util to use the https://github.com/tokio-rs/tokio, with revision f4d9b3061c9eb932d98c2b1c5afb75c62e56638a.

However, there's no guarantees that this does not break anything.

I assume what you're after is deterministic builds, which are not super straightforward to do -- and I have no experience with them, unfortunately.

stefan-mysten avatar Jun 23 '25 21:06 stefan-mysten

Try to set the tokio-macros and tokio-util to use the https://github.com/tokio-rs/tokio, with revision f4d9b3061c9eb932d98c2b1c5afb75c62e56638a.

However, there's no guarantees that this does not break anything.

Thanks, I'll give it a try and see if it compiles.

I assume what you're after is deterministic builds, which are not super straightforward to do -- and I have no experience with them, unfortunately.

don't worry, I can take care of the deterministic part, all I ask from you/upstream is that you make sure sui can be built with vendor dependencies.

shackra avatar Jun 23 '25 21:06 shackra

cargo vendor couldn't find that revision, and it actually does not exist in tokio-rs/tokio https://github.com/search?q=repo%3Atokio-rs%2Ftokio%20f4d9b3061c9eb932d98c2b1c5afb75c62e56638a&type=code 🤔

shackra avatar Jun 23 '25 21:06 shackra

I used this version to build locally: eecfb1ec10eefaea72ea8f95a25c72a76b8df9a9. Try that.

stefan-mysten avatar Jun 23 '25 21:06 stefan-mysten

I used this version to build locally: eecfb1ec10eefaea72ea8f95a25c72a76b8df9a9. Try that.

I'll go back to this, work got in the way and I haven't had much time for anything but sleep

shackra avatar Jul 07 '25 04:07 shackra

I came back to this and... well, yeah, I can't successfully patch Sui to remove the duplicated dependency. I'll try to package this some other way to avoid using rustPlatform.buildRustPackage

shackra avatar Aug 12 '25 10:08 shackra

Nvm, during build phase there is no internet connection, so trying to run cargo build will fail:

error: Cannot build '/nix/store/a2n48bvy6fys84gvbhf4wrrmkk7vd3y8-sui-unstable-1.53.2.drv'.
       Reason: builder failed with exit code 101.
       Output paths:
         /nix/store/rng3xdbmly5kx3bjsgqlxhv5bwkv83dv-sui-unstable-1.53.2
       Last 25 log lines:
       > Running phase: configurePhase
       > no configure script, doing nothing
       > Running phase: buildPhase
       >     Updating git repository `https://github.com/amnn/async-graphql`
       > warning: spurious network error (3 tries remaining): failed to resolve address for github.com: Temporary failure in name resolution; class=Net (12)
       > warning: spurious network error (2 tries remaining): failed to resolve address for github.com: Temporary failure in name resolution; class=Net (12)
       > warning: spurious network error (1 tries remaining): failed to resolve address for github.com: Temporary failure in name resolution; class=Net (12)
       > error: failed to load source for dependency `async-graphql`
       >
       > Caused by:
       >   Unable to update https://github.com/amnn/async-graphql?branch=v7.0.1-react-18-graphiql-4#accb2362
       >
       > Caused by:
       >   failed to clone into: /build/source/.cargo/git/db/async-graphql-7336e61dcafca7ed
       >
       > Caused by:
       >   revision accb23623380994ae8a02218858e23eb6ec66187 not found
       >
       > Caused by:
       >   network failure seems to have happened
       >   if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
       >   https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
       >
       > Caused by:
       >   failed to resolve address for github.com: Temporary failure in name resolution; class=Net (12)

maybe I should try package suiup.

I much prefer https://github.com/MystenLabs/tokio-msim-fork stop being a dependency in the project tho

shackra avatar Aug 12 '25 10:08 shackra

if I'm not misreading suiup source code, it just downloads binaries from this repository and installs them as-is, correct? if so, that won't work for NixOS :(

shackra avatar Aug 12 '25 10:08 shackra

@shackra sorry to hear. Unfortunately, tokio-msim is a hard dependency and cannot be removed.

suiup is a tool to manage versions of different CLIs in the Sui ecosystem; in general, it works with releases and does not build from scratch unless requested, so I don't think it can help you in this case.

stefan-mysten avatar Aug 12 '25 15:08 stefan-mysten

I see. Well, I tried!

El martes 12 de agosto del 2025, stefan-mysten escribió:

stefan-mysten left a comment (MystenLabs/sui#22378)

@shackra sorry to hear. Unfortunately, tokio-msim is a hard dependency and cannot be removed.

suiup is a tool to manage versions of different CLIs in the Sui ecosystem; in general, it works with releases and does not build from scratch unless requested, so I don't think it can help you in this case.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

-- Jorge Araya

Contacto: Telegram: t.me/shackra · Signal: Shackra.28

shackra avatar Aug 12 '25 15:08 shackra