[dependencies] update jemalloc to tikv-jemalloc for better compatilibity

[0x5ea000000]

Description

List dependencies that are required for this change:

jemallocator = { package = "tikv-jemallocator", version = "0.5.0", features = [
    "profiling",
    "unprefixed_malloc_on_supported_platforms",
] }
jemalloc-sys = { package = "tikv-jemalloc-sys", version = "0.5.4" }

Jemalloc is currently being maintained by tikv and distributed under two names: jemallloc and tikv-jemalloc, both do the same thing. It's recommended to use tikv-xxx versions for better compatibility. I am trying to use Aptos as the settlement layer and data availability layer for Madara Starknet stack and have conflict between 2 version of jemalloc.

Updating git repository `https://github.com/aptos-labs/aptos-core`
    Updating crates.io index
error: failed to select a version for `jemalloc-sys`.
    ... required by package `jemallocator v0.5.0`
    ... which satisfies dependency `jemallocator = "^0.5.0"` of package `aptos-block-partitioner v0.1.0 (https://github.com/aptos-labs/aptos-core#a270a1be)`
    ... which satisfies git dependency `aptos-block-partitioner` of package `aptos-vm v0.1.0 (https://github.com/aptos-labs/aptos-core#a270a1be)`
    ... which satisfies git dependency `aptos-vm` of package `aptos-api-types v0.0.1 (https://github.com/aptos-labs/aptos-core#a270a1be)`
    ... which satisfies git dependency `aptos-api-types` of package `aptos-sdk v0.0.3 (https://github.com/aptos-labs/aptos-core#a270a1be)`
    ... which satisfies git dependency `aptos-sdk` of package `mc-settlement v0.1.0 (/home/administrator/madara/crates/client/settlement)`
    ... which satisfies path dependency `mc-settlement` (locked to 0.1.0) of package `madara v0.7.0 (/home/administrator/madara/crates/node)`
versions that meet the requirements `^0.5.0` are: 0.5.4+5.3.0-patched, 0.5.3+5.3.0-patched, 0.5.2+5.3.0-patched, 0.5.1+5.3.0-patched, 0.5.0+5.3.0

the package `jemalloc-sys` links to the native library `jemalloc`, but it conflicts with a previous package which links to `jemalloc` as well:
package `tikv-jemalloc-sys v0.5.4+5.3.0-patched`
    ... which satisfies dependency `tikv-jemalloc-sys = "^0.5"` (locked to 0.5.4+5.3.0-patched) of package `librocksdb-sys v0.11.0+8.1.1`
    ... which satisfies dependency `librocksdb-sys = "^0.11.0"` (locked to 0.11.0+8.1.1) of package `rocksdb v0.21.0`
    ... which satisfies dependency `rocksdb = "^0.21"` (locked to 0.21.0) of package `kvdb-rocksdb v0.19.0`
    ... which satisfies dependency `kvdb-rocksdb = "^0.19.0"` (locked to 0.19.0) of package `mc-db v0.7.0 (/home/administrator/madara/crates/client/db)`
    ... which satisfies path dependency `mc-db` (locked to 0.7.0) of package `madara v0.7.0 (/home/administrator/madara/crates/node)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the `links = "jemalloc"` value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

failed to select a version for `jemalloc-sys` which could resolve this conflict

Type of Change

• [ ] New feature
• [ ] Bug fix
• [ ] Breaking change
• [ ] Performance improvement
• [ ] Refactoring
• [x] Dependency update
• [ ] Documentation update

Which Components or Systems Does This Change Impact?

• [ ] Validator Node
• [ ] Full Node (API, Indexer, etc.)
• [ ] Move/Aptos Virtual Machine
• [ ] Aptos Framework
• [ ] Aptos CLI/SDK
• [ ] Developer Infrastructure
• [ ] Other (specify)

How Has This Been Tested?

Key Areas to Review

Checklist

• [x] I have read and followed the CONTRIBUTING doc
• [ ] I have performed a self-review of my own code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I identified and added all stakeholders and component owners affected by this change as reviewers
• [ ] I tested both happy and unhappy path of the functionality
• [ ] I have made corresponding changes to the documentation

[trunk-io[bot]]

⏱️ 12s total CI duration on this PR

Job	Cumulative Duration	Recent Runs
permission-check	4s	🟥
permission-check	3s	🟥
permission-check	3s	🟥
permission-check	2s	🟥

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[msmouse]

Thank you for the contribution. Looks reasonable. Could you please rebase and let's make it through all the tests?

[0x5ea000000]

Thank you for the contribution. Looks reasonable. Could you please rebase and let's make it through all the tests?

Thank you for reviewing my pull request and for accepting the change! I really appreciate your feedback. I'll get right on it and rebase my changes and ensure they pass all the tests.

[github-actions[bot]]

Forge is running suite realistic_env_max_load on 7b16854087d799189e202b21a620aeb0406604bb

• Grafana dashboard (auto-refresh)
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on aptos-node-v1.10.1 ==> 7b16854087d799189e202b21a620aeb0406604bb

• Grafana dashboard (auto-refresh)
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite compat success on aptos-node-v1.10.1 ==> 7b16854087d799189e202b21a620aeb0406604bb

Compatibility test results for aptos-node-v1.10.1 ==> 7b16854087d799189e202b21a620aeb0406604bb (PR)
1. Check liveness of validators at old version: aptos-node-v1.10.1
compatibility::simple-validator-upgrade::liveness-check : committed: 6629 txn/s, latency: 5039 ms, (p50: 4800 ms, p90: 9000 ms, p99: 10200 ms), latency samples: 232020
2. Upgrading first Validator to new version: 7b16854087d799189e202b21a620aeb0406604bb
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1562 txn/s, latency: 18716 ms, (p50: 23400 ms, p90: 30300 ms, p99: 30500 ms), latency samples: 79680
3. Upgrading rest of first batch to new version: 7b16854087d799189e202b21a620aeb0406604bb
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1573 txn/s, latency: 18867 ms, (p50: 19900 ms, p90: 24300 ms, p99: 24700 ms), latency samples: 84980
4. upgrading second batch to new version: 7b16854087d799189e202b21a620aeb0406604bb
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3598 txn/s, latency: 8733 ms, (p50: 9700 ms, p90: 12600 ms, p99: 12900 ms), latency samples: 143940
5. check swarm health
Compatibility test for aptos-node-v1.10.1 ==> 7b16854087d799189e202b21a620aeb0406604bb passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite realistic_env_max_load success on 7b16854087d799189e202b21a620aeb0406604bb

two traffics test: inner traffic : committed: 8201 txn/s, latency: 4779 ms, (p50: 4500 ms, p90: 5700 ms, p99: 10200 ms), latency samples: 3543060
two traffics test : committed: 100 txn/s, latency: 1862 ms, (p50: 1800 ms, p90: 2100 ms, p99: 7800 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.205, avg: 0.202", "QsPosToProposal: max: 0.238, avg: 0.229", "ConsensusProposalToOrdered: max: 0.442, avg: 0.406", "ConsensusOrderedToCommit: max: 0.327, avg: 0.314", "ConsensusProposalToCommit: max: 0.733, avg: 0.720"]
Max round gap was 1 [limit 4] at version 1321279. Max no progress secs was 4.736854 [limit 15] at version 1321279.
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[0x5ea000000]

@msmouse Long time no see sir, I forgot this PR due to a lot of things that need to be done. Last time when you approved this PR, it had some failures in tests and I've updated it. Please kick off the test again

[github-actions[bot]]

Forge is running suite realistic_env_max_load on ef15a3f0076da839a58464fd728afba5d21bd7f4

• Grafana dashboard (auto-refresh)
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> ef15a3f0076da839a58464fd728afba5d21bd7f4

• Grafana dashboard (auto-refresh)
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite realistic_env_max_load success on ef15a3f0076da839a58464fd728afba5d21bd7f4

two traffics test: inner traffic : committed: 8537 txn/s, latency: 4591 ms, (p50: 4400 ms, p90: 5400 ms, p99: 10200 ms), latency samples: 3688300
two traffics test : committed: 100 txn/s, latency: 1954 ms, (p50: 1800 ms, p90: 2100 ms, p99: 6000 ms), latency samples: 1720
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.204, avg: 0.202", "QsPosToProposal: max: 0.205, avg: 0.188", "ConsensusProposalToOrdered: max: 0.445, avg: 0.407", "ConsensusOrderedToCommit: max: 0.353, avg: 0.337", "ConsensusProposalToCommit: max: 0.758, avg: 0.743"]
Max round gap was 1 [limit 4] at version 1737156. Max no progress secs was 5.089176 [limit 15] at version 1737156.
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> ef15a3f0076da839a58464fd728afba5d21bd7f4

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> ef15a3f0076da839a58464fd728afba5d21bd7f4 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6315 txn/s, latency: 5122 ms, (p50: 4800 ms, p90: 9400 ms, p99: 10600 ms), latency samples: 233660
2. Upgrading first Validator to new version: ef15a3f0076da839a58464fd728afba5d21bd7f4
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1577 txn/s, latency: 17929 ms, (p50: 19300 ms, p90: 25100 ms, p99: 25700 ms), latency samples: 86740
3. Upgrading rest of first batch to new version: ef15a3f0076da839a58464fd728afba5d21bd7f4
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1657 txn/s, latency: 15966 ms, (p50: 19000 ms, p90: 23000 ms, p99: 23900 ms), latency samples: 89500
4. upgrading second batch to new version: ef15a3f0076da839a58464fd728afba5d21bd7f4
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3457 txn/s, latency: 8729 ms, (p50: 9900 ms, p90: 11800 ms, p99: 12600 ms), latency samples: 145220
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> ef15a3f0076da839a58464fd728afba5d21bd7f4 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[msmouse]

hmm.. I'm sorry but I guess you need to rebase it again. (I copied your change and rebased it and tests pass https://github.com/aptos-labs/aptos-core/pull/13225)

[0x5ea000000]

I rebased it

[0x5ea000000]

@msmouse could you recheck this PR?

[msmouse]

@0x5ea000000 hmm.. looks like it does build this time..

[github-actions[bot]]

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

[benluelo]

i've hit this issue as well - can this be revived?

[0x5ea000000]

i've hit this issue as well - can this be revived?

Now it's more complicated that aptos-core repo and aptos-indexer-proccessor repo has cyclic dependency and both are using jemalloc. So both should be changed to tikv version on the same time :(

[benluelo]

yeah, i also hit the cyclic dependency while trying to fix this myself - if it were up to me, i would revert https://github.com/aptos-labs/aptos-core/pull/11562, since it somewhat defeats the point of having a "core" repo. i also hit many other issues with conflicting -sys dependencies, since the rest client brings in essentially the full node and vm(?), which was also a bit of a mess to fix. (this is what i've had to do so far: https://github.com/aptos-labs/aptos-core/compare/main...unionlabs:aptos-core:main)

there are also other issues with conflicting dependencies:

• https://github.com/aptos-labs/aptos-core/issues/7342
• https://github.com/aptos-labs/aptos-core/issues/11539

and i personally also hit an issue with rocksdb (just trying to use the rest client!). we've forked and patched for our needs, but if some of these issues could be resolved upstream, it would make integration much easier.