aptos-core icon indicating copy to clipboard operation
aptos-core copied to clipboard

[Compiler-V2] Add locations of arguments for stackless bytecode

Open fEst1ck opened this issue 10 months ago • 12 comments

Description

This PR adds location information of the target and source registers for each stackless bytecode. We track this in the compiler V2 bytecode generator when constructing stackless bytecode from model ASTs, and record them in fields of FunctionData

pub struct FunctionData {
    ...
    /// Maps the attribute id of a bytecode to the ids of the source temporaries.
    pub target_locations: BTreeMap<AttrId, Vec<AttrId>>,
    /// Maps the attribute id of a bytecode to the ids of the source temporaries.
    pub src_locations: BTreeMap<AttrId, Vec<AttrId>>,
    ...
}

As an example use case, we use this to improve the locations in error messages from ability checking.

ability-check/assign.exp

 Diagnostics:
 error: local `s` of type `assign::S` does not have the `drop` ability
-   ┌─ tests/ability-check/assign.move:17:9
+   ┌─ tests/ability-check/assign.move:17:10
    │
 17 │         *s = S { f: 42, g: T { h: 42 } };
-   │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reference content dropped here
+   │          ^ reference content dropped here

ability-check/explicit_move.exp

 Diagnostics:
 error: local `x` of type `m::R` does not have the `copy` ability
-   ┌─ tests/ability-check/explicit_move.move:12:9
+   ┌─ tests/ability-check/explicit_move.move:12:14
    │
 12 │         some(x);
-   │         ^^^^^^^ copy needed here because value is still in use
+   │              ^ copy needed here because value is still in use
 13 │         some(x);
    │         ------- used here

TODO: When constructing bytecode with new attribute ids in later transformations, we also need to track the locations of its arguments.

Fixes #12974

Type of Change

  • [x] New feature
  • [ ] Bug fix
  • [ ] Breaking change
  • [ ] Performance improvement
  • [ ] Refactoring
  • [ ] Dependency update
  • [ ] Documentation update
  • [ ] Tests

Which Components or Systems Does This Change Impact?

  • [ ] Validator Node
  • [ ] Full Node (API, Indexer, etc.)
  • [x] Move/Aptos Virtual Machine
  • [ ] Aptos Framework
  • [ ] Aptos CLI/SDK
  • [ ] Developer Infrastructure
  • [ ] Other (specify)

fEst1ck avatar Apr 25 '24 05:04 fEst1ck

⏱️ 5h 2m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-targeted-unit-tests 1h 28m 🟩🟩🟩 (+1 more)
rust-move-unit-coverage 1h 21m 🟩🟩🟩🟩 (+1 more)
rust-move-tests 52m 🟥🟥🟩🟩 (+1 more)
rust-lints 28m 🟩🟩🟩🟥 (+1 more)
run-tests-main-branch 25m 🟩🟩🟩🟩🟩 (+1 more)
general-lints 11m 🟩🟩🟩🟩🟩 (+1 more)
check-dynamic-deps 10m 🟩🟩🟩🟩🟩 (+2 more)
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+1 more)
permission-check 24s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 23s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 20s 🟩🟩🟩🟩🟩 (+1 more)

settingsfeedbackdocs ⋅ learn more about trunk.io

trunk-io[bot] avatar Apr 25 '24 05:04 trunk-io[bot]

Maybe you forgot to push some updated code?

brmataptos avatar Apr 30 '24 03:04 brmataptos

All comments addressed. PTAL @vineethk @brmataptos

fEst1ck avatar May 02 '24 01:05 fEst1ck

Forge is running suite realistic_env_max_load on 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a

github-actions[bot] avatar May 02 '24 19:05 github-actions[bot]

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

:white_check_mark: Forge suite realistic_env_max_load success on 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a

two traffics test: inner traffic : committed: 8220 txn/s, latency: 4777 ms, (p50: 4500 ms, p90: 5700 ms, p99: 9900 ms), latency samples: 3543140
two traffics test : committed: 100 txn/s, latency: 1885 ms, (p50: 1800 ms, p90: 2100 ms, p99: 2400 ms), latency samples: 1840
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.207, avg: 0.201", "QsPosToProposal: max: 0.229, avg: 0.217", "ConsensusProposalToOrdered: max: 0.465, avg: 0.425", "ConsensusOrderedToCommit: max: 0.367, avg: 0.352", "ConsensusProposalToCommit: max: 0.787, avg: 0.777"]
Max round gap was 1 [limit 4] at version 1768293. Max no progress secs was 4.855497 [limit 15] at version 1768293.
Test Ok

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6124 txn/s, latency: 5334 ms, (p50: 4800 ms, p90: 8100 ms, p99: 15900 ms), latency samples: 214340
2. Upgrading first Validator to new version: 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1790 txn/s, latency: 16334 ms, (p50: 18400 ms, p90: 23300 ms, p99: 23600 ms), latency samples: 91300
3. Upgrading rest of first batch to new version: 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1601 txn/s, latency: 17754 ms, (p50: 19600 ms, p90: 23000 ms, p99: 23500 ms), latency samples: 84860
4. upgrading second batch to new version: 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3628 txn/s, latency: 8853 ms, (p50: 9600 ms, p90: 12600 ms, p99: 12700 ms), latency samples: 145140
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 9e6d0d8b3a3bf1aab67434bc85e7725a75a1751a passed
Test Ok

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

Forge is running suite realistic_env_max_load on f1464ace38d1d2b64a291db767270de608e0f7ea

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1464ace38d1d2b64a291db767270de608e0f7ea

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1464ace38d1d2b64a291db767270de608e0f7ea

github-actions[bot] avatar May 02 '24 20:05 github-actions[bot]

I'm a rather concerned about this.

  1. What happens if we rewrite code? Will those locations get out of sync?
  2. This currently only addresses the ability checker. How does this work with other phases?
  3. There is a significant overhead with storing all those additional attribute ids. Why don't we directly have BTreeMap<AttrId, Vec<Loc>>?

This looks like a small change, but it will have significant impact on how we can/have to write bytecode processors in the future, so I think we first should have some design discussions (perhaps also a small design doc which discusses the options we have here).

FWIW, one alternative design would be to get the location of an argument from the bytecode which defines the temporary. This may require an analysis and be more expensive to calculate, on the other hand, it has zero redundancy, should be stable regards rewriting, and the extra overhead is only needed if errors need to be reported.

wrwg avatar May 06 '24 02:05 wrwg

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

github-actions[bot] avatar Jun 21 '24 01:06 github-actions[bot]

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

github-actions[bot] avatar Aug 12 '24 01:08 github-actions[bot]