Mangled dependency import remappings

[KPrasch]

Environment information

• OS: Ubuntu 20.04
• Python Version: 3.12
• ape and plugin versions:

$ ape --version
0.7.7

$ ape plugins list
Installed Plugins
  solidity    0.7.1

Ape Config

name: nucypher

plugins:
  - name: solidity

dependencies:
  - name: nucypher-contracts
    github: nucypher/nucypher-contracts
    ref: main

  - name: openzeppelin
    github: OpenZeppelin/openzeppelin-contracts
    version: 5.0.0

solidity:
  version: 0.8.23
  evm_version: paris
  import_remapping:
    - "@openzeppelin/contracts=openzeppelin/v5.0.0"

test:
  provider:
    chain_id: 131277322940537  # ensure ape doesn't change chain id to 1337
  mnemonic: test test test test test test test test test test test junk
  number_of_accounts: 30

What went wrong?

After upgrading from ape 0.6.27 -> 0.7.7 and clearing the local packages cache CompilationError is raised when accessing dependency contracts. The missing source paths have mangled and unusual paths like matic/@openzeppelin-upgradeable/contracts/ and contracts/coordination/@openzeppelin-upgradeable/contracts/.

Observations

• according to the logs all contacts compiled successfully (see below)
• It appears to be an issue resolving import remappings using the dependency API.
• both the local project the dependency are using the same version of OZ
• downgrading & clearing local packages resolves the issue
• adding local remappings and config overrides seem to have no impact

Please include information like:

• what command you ran

pytest

• the code that caused the failure (see this link for help with formatting code)

dependency = project.dependencies["nucypher-contracts"]
...
account.deploy(dependency.TestToken, TOTAL_SUPPLY) # <<< 

• full output of the error you received

    _t_token = deployer_account.deploy(nucypher_dependency.TestToken, TOTAL_SUPPLY)
E   ape.exceptions.CompilerError: Missing sources: 
'contracts/@threshold/contracts/staking/IStaking.sol, 
test/@openzeppelin/contracts/token/ERC20/IERC20.sol, 
contracts/@threshold/contracts/staking/IApplication.sol,
matic/@openzeppelin-upgradeable/contracts/access/AccessControlUpgradeable.sol, 
xchain/@fx-portal/contracts/tunnel/FxBaseRootTunnel.sol, 
xchain/@fx-portal/contracts/tunnel/FxBaseChildTunnel.sol, 
test/@openzeppelin/contracts/token/ERC20/ERC20.sol, 
threshold/@threshold/contracts/staking/IApplication.sol, 
xchain/@openzeppelin/contracts/access/Ownable.sol,
contracts/coordination/@openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol,
contracts/@openzeppelin/contracts/utils/math/SafeCast.sol,
contracts/coordination/@openzeppelin/contracts/utils/cryptography/ECDSA.sol, 
test/@openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol, 
contracts/coordination/@openzeppelin-upgradeable/contracts/access/AccessControlUpgradeable.sol, 
contracts/proxy/@openzeppelin/contracts/access/Ownable.sol, 
contracts/coordination/@openzeppelin/contracts/utils/cryptography/MessageHashUtils.sol, 
contracts/testnet/@openzeppelin/contracts/access/Ownable.sol, 
contracts/coordination/@openzeppelin-upgradeable/contracts/proxy/utils/Initializable.sol, 
test/@threshold/contracts/staking/IApplication.sol, 
contracts/@openzeppelin-upgradeable/contracts/access/OwnableUpgradeable.sol,
matic/@openzeppelin-upgradeable/contracts/proxy/utils/Initializable.sol,
contracts/@openzeppelin/contracts/access/Ownable.sol, 
contracts/coordination/@openzeppelin/contracts/token/ERC20/IERC20.sol,
contracts/@openzeppelin/contracts/utils/math/Math.sol, 
contracts/@openzeppelin/contracts/token/ERC20/IERC20.sol,
contracts/coordination/@openzeppelin-upgradeable/contracts/access/extensions/AccessControlDefaultAdminRulesUpgradeable.sol,
test/@threshold/contracts/staking/IStaking.sol, 
contracts/@openzeppelin/contracts/token/ERC20/ERC20.sol,
contracts/@openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol, 
contracts/testnet/@openzeppelin/contracts/token/ERC20/ERC20.sol'.
        deployer_account = <TestAccount 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266>
        nucypher_dependency = <GithubDependency github=nucypher/nucypher-contracts>
-------------------------------------- Captured log setup --------------------------------------
SUCCESS ape:logging.py:33 Contract 'RitualToken' deployed to: 0x5FbDB2315678afecb367f032d93F642f64180aa3
WARNING  ape:config.py:229 Unprocessed plugin config(s): import_remapping. Plugins may not be installed yet or keys may be mis-spelled.
INFO     ape:compilers.py:156 Compiling 'test/AdjudicatorTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'test/StakingEscrowTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/GlobalAllowList.sol'.
INFO     ape:compilers.py:156 Compiling 'test/TestToken.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/ITACoRootToChild.sol'.
INFO     ape:compilers.py:156 Compiling 'test/BetaProgramInitiatorTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/BadContracts.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/TACoChildApplication.sol'.
INFO     ape:compilers.py:156 Compiling 'threshold/IApplicationWithOperator.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/Destroyable.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/proxy/Upgradeable.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/ContractV2.sol'.
INFO     ape:compilers.py:156 Compiling 'aragon/interfaces/IERC900History.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/TestnetThresholdStaking.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/Coordinator.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/IFeeModel.sol'.
INFO     ape:compilers.py:156 Compiling 'threshold/ITACoChildApplication.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/ContractV4.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/Adjudicator.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/ReceiveFallbackTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'xchain/PolygonChild.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/FlatRateFeeModel.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/NuCypherToken.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/IStakingEscrow.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/ReEncryptionValidator.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/UmbralDeserializer.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/StakingEscrow.sol'.
INFO     ape:compilers.py:156 Compiling 'test/LibTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/Snapshot.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/testnet/TapirSet.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/testnet/OpenAccessAuthorizer.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/BLS12381.sol'.
INFO     ape:compilers.py:156 Compiling 'test/TACoChildApplicationTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'test/CoordinatorTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'test/TACoApplicationTestSet.sol'.
INFO     ape:compilers.py:156 Compiling 'matic/SubscriptionManager.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/SignatureVerifier.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/TACoApplication.sol'.
INFO     ape:compilers.py:156 Compiling 'threshold/IApplicationWithDecreaseDelay.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/BetaProgramInitiator.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/testnet/LynxSet.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/IEncryptionAuthorizer.sol'.
INFO     ape:compilers.py:156 Compiling 'test/ReceiveApprovalMethodMock.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/lib/Bits.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/IReimbursementPool.sol'.
INFO     ape:compilers.py:156 Compiling 'test/Dummy.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/ContractV1.sol'.
INFO     ape:compilers.py:156 Compiling 'xchain/PolygonRoot.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/coordination/ITACoChildToRoot.sol'.
INFO     ape:compilers.py:156 Compiling 'test/proxy/ContractV3.sol'.
INFO     ape:compilers.py:156 Compiling 'contracts/proxy/Dispatcher.sol'.

How can it be fixed?

Fill this in if you have ideas on how the bug could be fixed.

[linear[bot]]

APE-1679 Mangled dependency import remappings

[antazoey]

Hello, sorry you are having this issue. It seems wild, I have no idea what is going on there.

Good news: I have been hard at work for a week and will be for a little longer on a mega refactor / re-design of the project, dependencies, and compilation systems in Ape. It is a million times better.

I will ensure this issue is resolved in the releasing of the refactor.

Bad news : it won't be out until 0.8, but it's the last big piece we need for 0.8. I can try to patch in 0.7 but that would take away from 0.8. I can be convinced to do that though.

Sorry again and thank you for reporting.

[antazoey]

I have learned a bit since I last commented. I am pretty sure the issue lies in this part of the code: https://github.com/ApeWorX/ape-solidity/blob/main/ape_solidity/compiler.py#L1098-L1114

[antazoey]

WORKAROUND:

So this won't be a problem in 0.8, but in the meantime, just copy in the same solidity config to the config override of your dependency, delete the package from the .ape/packages, and then re-try.

name: nucypher

plugins:
  - name: solidity

dependencies:
  - name: nucypher-contracts
    github: nucypher/nucypher-contracts
    ref: main
    config_override:
      solidity:
        version: 0.8.23
        evm_version: paris
        import_remapping:
          - "@openzeppelin/contracts=openzeppelin/v5.0.0"
          - "@openzeppelin-upgradeable/contracts=openzeppelin-upgradeable/v5.0.0"
          - "@fx-portal/contracts=fx-portal/v1.0.5"
          - "@threshold/contracts=threshold/v1.2.1"

  - name: openzeppelin
    github: OpenZeppelin/openzeppelin-contracts
    version: 5.0.0

solidity:
  version: 0.8.23
  evm_version: paris
  import_remapping:
    - "@openzeppelin/contracts=openzeppelin/v5.0.0"

test:
  provider:
    chain_id: 131277322940537  # ensure ape doesn't change chain id to 1337
  mnemonic: test test test test test test test test test test test junk
  number_of_accounts: 30

After doing this, things work.

In [3]: dependency.TestToken
Out[3]: <TestToken>

[derekpierre]

@antazoey - the workaround works. Thanks!