Add support for ethers.js

[Perseverance]

Goal

The goal of this issue is to govern the conversation of enabling new/additional libraries in the zos-lib, thus decoupling zos-lib from web3.js. This will allow more freedom to the developers to use the library they hold in the highest regard. This is a continuation/summary of an off-github communication between the etherlime and zos team. (cc @spalladino)

Context

The team at etherlime would love to implement ZOSDeployer functionality allowing our users to easily deploy and upgrade their contracts through etherlime deployment scripts.

Possible solutions

Temporary solution - Etherlime to include web3.js too

zos-lib is based on web3.js, while etherlime is based on ethers.js. In ideal world we would love to maintain our only library dependancy to be on ethers.js and not bloat (even more :D) our package. At the current time we will use the zos-lib as it is and accept the incoming web3.js dependancy.

In order for all of this to work, etherlimes ZOSDeployer would need to initialise ZWeb3, and make it work with the supplied privateKey (etherlime does not use any configs or hd-wallet plugins so the web3.currentProvider is not populated by default with the inputed private key). All of this can possibly be done by web3.eth.accounts.wallet (unconfirmed) and the result of createProxy can be wrapped in ethers contract, for the developer to continue using normally.

Permanent solution - Refactor zos-lib to be framework-agnostic

In order for a more long-term permanent solution to be applied, the zos-lib would need to be decoupled from web3.js. Here are the places (based on my current findings) that web3.js impacts and would require changes (sometimes even breaking-changes)

ZWeb3.ts

The main class that uses web3.js in zos-lib is the ZWeb3.ts and the only place where web3 is imported. This class does all kinds of framework work - from hashing to sending transactions and estimating gas. My proposition (that you can see in an pull request coming up soon) is create a ZWeb3Interface interface defining the things a ZWeb3 implementation should be able to perform. Then adding a second parameter - implementation allowing the developer to chose from supported ZWeb3 implementations (defaulting to web3.js for backwards compatability)

Contract.ts

Contracts.ts uses web3.js by proxy of ZWeb3.ts contract method. The contract method is very web3.js specific as in other libraries (f.e. ethers) The Interface and Contract objects are separated. While in web3.js you can create contract object without address, and later deploy it, this is not doable in ethers as you have three different objects for different purposes:

• Interface - if you know the ABI of a contract but not the address
• ContractFactory - if you know the ABI of a contract and want to deploy it
• Contract - if you know the ABI and have an address

All of these have different properties that are merged in the web3.js version of contract. The problem lies in the inability of any other library to return a singular object containing all three classes and behaving the same way of web3.js Contract. (I'm also not sure if this makes sense)

Anywhere that the Contract interface from Contract.ts is used

As the Contract interface from Contract.ts is pretty much an extension of Web3.js Contract the opinions of web3.js contract are transferred here too. Couple of examples, incompatible with other libraries is the use of methods object for containing all functions of a contract and the .call(), .send() methods that need to be called based on whether a transaction is view or not.

Decoupling from web3.js would mean refactoring every single place .call() or .send() is used.

The from parameter

Last but not least, ethers uses ethers.Wallet instance in order to connect to a smart contract and allow for seemless sign and send execution. This means that the developer needs to create and connect a wallet object (either through privateKey, mnemonic, metamask, etc) in order to interact a smart contract. This wallet has it's own address and therefore the from field is not really used in the ethers contracts (and actually throws if it is used). Basically everywhere that from is used a custom logic for creation and connection of ethers.Wallet needs to be applied. I am scared to find out how many these are :D

Initial progress and research

We've been able to make a good progress on the Temporary solution, but the more permanent one has been a tricky one.

I am submitting a pull request where you can see the state of my work. Most of the next things are very sensitive and possibly breaking and I'd like to throw my PR open so that it is seen by the rest of the community :)

[spalladino]

Thanks so much @Perseverance for the work and the related writeup! As discussed offline, this decision is an important one, and exceeds an integration of zOS and etherlime, or even zOS itself if we are to maintain consistency in js libraries across all Zeppelin products.

An alternative option to being library-agnostic is to use very lightweight libraries in zOS itself (the ethereum-js family being a good candidate, to minimize bloating), while developing adapters just for the contracts the library returns. So when a user runs createProxy, we can return a web3, ethers, truffle, etherlime, or whatever contract abstraction it is - even if under the hood we used something different entirely. We'll iterate these ideas and get back.

---

Regarding the issue on the tight coupling between zos-lib and web3 contracts, due to the usage of contract.methods.method().call(), I think we can do something similar to the Transactions#sendTransaction function. This is, instead of using the contract methods directly, have a standalone function callContract(contract, fn, args) that handles the call. This should allow us to easily swap one implementation for a different one.

As for the from parameter, since all transactions are condensed in the Transactions module, it should be easy to change in a single place if we need to do so.

[Perseverance]

A bit of update from the ZOSDeployer from etherlime.

We've augmented the deployer with the addition of a wallet to the in memory array:

let wallet = await ZWeb3.eth().accounts.wallet.add(this.signer.privateKey)
txParams.from = wallet.address

Trying to connect to infura (the most common usecase in etherlime) results in:

Error: Returned error: The method `eth_sendTransaction` does not exist/is not available

So apparently (and actually quite logically, I just haven't thought about it) web3.js uses eth_sendTransaction that relies on an unlocked account in the node - something that cannot happen in Infura for obvious reasons. Unfortunately, even the addition of the wallet in the internal wallets repository did not make web3.js to sign the transaction and broadcast it through eth_sendRawTransaction.

So at this point, I am not sure how one can deploy on ropsten without relying on truffles provider to be supplied (through hd-wallet-provider).

I guess the intermediary solution is off limits now and we should start thinking about the permanent one :S

[spalladino]

@Perseverance note that zOS is creating a new instance of web3.js on every call - only the provider is persisted among different requests. This is probably causing that the added wallet is lost right after you've added it. I'd suggest trying to change the web3() function so it keeps a single instance of web3, and see if it works then!

[josipbagaric]

Has any progress been made on this issue? Looking forward to having Ethers.js support in upgradable smart contracts.

[frangio]

We are making progress! See #1528 "Redesigning the JavaScript Library".