Make `flatten` task smarter

[fvictorio]

The flatten task only generates the dependency graph of the project's files and merges them into a single file (removing all the import statements). This works for some scenarios, but in others, it generates a file that cannot be compiled. Examples of this are having multiple SPDX comments in the same file, or having multiple pragma abicoder, even if they are compatible.

We want to make this task smarter so that it generates compilable files most of the time.

This document has a more in-depth explanation of what we want to do. At the end there is a list of test cases that should make the expected behavior clearer.

[alcuadrado]

Hey, @nachomazzara and @boringcrypto, wdyt about this document?

[boringcrypto]

This is great! The only thing I'd like to add would be a way to select where the output goes. I'm on windows, so I can only pipe everything to a file, including console.logs and stdout messages, etc.

[fvictorio]

I think we can add a --output param

[nachomazzara]

Hey all! thanks for taking this seriously. It has been a big pain for me (A paranoid man who likes to deploy contracts manually)

I like the proposal! let's make it happen 🚀

[ihorbond]

@fvictorio was this ticket ever resolved? Came here because of duplicate licenses too

[boringcrypto]

I've been using this: https://github.com/nomiclabs/hardhat/issues/1050#issuecomment-840444798 - hacky, but solved my problem

[TrejGun]

guys please, this is really annoying

[partylikeits1983]

quick and dirty way with python:

$ npx hardhat flatten contracts/bank.sol > contracts/flatten.sol

import os

bad_words = ['SPDX', 'pragma']

with open('/home/ubuntu/Desktop/defiBank/contracts/flatten.sol') as oldfile, open('/home/ubuntu/Desktop/defiBank/contracts/flat.sol', 'w') as newfile:
    for line in oldfile:
        if not any(bad_word in line for bad_word in bad_words):
            newfile.write(line)
            
filename = "/home/ubuntu/Desktop/defiBank/contracts/flat.sol"

string = "// SPDX-License-Identifier: MIT \n pragma solidity ^0.8.0; \n"
            
def insert(originalfile,string):
    with open(originalfile,'r') as f:
        with open('newfile.txt','w') as f2: 
            f2.write(string)
            f2.write(f.read())
    os.rename('newfile.txt',originalfile)
    
insert(filename,string)

os.remove("/home/ubuntu/Desktop/defiBank/contracts/flatten.sol")

[ghost]

For anyone interested, I have ported the tsort library to Typescript

https://github.com/ayanamitech/tsort-ts

cc @boringcrypto @fvictorio

[TrejGun]

For anyone interested, I have ported the tsort library to Typescript

https://github.com/ayanamitech/tsort-ts

cc @boringcrypto @fvictorio

the link is broken

[TrejGun]

guys, please! remix, etherscan, nethereum and I guess much more tools requires only one SPDX. Its a pain to remove duplicates manually

[abarbatei]

@TrejGun I have been using the custom task described here, it basically adds a flat command that does what flatten should do: https://rubydusa.medium.com/parsererror-multiple-spdx-license-identifiers-found-in-source-file-de4d61ffab64

task("flat", "Flattens and prints contracts and their dependencies (Resolves licenses)")
  .addOptionalVariadicPositionalParam("files", "The files to flatten", undefined, types.inputFile)
  .setAction(async ({ files }, hre) => {
    let flattened = await hre.run("flatten:get-flattened-sources", { files });
    
    // Remove every line started with "// SPDX-License-Identifier:"
    flattened = flattened.replace(/SPDX-License-Identifier:/gm, "License-Identifier:");
    flattened = `// SPDX-License-Identifier: MIXED\n\n${flattened}`;

    // Remove every line started with "pragma experimental ABIEncoderV2;" except the first one
    flattened = flattened.replace(/pragma experimental ABIEncoderV2;\n/gm, ((i) => (m) => (!i++ ? m : ""))(0));
    console.log(flattened);
  });

Hope it helps

[TrejGun]

@abarbatei thanks a lot!

[fvictorio]

This was released in Hardhat v2.17.1

[SvenMeyer]

@fvictorio I am using hardhat v2.17.3 and "hardhat-gas-reporter": "^1.0.4", but still with hardhat export-flat I get ParserError: Multiple SPDX license identifiers found in source file What am I missing ?

[SvenMeyer]

my temporary (manual) "solution :

$ sed -i '/SPDX-License-Identifier/d' flat/contract.sol