crytic_compile.py | get_line_from_offset() pulling from incorrect file

[plotchy]

Ran into issues involving KeyError XXX similar to https://github.com/crytic/slither/issues/1324

Did some basic debugging and noticed this is due to sourceMap iteration going out of bounds for particular contracts in the initialization stages of crytic_compile.

I added in some basic print statements to the get_line_from_offset() fn and noticed that changing unrelated contracts would adjust the KeyError OoB value for another unrelated contract.

Set Up

Two contract files:

// ./src/A.sol
pragma solidity ^0.8.4;

contract BaseContract{
    uint256 one;
    uint256[50] private __gap;
    uint256 two;
}

contract DerivedContract is BaseContract{
    uint256 three;
    uint256[50] private __gap;
    uint256 four;
}

// ./src/complex_func.sol
pragma solidity ^0.4.24;
// pragma solidity ^0.8.12;

contract Complex {
    uint i0 = 0;
    uint i1 = 0;
    uint i2 = 0;
    uint i3 = 0;
    uint i4 = 0;
    uint i5 = 0;
    uint i6 = 0;

    function complexStateVars() external {
        i0 = 1;
    }
}

Command: slither . As I'm in a foundry project, crytic_compile runs command forge build --extra-output abi --extra-output userdoc --extra-output devdoc --extra-output evm.methodIdentifiers --force under the hood

Debugging Output

FIrst, I added a print statement to show the file file offset, line, and char offset in the get_line_from_offset() fn

https://github.com/crytic/crytic-compile/blob/6bb6e7281b5918789d4045903a7210ecd00e21db/crytic_compile/crytic_compile.py#L273-L292

def get_line_from_offset(self, filename: Union[Filename, str], offset: int) -> Tuple[int, int]:
        ### snip ###

        lines_delimiters = self._cached_offset_to_line[file]

        # added this print:
        print("File offset: " + str(offset) + ", line: " + str(lines_delimiters[offset][0]) + ", char: " + str(lines_delimiters[offset][1]) + " from file: " + str(file.short))

        return lines_delimiters[offset]

Which provides this type of output:

File offset: 0, line: 1, char: 1 from file: src/complex_func.sol
File offset: 24, line: 1, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol

Where two paths diverge

Running complex_func.sol as solidity ^0.4.24 causes a different output from running it as solidity ^0.8.12

Under 0.8.12:

File offset: 28, line: 2, char: 1 from file: src/complex_func.sol
File offset: 52, line: 2, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol
File offset: 77, line: 5, char: 5 from file: src/complex_func.sol  ##Note: Scope has changed to complex_func for inner level contract nodes!
File offset: 88, line: 5, char: 16 from file: src/complex_func.sol
File offset: 94, line: 6, char: 5 from file: src/complex_func.sol
File offset: 105, line: 6, char: 16 from file: src/complex_func.sol
File offset: 111, line: 7, char: 5 from file: src/complex_func.sol
File offset: 122, line: 7, char: 16 from file: src/complex_func.sol
File offset: 128, line: 8, char: 5 from file: src/complex_func.sol
File offset: 139, line: 8, char: 16 from file: src/complex_func.sol
File offset: 145, line: 9, char: 5 from file: src/complex_func.sol
File offset: 156, line: 9, char: 16 from file: src/complex_func.sol
File offset: 162, line: 10, char: 5 from file: src/complex_func.sol
File offset: 173, line: 10, char: 16 from file: src/complex_func.sol
File offset: 179, line: 11, char: 5 from file: src/complex_func.sol
File offset: 190, line: 11, char: 16 from file: src/complex_func.sol
File offset: 197, line: 13, char: 5 from file: src/complex_func.sol
File offset: 257, line: 15, char: 6 from file: src/complex_func.sol

• Compilation succeeds Seems like sourceMapping goes from: complex_func - file level A - file level complex_func - contract level A - contract level ...

Under 0.4.24:

File offset: 0, line: 1, char: 1 from file: src/complex_func.sol
File offset: 24, line: 1, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol
File offset: 77, line: 5, char: 13 from file: src/A.sol  # Note: Why does this stay in file A.sol? And why is it referring to complex_func's offset 77? This is incorrect
File offset: 88, line: 5, char: 24 from file: src/A.sol
File offset: 94, line: 5, char: 30 from file: src/A.sol
File offset: 105, line: 6, char: 10 from file: src/A.sol
File offset: 111, line: 6, char: 16 from file: src/A.sol
File offset: 122, line: 9, char: 7 from file: src/A.sol
File offset: 128, line: 9, char: 13 from file: src/A.sol
File offset: 139, line: 9, char: 24 from file: src/A.sol
File offset: 145, line: 9, char: 30 from file: src/A.sol
File offset: 156, line: 9, char: 41 from file: src/A.sol
File offset: 162, line: 10, char: 5 from file: src/A.sol
File offset: 173, line: 10, char: 16 from file: src/A.sol
File offset: 179, line: 11, char: 3 from file: src/A.sol
File offset: 190, line: 11, char: 14 from file: src/A.sol
File offset: 197, line: 11, char: 21 from file: src/A.sol

• Compilation fails
• Appears that after switching to context of A.sol, it remains there and attempts to map complex_func.sol's source to it

Eventually output continues to a traceback saying KeyError 257, which is OoB in A.sol source. Offset 257 happens to coincide with complex_func.sol's ending curly brace of fn complexStateVars()

Adding lines/comments to complex_func will relate to a new KeyError offset within A.sol.

I'm not sure where the context switching logic is within crytic_compile, but it seems to be working incorrectly for mismatched solidity versions coming from foundry/hardhat repos.

[plotchy]

Likely root issue of both https://github.com/crytic/crytic-compile/issues/269 and https://github.com/crytic/slither/issues/1241 as well