foundry icon indicating copy to clipboard operation
foundry copied to clipboard

feat: support `--via-ir` flag when running `forge coverage`

Open colinnielsen opened this issue 2 years ago • 23 comments

Component

Forge

Have you ensured that all of these are up to date?

  • [X] Foundry
  • [X] Foundryup

What version of Foundry are you on?

forge 0.2.0 (d896050 2022-10-21T00:08:06.615745Z)

What command(s) is the bug in?

forge coverage --via-ir --report lcov

Operating System

macOS (Apple Silicon)

Describe the bug

Looks like COMPILER OPTIONS are listed when forge coverage -h is run, however it appears that the coverage command does not build through the YUL pipeline. Screen Shot 2022-10-21 at 5 27 21 PM

colinnielsen avatar Oct 21 '22 23:10 colinnielsen

Looks like forge coverage also ignores the via_ir flag in the foundry.toml

colinnielsen avatar Oct 21 '22 23:10 colinnielsen

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

mattsse avatar Oct 22 '22 07:10 mattsse

Shoot... Is there a way around this? I'm worried that this feature would be bricked once solidity moves to the IR pipeline.

colinnielsen avatar Oct 24 '22 15:10 colinnielsen

I've been having this same error without using --via-ir. My project compiles fine with forge build and forge test, but throws Stack too deep when running forge coverage. Also on Apple Silicon with forge 0.2.0 (7f4fb5 2022-11-02).

Oighty avatar Nov 03 '22 18:11 Oighty

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track of the original AST afterward right now.

we should make this clearer on the website.

mattsse avatar Nov 03 '22 18:11 mattsse

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

Is the need for tracking AST isolated to the test contracts? Or does it apply to the primary source code as well?

frank-lim-partior avatar Nov 28 '22 09:11 frank-lim-partior

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

I'm facing the same issue. Having this limitation in mind, do we have any alternative to see test coverage when getting Stack too deep error?

0xrusowsky avatar Dec 23 '22 09:12 0xrusowsky

we should make this clearer on the website.

@mattsse what do you mean by "website"? The Foundry Book? Because if yes, I don't see any documentation about coverage on there.

I am not sufficiently experienced with test coverage software to understand why it is difficult (or impossible) to run coverage over IR-optimized code, but I agree with the commend made by @colinnielsen above - if coverage is not made to work with IR, it will not be as popular as it could be.

PaulRBerg avatar Dec 27 '22 21:12 PaulRBerg

My code compile without --via-ir, and tests works fine, but still getting stack too deep in coverage.

forge 0.2.0 (08a629a 2023-06-03T00:04:10.749254000Z)

haythemsellami avatar Jun 13 '23 05:06 haythemsellami

My code compile without --via-ir, but still getting stack too deep in coverage.

forge 0.2.0 (08a629a 2023-06-03T00:04:10.749254000Z)

This is an annoying error and probably will be a big blocker for Foundry adoption.Also, how to get more output from the CLI command, it does not show to me where the stack too deep error exactly is:

[⠰] Compiling...
[⠒] Compiling 100 files with 0.8.20
[⠘] Solc 0.8.20 finished in 4.24s
Error:
Compiler run failed:
Error: Compiler error (/solidity/libyul/backends/evm/AsmCodeGen.cpp:68):Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables. When compiling inline assembly: Variable headStart is 1 slot(s) too deep inside the stack. Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables.

haythemsellami avatar Jun 13 '23 05:06 haythemsellami

I have the same issue with forge coverage command.

image

smak0v avatar Jun 26 '23 09:06 smak0v

I've been having this same error without using --via-ir. My project compiles fine with forge build and forge test, but throws Stack too deep when running forge coverage. Also on Apple Silicon with forge 0.2.0 (7f4fb5 2022-11-02).

i have the exact same issue

ethereumdegen avatar Oct 19 '23 17:10 ethereumdegen

Hi, I have the exact same issue, is this fix planned?

golden-expiriensu avatar Oct 25 '23 08:10 golden-expiriensu

I can pass forge build without --via-ir, but still occur the same error while running forge coverage. image Does anyone know how to fix it?

kevinsslin avatar Nov 20 '23 10:11 kevinsslin

Same issue here

Pai-Sho avatar Jan 26 '24 19:01 Pai-Sho

Do we have any workarounds for this issue yet?

QEDK avatar Jan 27 '24 11:01 QEDK

forge coverage --ir-minimum works

Pai-Sho avatar Jan 31 '24 15:01 Pai-Sho

forge coverage --ir-minimum is a life saver!

I also had to replace all my assembly { with assembly ("memory-safe") { for it to start working.

sanbir avatar Jun 07 '24 10:06 sanbir

forge coverage --ir-minimum is a life saver!

I also had to replace all my assembly { with assembly ("memory-safe") { for it to start working.

@sanbir To confirm, your assembly blocks are in fact memory safe before you add the annotation?

NoahMarconi avatar Jun 07 '24 13:06 NoahMarconi

Looks like this is a current limitation of forge coverage because --via-ir transforms the AST so there is no good way of keeping track afterwards. Converted the ticket into a feature request so we can look into how we could possible overcome this limitation.

zerosnacks avatar Aug 02 '24 14:08 zerosnacks

forge coverage --ir-minimum is runnning with some limitations on --via-ir. we can use --ir-minimum as an alternate for sometime

mohib231 avatar Aug 06 '24 19:08 mohib231