Feature: Test on save

[Willyham]

Hello, thank you for your work on this incredible extension.

I've like to request a 'test on save' feature, which like it says, runs tests every time you save. With most projects using environments like Hardhat in which running tests can take quite a while, this feature might not be so useful. However, forge is generally quick enough to recompile and execute many tests in just a few seconds.

Ideally the feature would allow for an arbitrary test command to run if enabled.

There are a few extensions which follow on from this feature which would also be incredibly useful:

• A 'cover on save' where test coverage data is generated and then used to highlight the editor.
• A way to 'click to run' a single test or single test file

Examples of these features from the vscode-go project:

I'm happy to work on adding these, but I'd appreciate some pointers as to the best way to implement them in the architecture of the plugin.

[juanfranblanco]

Hi @Willyham good idea, I think this can be achieved in a "simple way" by configuring the extension with "tasks to run", (for example test) each task can have a path, and then what to run after save. What do you think? Then things can move on onto other specific areas, i.e run specific test. Edit: This can be generic and / or preconfigured for specific frameworks so the user can setup this quickly.

[Willyham]

Having a generic way to run a task seems like a great internal solution. Yeah I agree that a simple solution in the short term is to just allow arbitrary commands to run at save time. After that it might also make sense to define a specific 'test' action, which can be triggered by a command action or by the save event, and default framework configs etc.

So my proposal would be to have:

• An internal runTask(args...) Promise<string> or similar internal interface to execute arbitrary commands.
• A single config option for Run commands on save: which allows multiple arbitrary commands as strings.
• An output window which shows the result of those commands labelled Solidity tasks or something.

This will let us implement test on save, which is what I'd really like to have the in short term. Building better config on top of that for tests specifically and frameworks seems like a good addition afterwards.

For implementing "coverage on save" I think we'd need to have the specific configuration rather than arbitrary tasks because we need to run some post processing after the command (to parse the result and display in the editor). But we can cross that bridge when we get to it.

Will you have any time to take this on in the near term? If not, can you point me to the place that you think this should be implemented? I don't have much vscode experience but I'm assuming that this would just be implemented on the client side?

[juanfranblanco]

Can you provide the example command? An array of commands prefixed by the name can be set by default, and then on save those commands can be set: ie Test-Forge -> "xxx" and on save commands ("Test-Forge;Coverage-Forge..")

[Willyham]

Command for running the tests? Just forge test -vv should be enough for now. Output is written to stdout 👍

(forge doesn't support coverage yet but they're working on it)