Ability to execute scribble programmatically

[wighawag]

Currently scribble is designed to be used from cli to make it easier to write tool around scribble, it would be great if we could execute it programmatically from js. Maybe it is just a matter of documentation ?

[cd1m0]

Hi! This is a great idea. We currently don't have the bandwith to build a proper library interface for scribble, but will keep the issue around to track this. Just to understand, what would you like to build on top of it? Perhaps this is something that can be solved with the capabilities of the underlying parsing/AST modification library - https://github.com/consensys/solc-typed-ast

[wighawag]

Hi @cd1m0 the idea was to write a plugin for hardhat and the current cli is a bit limiting for programmatic execution of scribble

[cd1m0]

Can you describe what you want the API would expose? Especially if you can highlight functionality that you want in the API, that the CLI doesn't expose. If we can define a small API, maybe we can do this quicker

[wighawag]

I need to look into it again but from memory, what I would like is

• ability to act on a whole folder
• ability to act in memory only, so the I give a folder (a path) or the list of file and their content in input and the api give me back the modified source

The latter would work very well with hardhat as it has the ability to request source to be transformed before being tested

[JoranHonig]

@wighawag is what you would like to do something like:

hardhat scribble test

Which would run the tests with scribble instrumentation?

[wighawag]

@JoranHonig yes, basically, that would be what the user would do. ideally this would happen in memory, like my preprocessor plugin currently work (https://github.com/wighawag/hardhat-preprocessor)

[wighawag]

Actually probably more like hardhat test --scribble

[cd1m0]

When passing in a whole directory worth of files in-memory, what is the data structure that Hardhat uses? Can you point me at an example in the code of a bunch of files or compilation artifacts being passed in to another plugin in-memory?

[wighawag]

@cd1m0 the plugin mentioned above does it on a file per file basis : https://github.com/wighawag/hardhat-preprocessor/blob/8716fc7d330a31cf5bc9e3c5b8ed5937fe450871/src/index.ts#L121

If this is not enough, a change in hardhat should not be too hard so you can process the whole source globally, but I don't think this is required for scribble ?

[cd1m0]

There are 2 possible input modes for Scribble:

1. An entire JSON compiler artifact

2. A list of Solidity file names.

In the second mode, note that even if you only have annotations in A.sol, scribble may need to re-write some other files that are imported by A.sol. This is usually true in the case when you have contract-wide invariants.

So if you are passing files in-memory, you need to specify all files needed to compile a contract, and also probably let Scribble know whats the original path for each one, so that we can resolve imports.

[wighawag]

@cd1m0 thanks for the info. this might indeed require some changes to hardhat, but should not be to hard to add . So if scribble offered an api to pass the data as argument and get the result as output, then all can be done in memory