ethereum
[CLI] Rewrite the command line interface (solc) and consider using StandardCompiler as backend.
CommandLineInterface.cpp is just one single huge mess right now :-). See other related issues #9579, #8722, #5306.
For an initial implementation see #11960.
Tasks:
- [x] #6900 (to avoid the need to rewrite this JSON code)
- [ ] #10278 (to remove the need creating a translator to
--combined-json) - [x] #11507
- [ ] Support displaying error ids (this is optional, we could manually craft the error output, but if we want to use
formattedMessagethen this is needed) - [ ] After doing this transition, the back-and-forth conversion done in
CommandLineParseris no longer needed as the JSON should just expect the input strings
The rewrite is also a good opportunity to fix confusing options:
- [ ] #14364
- [ ] #13714
Strange there wasn't an issue for this, it has been one goal for me at least :+1:
Of course there was :) #7813. I'm closing it in favor of this one here. As short as its description is, it's still more detailed than in the old one :)
IMHO another argument for this: as far as I'm concerned https://github.com/ethereum/solidity/pull/9954 should not have been done like that, but rather the command line interface be fixed.
Need to also keep #8847 in mind, such as:
Some options available to the standard-json format do not seem to be available as CLI flags. I'm missing in particular source mappings for yul which was introduced only for the json options in #8378. It would be nice if feature parity between CLI flags and json opts was pursued as a general rule.
From https://github.com/ethereum/solidity/pull/10241#pullrequestreview-526581636: I think there should be just one top-level try...catch block around everything in the CLI interface that catches exceptions not properly handled below, reports any diagnostic info it finds and exits. Currently there are multiple ad-hoc try...catch with very limited scope that look as if they were going to handle errors but all they don't actually do anything, they just report and exit and that should be delegated to the top level.
I think we should really do #5303 before this and get full test coverage of all existing options. Otherwise we'll be dealing with random bugs that nobody noticed for months to come.
I have actually tried to start this recently by adding a boost test suite for CommandLineInterface class but it requires either making a lot of stuff public or splitting it into smaller components that can be tested independently.
Actually, I wonder how attached people are to the current command line options...
Ideally I would suggest to drop all of them and replace them by an extremely simple 1:1 translation from StandardJSON to new command line options in a breaking release - at least if we can come up with a nice way to do so. That way unit tests for StandardCompiler are basically enough to cover both CLI and standard json and only a few "proper" command line and standard json tests on top of that would be enough - and we never have to actually bother with https://github.com/ethereum/solidity/issues/5303 too much at all.
If we're afraid to break tooling too much like that, I'd write a thin wrapper script that covers the most commonly used old command line options and translates them to the new options, but with a "use-at-your-own-peril" plus "better-update-your-stuff" policy... But yeah, I'm not sure we'll be able to agree on this and if we want to consider it, we'd probably need a draft of how the simple CLI to StandardJSON translation would look like in practice to decide if it is likely to increase the usage learning curve too much or not.
Actually, I wonder how attached people are to the current command line options...
See #10278. Answer is: pretty much :wink:
Also check https://github.com/ethereum/solidity/projects/44 for other proposed changes.
If I read that the main attachment is towards non-standard-json output options of the CLI. Actually I agree that we should keep those... or put differently: I would keep --standard-json as CLI option anyways, meaning proper json input and output - but if the compiler is instead invoked using the "simple translation of CLI options to standard-json" I also wouldn't output standard-json, but rather just plainly output the requested output components. And we could keep --combined-json to pack the output like that instead... but yeah, granted: things get a bit fuzzy there, I hadn't thought this through entirely...
Exactly, I think the goal here is to have a cleaner interface between the CLI and the CompilerStack.
Added a task list here and created an initial implementation attempt in #11960.
@axic There are some more steps that need to be added to the list. Standard JSON has no equivalent for:
-
--import-astmode -
--import-asm-jsonmode (soon to be added to CLI in #11807) -
--linkmode
- #11507
Do we actually need it for the rewrite though? We can just craft the colored message from the components included in the JSON output. I'd even argue that it's the better way to do it because it ensures that we're not missing anything that would be necessary for external tools do the same thing.
- #6900 (to avoid the need to rewrite this JSON code)
I don't understand why this is needed to rewrite the CLI. Are you referring to the code for formatting JSON output in StandardCompiler? Why would we rewrite it?
Another thing that's not available in Standard JSON is the Yul->Ewasm translation (i.e. solc input.yul --strict-assembly --yul-dialect evm --machine ewasm).
Related:
- https://github.com/ethereum/solidity/issues/14364
- https://github.com/ethereum/solidity/issues/13714
- https://github.com/ethereum/solidity/issues/10278
Added to the description. Note that #10278 was already there. I also removed Support parser error recovery setting in standard json, which was obsoleted by #14395.