stdlib prototype

[axic]

Part of #10282.

TODO:

• [ ] provide a way to export the standard library as a directory hierarchy from the commandline interface (or standard-json - then it would be a json map of filename -> string)
• [ ] only include standard library files in the compiler stack input files if they were imported (use the import file resolution mechanism for that)
• [ ] if a file is included whose name clashes with a file in the standard library (or maybe even the prefix std/), check if its contents match the built-in contents and don't do anything. Otherwise warn.

If the three points are combined, you can export the standard library to a folder and importing from that folder would do the same as importing the built-in standard library.

The contents of the standard library are of course version-dependent.

[axic]

I wonder if we should just start committing stdlib under semanticTests/externalSources and iterating on it? And once it is in a useful state, git move those files to /stdlib and get it exposed in the compiler.

[leonardoalt]

@axic wen stdlib

[axic]

Pushed here a new approach, where a compiler flag enables the stdlib. This way perhaps we can merge these 3 precompiles as a start.

[stackenbotten]

There was an error when running chk_coding_style for commit a545739d9abb37fceb7c4f8a59acb779be6e0cb5:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit ca9b70c19e8b86f462076c2c40f0eaeee6b2966c:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[chriseth]

We wanted to do this via a pragma now, right?

[stackenbotten]

There was an error when running chk_coding_style for commit 988e1ce4eb3c5b8bae58424cbff44821bd37fffc:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit 90c67d4e88b9d9e4b6ed006d9b9beabf21ca5b67:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[chriseth]

Still voting for the explicit exports, which might be really needed here: https://github.com/ethereum/solidity/issues/6482#issuecomment-659044989

[chriseth]

We should use a pragma to activate the standard library behaviour on a per-file basis. For this, we need a list of symbols that should be excluded from the global symbols in the lookup when the pragma is active. This can be implemented in the Name Resolver - whenever we look up a symbol from a file with the pragma and reach the 'nullptr' scope, we only return names that are not in the exclusion list.

[axic]

We should use a pragma to activate the standard library behaviour on a per-file basis.

That is how I started initially (see the commit history), but the the symbol resolution was messy as well as symbol visibility in imports. The benefit of having the pragma is being able to mix existing libraries/code.

[stackenbotten]

There was an error when running chk_coding_style for commit 0b5bc782676ce79807a9d9b12322596cc3ffbc84:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit e36e413c638610f5f4633a0891adc98b660b6fbf:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit 946d5eb8e0eae507123b43846a9cec1738382eb5:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit e53ea2b23315ab70e0f8e3996a0781884da07fd4:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[stackenbotten]

There was an error when running chk_coding_style for commit b56ec71dd249e6a62bc52714715768ee64484ef8:

Coding style error:
libsolidity/interface/CompilerStack.cpp:80:#include "solstdlib.h"

Please check that your changes are working as intended.

[github-actions[bot]]

This pull request is stale because it has been open for 14 days with no activity. It will be closed in 7 days unless the stale label is removed.

[ekpyron]

Removing the stale label here - but @nikola-matic, you wanted to check and report the state of this, s.t. we can decide all remaining open questions, if any (I guess it's mainly stuff like the name used in importing and such?), and actually move this forward, right?

[For the record: this only occurs in the roadmap as a "future" roadmap item - but that is supposed to mean to be finished with this - this PR is only the very first small step towards moving everything that can be moved and the entire thing will both take a lot of time and require generics - but that doesn't mean that we shouldn't still start and do what we can do "now" or at least soonish - no immediate hurry, but we also shouldn't just keep this open and go to waste forever]

[github-actions[bot]]

This pull request is stale because it has been open for 14 days with no activity. It will be closed in 7 days unless the stale label is removed.

[nikola-matic]

Removing the stale label here - but @nikola-matic, you wanted to check and report the state of this, s.t. we can decide all remaining open questions, if any (I guess it's mainly stuff like the name used in importing and such?), and actually move this forward, right?

[For the record: this only occurs in the roadmap as a "future" roadmap item - but that is supposed to mean to be finished with this - this PR is only the very first small step towards moving everything that can be moved and the entire thing will both take a lot of time and require generics - but that doesn't mean that we shouldn't still start and do what we can do "now" or at least soonish - no immediate hurry, but we also shouldn't just keep this open and go to waste forever]

We can add the roadmap label to this PR, so it doesn't get the bot's attention, right @r0qs?

[r0qs]

~Yes, or even use must have or must have eventually with prevent the bot to mark it as stale.~ Just a small correction since must have is only used for issues and not PRs. So just use roadmap as you said.

[cameel]

Decision from the call: we'd prefer a special syntax for the stdlib imports:

import std.cryprography;

[nikola-matic]

Decision from the call: we'd prefer a special syntax for the stdlib imports:

import std.cryprography;

Changes to Parser::parseImportDirective() and the grammar then. Would be nice to crosspost this to the stdlib issue. Might be a bit annoying (haven't looked into it yet) to protect against allowing this syntax without the pragma being enabled.

[cameel]

I did cross-post there already. Small changes in the parser will of course be needed (the stdlib import will use an identifier path) but this will all be behind an experimental pragma so non-breaking.

And I don't think we'll be keeping the ANTLR grammar up to date with the experimental changes. @ekpyron already thinks that even trying to keep it up to date as is is too much of a distraction.

[nikola-matic]

I did cross-post there already. Small changes in the parser will of course be needed (the stdlib import will use an identifier path) but this will all be behind an experimental pragma so non-breaking.

And I don't think we'll be keeping the ANTLR grammar up to date with the experimental changes. @ekpyron already thinks that even trying to keep it up to date as is is too much of a distraction.

Ah, so we're changin the pragma directive as well. Should this be implemented separately as a prerequisite for stdlib, and then this PR updated respectively?

[cameel]

I think separately, because we want it done and merged in the very near future. It should not be held back by discussions of other changes in this PR.

[cameel]

@ekpyron said on the call that anyone not busy with other tasks could start working on that already. We'll discuss further steps next week.

[ekpyron]

Yeah, we won't maintain an antlr grammar for the experimental version of the language - we can do that once we have stabilized it :-).

And yes, this PR will be rebased on top of a generic "pragma experimental solidity-next;" PR, which we'll have soon (after that we can drop the pragma stdlib here) - and that experimental pragma will work in such a way that it can enable an experimental parser mode that can then be permissive in the import directive and allow this new style.

@ekpyron said on the call that anyone not busy with other tasks could start working on that already. We'll discuss further steps next week.

"That" being the pragma experimental solidity-next; PR first - and once that's done rebasing this PR on top of that, if there's even more time to spare.

[ekpyron]

Closing in favour of https://github.com/ethereum/solidity/pull/14249