Assembly language feature: add new levels for Yul

[pcaversaccio]

Abstract

Add new levels for Yul: Yul+ being a higher-level version and Yul- being a lower-level version.

Motivation

The current version of Yul can be viewed as a pretty high-level assembly language since it provides e.g. no access to stack and control flow instructions. I open this issue to initiate a discussion on adding potential new levels to the current version of Yul. The idea started with my Twitter thread here and feedback by @leonardoalt. The current idea is to add new levels of Yul: Yul+ being a higher-level version and Yul- being a lower-level version with more compilation stack layers. Building something similar to LLVM IR would be really nice.

[axic]

In fact we already did this 2-3 years ago, when we discussed "Yul++" (see this hackmd) upon prompt by @SilentCicero, which eventually turned into yulp.

What other Yul+ do you have in mind?

Regarding the lower-level we did had similar discussions in mind over the years, last when the new stack allocator was created and it turned unneeded.

[pcaversaccio]

Thanks, I'm aware of yulp which is however not maintained by the Solidity team if I understand correctly and there is also no active development going on (the repo is even archived). So in terms of design, I must admit I actually like yulp and could be a good starting point for Yul+.

Regarding the low-level version I have the following points to mention:

• Usually, an assembly language provides access to stack and control flow instructions, Yul does currently not.
• IMO to make Solidity/Yul more competitive with Vyper. One reason why Vyper can outperform seemingly well-optimized Yul is that Yul is still a high-level language and the Yul compiler still has to do stack scheduling, control flow analysis etc. Allowing for direct control flow via JUMPs and access to stack via e.g. SWAP or DUP could help to write even more efficient contracts.

[Vectorized]

I’m in for more access to more lower level stuff.

Unless switch statements can be auto optimized to be as efficient as jump tables.

I’ve written a Yul based sort (https://gist.github.com/Vectorized/7b3a1fff3832bad126fdcba0ae785275) that costs 2x more gas compared to the 1st and 2nd place entries of 2018’s Solidity golfing contest (the top 2 entires abuse jump tables extensively to implement sorting networks efficiently).

[devtooligan]

I support giving devs access to lower level functionality specifically access to stack and JUMPs.

[nvnx7]

I also favor a Yul+/Yul++ variant. This would be immensely beneficial for writing code where a manual high optimization is needed (e.g. in cryptography) that neither Solidity nor current Yul can provide. weierstrudel by AztecProtocol is a good example - they had to write huff lang for weierstrudel implementation to achieve that kind of optimization. Like @pcaversaccio said - I think it'd be nice to have another Yul version (Yul+/Yul++) that has access to stack-related opcodes (PUSH, DUP, SWAP etc) along with what Yul already has.

[0xalpharush]

With regards to a higher level intermediate representation, I think it would be ideal to go beyond Yul++ and use something that does not access memory/ storage directly a la slithIR. This is more conducive to analysis/ optimizations that are prohibitive or difficult to perform on Yul.

More broadly, I'm in favor of moving towards a multi-level representation that is progressively lowered. Ideally, this would be very modular and allow third-parties to write optimization passes or do source-to-source translation with greater ease.

[0age]

+1, Solidity needs a mechanism to rip out all the guard rails when necessary or the incentive to eject entirely becomes too great when maximal efficiency is required.

[ControlCplusControlV]

I think Yul++ makes the most sense name wise for this extension as Yul+ already exists although archived Yul++ would differentiate it from that (For example I have a Yul+ <-> Foundry repo which would populate when people are searching for this Yul extension). I do favor adding this though, especially as features like #9889 and many from Yul+ like mstructs would help make it easier to write Yul in a way that is easy to type and efficient to run while lower level controls help occupy the niche languages like Huff are moving into fill. I don't think lower levels should be differentiated with a name or different compiler like Yul-, rather adding unsafe blocks in a Yul extension, so that high level abstractions can be used where they make sense (so something like an mstruct), while compute intensive areas the guard-rails can be taken off to go faster (or the entire program can be in an unsafe block if wanted)

[krasi-georgiev]

+1 for allowing jumps and labels

I am trying to reverse a contract based on some pseudo code from the ethervm decompiler. It generated the code with goto and labels so without jumps in yul can't think of any other to reverse a contract from.

[devtooligan]

One feature that would be incredibly impactful would be the ability to insert pure opcodes/mneumonics. This would give the ultimate flexibility and control and many other feature requests could be solved by this one feature. Furthermore, this change would allow for interoperability with Huff or any other language or tool that compiles to bytecode.

As for the actual implementation, one idea would be that opcodes can only be inserted inside Yul functions, or perhaps there's a special type of function just for this where the args get put on the stack and the return values are what gets added to the stack.

function mySolidityFunc() {
  assembly {
    opcodes myOpcodeFunc(arg1, arg2) => ret {
          DUP1     // [arg1, arg1, arg2]
          MUL      //  [arg1^2,  arg2]
          SUB      //  [arg1^2 - arg2]
    }
   let ans := myOpcodeFunc(2, 1)
}

[leonardoalt]

@devtooligan how is that different from a normal Yul function? You already have access to mul and sub, the only missing thing from your example is stack access which is rather a different feature.

[devtooligan]

In my mind it's about being able to paste in bytecode. Can be hundreds sloc. Don't need or want to rewrite in Yul if maintaining it with Huff.

You don't like the idea, @leonardoalt? What are your concerns?

[leonardoalt]

Most of the Yul built-in functions translate directly into the opcodes, so I feel like it's just redundant.

[devtooligan]

Right now Huff can only deploy whole contracts. Would be aweseome if we could find a way for Huff to interact w Solidity.

[leonardoalt]

Right now Huff can only deploy whole contracts. Would be aweseome if we could find a way for Huff to interact w Solidity.

I guess you could use Yul's verbatim for that?

[pcaversaccio]

I think it's about interoperability on the opcode level here. The same would apply to Fe probably. The question we need to ask ourselves is whether we consider this an isolated Solidity/Yul issue or we think about transpilation/interoperability. IMHO when Huff becomes more mature you will work only with Huff and not consider it porting it to Solidity/Yul. So maybe such a feature request is a lifecycle problem. Also, as pointed out by @leonardoalt above, it's already possible to create bytecode sequences that will not be modified by the optimizer.

[devtooligan]

@leonardoalt Thanks this looks promising 🫡

[leonardoalt]

@leonardoalt Thanks this looks promising 🫡

It looks like the exact thing you need for your initial comment haha

[krasi-georgiev]

What about jumps and labels?

[devtooligan]

@leonardoalt Thanks this looks promising 🫡

It looks like the exact thing you need for your initial comment haha

Ser my initial research has shown that verbatim is not allowed in Solidity.

Found this related issue: https://github.com/ethereum/solidity/issues/12067

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 90 days. It will be automatically closed in 7 days.

[pcaversaccio]

This is still relevant. Please don't close.

[pcaversaccio]

@cameel could you please remove the stale label? I'm concerned that otherwise this issue will be closed.

[cameel]

Sure, but don't worry, the bot would remove the label anyway, it just runs only once per day. So the issue would not be closed since you commented on it.

EDIT: Ah, I see it even already did.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 90 days. It will be automatically closed in 7 days.

[pcaversaccio]

This is still relevant. Please don't close.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 90 days. It will be automatically closed in 7 days.

[pcaversaccio]

This is still relevant. Please don't close.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 90 days. It will be automatically closed in 7 days.

[pcaversaccio]

This is still relevant. Don't close.