Reference types design

[CodeSandwich]

Abstract

Solidity references have limited capabilities and are sometimes tricky to use. I believe that their design needs to be improved to make the language more powerful, easier to learn and harder to write bugs in.

I'm raising this issue because it seems widely recognized when talking to Solidity users, but I couldn't find any issue or even a discussion addressing it. Here are a few pain points I've been personally wrestling with, but I'm sure that there are many more cases like this.

No primitive types memory references

It's impossible to create a reference to memory containing a primitive type:

function foo() public pure {
    bool memory a; // Illegal
}

The primitive must be wrapped in a structure or an array to do that.

No storage of references

It's impossible to store any reference anywhere except a local variable:

MyStruct public a;
MyStruct storage public b; // Illegal

function foo(Mystruct calldata c) public {
    b = a; // Illegal
    MyStruct memory storage d = a; // Illegal
    MyStruct memory calldata e = c; // Illegal
}

No reference comparisons

It's impossible to compare two references and check if they point at the same location:

function foo(MyStruct memory a, MyStruct memory b) public pure {
    require(a != b); // Illegal
}

It's also not possible to easily compare the referenced values.

No copying between memory locations

It's impossible to directly copy data between two memory locations:

function foo(MyStruct memory a, MyStruct memory b) public view {
    a = b; // Does not copy
    a = MyStruct(b.x, b.y, b.z); // Workaround
}

Confusing assignment rules

The assignments are tricky, depending on exact types they may create copies of data or references. The reviewer of code must know the context of an assignment and remember the rules to understand what's happening and spot potential bugs or inefficiencies:

function foo() public {
    ...
    a = b; // Does it copy or create a reference?
    ...
}

Wrap-up

Many of these pain points compose into larger problems when coding, e.g. lack of in-memory references and lack of primitives references. Some problems, like lack of memory array slices, may or may not be solvable with a clever references design, but I've intentionally omitted them not to derail any potential conversation into the gray area. All limitations I've described here are often worked around using inline assembly, which is inelegant, dangerous and produces fragile code.

Motivation, Specification, Backwards Compatibility

In this sections I'm supposed to propose a solution, but unfortunately I can't. I don't have the knowledge or experience around programming languages design and I'm not even familiar with Solidity source code, as of now all I can do is flag the problem.

I'm afraid that another evolutionary tweak to the references design can't solve these issues, probably a deeper reconsideration and a holistic approach are needed. Maybe existing languages could be used as an established template, especially lower-level ones?

Backwards compatibility is a big problem with deep changes like this. Maybe two designs could coexist, e.g. a project could use the new one but its dependencies could still be written using the old one?

[gpersoon]

Here are a few examples that are not obvious:

// SPDX-License-Identifier: MIT
pragma solidity 0.8.16;
import "hardhat/console.sol"; 

contract ReturnViaMemoryParameter {
  struct Book { 
      uint id;
   }
   function testmem(Book memory x) public pure {
       x.id += 1;
   }
   Book y1;
   constructor () {
       Book memory y2;
       testmem(y1);console.log(y1.id); // 0
       testmem(y2);console.log(y2.id); // 1
   }
}

// SPDX-License-Identifier: MIT
pragma solidity 0.8.16;
import "hardhat/console.sol"; 

contract MemoryAsPointer {
   struct Book { 
      uint id;
   }
   constructor () {
       Book memory y1;
       Book memory y2=y1; 
       y2.id=7;
       console.log(y1.id,y2.id); // 7,7
   }
}

Suggestion

Have a keyword that indicates if values are copied/passed by value or by reference

[ekpyron]

This general complex of issues is known and we plan to change this in the future one way or another. Unfortunately, it's not entirely trivial to come up with a full schema that is both sufficiently flexible and at the same time fool-proof... especially while not inadvertently overcomplicating the language. So yes, the current situation is definitely bad, but we don't yet have consensus about the best design forward.

[nishant-sachdeva]

@ekpyron Should we have a discussion in the next meet if we want to come together and make a proper roadmap on how to go about this ?

[NunoFilipeSantos]

This is a topic better suited for our Forum. Closing for now.