dapple libraries design issues

[mhhf]

Since delegatecall is now available libraries should become First-class citizen of packages. Here we should discuss on how dapple treats libraries A few thoughts:

dappfile - analog to objects:

environments:
  morden:
    objects:
      auth_factory1:
        class: DSAuthFactory
        address: '0x7639a37b8ecbd68e15b8e173897c6c0dea462452'
    libraries:
      Math:  // <- this name is unnecessary
        class: Math
        address: '0x0123456789012345678901234567890123456789'

search

• addresses of libraries are searched recursively beginning from the current dappfile through every dependency (Level-order)
• a warning can be thrown if several different libraries with the same class name are found in one level.

test

• libraries must be present (imported) and gets redeployed and linked on every test

build

• if no library is found, code is build with solc native Library_ substitution

run

Library Math and String can be deployed:

...
new Math(); // internally Math: 0x1234
// or
var math = new Math() // note here that a new Math contract is NOT deployed but the address of the know is returned
var c1 = MathUser() // a contract which uses math is deployed, no address needs to get passed
var c2 = StringUser() // throws an error, because String library is not deployed or found yet.

Thoughts? Did I miss something?

[nmushegian]

Rather than overloading the new keyword, I think importing libraries should overload importing constants, which is how we describe non-library singleton contracts.

import maker;
import std;
var registry = maker.registry;
var string = std.String;
var eq = string.compare(...)

[nmushegian]

Does this mean that we will be recompiling in between steps, since a library can be made available part way through execution? This is the direction we were headed with constant macros, which was the library-like workaround ryan and I were working on.

On the other hand, if we make a rule that each script is only something that can be done in between builds, meaning all libraries have to exist at compile time, then we could do more interesting static analysis on the scripts

[mhhf]

Rather than overloading the new keyword, I think importing libraries should overload importing constants, which is how we describe non-library singleton contracts.

Consider the case: let Math be a library:

var a = new Math();
var b = new Math();

should a==b or a!=b. Since Math is a library and a and b have the same binary code my line of thought was that new should be intelligent in the sense that it detects a redeploy of a singleton library and prevents this by linking b against a.

import std;
var a = new Math();
export Math;

if a == std.Math a should be linked as well.

In the case that Math has changed, a should be deployed in the normal sense and exported to the dappfile.

Does this mean that we will be recompiling in between steps

no recompilation necessary: if a library is not available during compile time (we have to think about if it ever will) solc returns a placeholder __Math____ which will be substituted on deploy if the library is available or throw an error otherwise.

[nmushegian]

no recompilation necessary: if a library is not available during compile time (we have to think about if it ever will) solc returns a placeholder Math__ which will be substituted on deploy if the library is available or throw an error otherwise.

Right, there's some intermediate state that's not quite deployable and also depends on a specific script's execution. This also means that our DSL no longer maps directly to sequences of statements that can be executed as either messages or transactions (unless we put a linker into the interpreter contract =]]]]) so maybe there is something to the idea of requiring a script to have a complete library environment for context (this can be check statically)

[mhhf]

unless we put a linker into the interpreter contract =]]]]

If i understand linker correctly, this is easy todo with a in memory function:

Bytes.substitude(bytes memory source, uint from, uint to, bytes what)

and

Bytes.indexOf(bytes source, bytes search) returns (uint position)

used in

Contract.link(bytes memory source, bytes libName, address target)

so maybe there is something to the idea of requiring a script to have a complete library environment for context (this can be check statically)

this can be done on top of the DappleScript type checker, once it is available

[mhhf]

Do we want to support multiple libraries with the same name during the development process? e.g. Math provided and included by two different packages with different functionality

[nmushegian]

I'm not sure, there is no language-level support for namespaces so contract names will collide if you try to use two different versions of "the same" contract from one file. What I'm saying is the solution should look the same for libraries and contracts, and I'm not sure there's an easy way to do it without a solidity upgrade or a preprocessor of some kind

[mhhf]

you could think of the following scenario. package A and B provide different Libraries named Math. You have two contract files a.sol and b.sol. a.sol imports and links against A.Math and b.sol imports and links against B.Math, so from solidity point of view, everything works fine. However this would be not supported with the current setup. Should we support this case?

[nmushegian]

Yeah, we should. Can't think be handled the same way contract type names are handled? @ryepdx can advise

[ryepdx]

~~Yeah, there should be no problem with handling them roughly the same way contract type names are handled. Basically just aliasing them to content hashes and doing regex replaces for every use of the original name as a library.~~

Edit: After talking with @mhhf in Slack, I retract the above statement. Basically we're going to have to satisfy ourselves with naively assuming identical library names refer to the same library until we can get chain forking in place.

[mhhf]

After talking to @ryepdx we decided to put multiple names and complex linking of libraries on hold right now and pick this task up with the chain forking task as complex refactoring to the pipelines have to be done.

[rainbreak]

Not sure where you are with this atm - guess it is mixed up with the chain forking work. Anyway, just noting that my current workaround for using dapple test when there are libraries in subpackages is dapple test --subpackages

The subpackage libraries don't get built otherwise as they aren't in classes.json.