Java Bindings: Generate classes for contract keys

[StephenOTT]

Dealing with Contract Keys in a typed manner is very difficult at the moment:

A contract key can be a variety of different types (anything Value.class returns). This leads to the actual java type being a simple String, a Daml Tuple.class or full classes generated by codegen.

This leads to complex type handling on a per Template basis having to understand what Value will be returned and handle it.

It would be much easier if ContractKey classes were always generated instead of the "sometimes yes and sometimes no".

What is the solution you would propose?

as described here: https://discuss.daml.com/t/input-limit-on-template-key/3029/3?u=stephenott there are multiple ways a template key could be established in the daml template.

It would be nice if the codegen generated a contract key class for each Template's contract key so we can work in a typed manner that is closer to the template instead of working with the complex variation that is returned by the DamlRecord.

Even if a contract key was a single field such as a string it would still be wrapped into the generated class representing the ContractKey for the specific template.

Further this would simplify dealing with TransactionTree results where the contractKey is being returned outside of the contract class that was generated (because contractKey is a field in CreatedEvent vs where a Template's inner "Contract" class has a typed contractKey field)

[stefanobaghino-da]

It would be much easier if ContractKey classes were always generated instead of the "sometimes yes and sometimes no".

Can you clarify what you mean by "sometimes yes and sometimes no"? If you mean that we should generate a type specific for the key of a template if the underlying key is a tuple, I'm not sure this is the right path to follow. I would make sure that if your contract key is a tuple, you get that tuple in your Java code as well, without making a special case. At the same time, if you defined a specific type for your key, this should be reflected in the generated type, without any wrapper.

What can definitely be improved is to facilitate the decoding of contract keys by making sure that the type of contract key for that template doesn't have to be known out-of-band. @S11001001 thoughts?

[StephenOTT]

@stefanobaghino-da some examples of Contract Key classes : Sometimes yes, and sometimes No:

Example 1

sample from: https://discuss.daml.com/t/input-limit-on-template-key/3029/3?u=stephenott

data ExampleKey = ExampleKey with
  party1 : Party
  party2 : Party
  party3 : Party
  id1 : Text
  id2 : Text
  id3 : Text

template Example
  with
    party1: Party
    party2: Party
    party3: Party
    id1: Text
    id2: Text
    id3: Text
  where
    signatory party1

    key ExampleKey party1 party2 party3 id1 id2 id3 : ExampleKey
    maintainer key.party1

ExampleKey would get generated as a full blown custom class.

---

Example 2

type ExampleKey = (Party, Party)

template Example
  with
    party1: Party
    party2: Party
    party3: Party
    id1: Text
    id2: Text
    id3: Text
  where
    signatory party1

    key (party1, party2) : ExampleKey
    maintainer key._1

ExampleKey would get generated as a da.types.Tuple2.class (custom Daml Java bindings class) and then it is wrapped as:

Is a key required or optional?

---

Example 3

type ExampleKey = (Party)

template Example
  with
    party1: Party
    party2: Party
    party3: Party
    id1: Text
    id2: Text
    id3: Text
  where
    signatory party1

    key party1 : ExampleKey
    maintainer key

ExampleKey would get generated as a String and then it is wrapped as:

---

Example 4

Again pulling from the discuss.daml forum post:

template Example
  with
    party1: Party
    party2: Party
    party3: Party
    id1: Text
    id2: Text
    id3: Text
  where
    signatory party1

    key ((party1, party2, party3), (id1, id2, id3)) : ((Party, Party, Party), (Text, Text, Text))
    maintainer key._1._1

This would generate nested Tuples3s in a Tuple 2.

So when working with the types, we are now mixed between multiple types, none of which have a common interface to define it as a contract key and it is a wide spectrum of polymorphic de/serialization support to manage.

Contractkey classes would provide a wrapper and a common interface to manage that provides expected results to navigate the lists of objects and manage de/serialization. It is similar logic to why create an Identifier class if you could create a Tuple3. The named types specific to the objects needs provide expected structure.

recap: If everything is of Type ContractKey then managing the transformations and movement of that data becomes easier compared to a wide range of polymorphic structures. The "Tuple" may still be within the Contractkey class, but the interface on that class allows developers to manage expectations through "if instance of ContractKey then do x or y, etc etc"

[StephenOTT]

@stefanobaghino-da there is actually a similar impl already in place in the java bindings: ContractId.class

  public static final class ContractId extends com.daml.ledger.javaapi.data.codegen.ContractId<SomeTemplate> {
    public ContractId(String contractId) {
      super(contractId);
    }
}
...

  public static class Contract implements com.daml.ledger.javaapi.data.Contract {
    public final ContractId id;
...
}

The ContractId class is currently being used to provide access to the exercise commands. Vs for exercideByKey commands they have been added into the root of Template class.

Example the methods

exerciseArchive(..) is within the ContractId class (which implements a common ContractId<T> interface, and exerciseByKeyArchive(...) is found i the Template data class

[S11001001]

What can definitely be improved is to facilitate the decoding of contract keys by making sure that the type of contract key for that template doesn't have to be known out-of-band.

@stefanobaghino-da The proper place for that idea in the Java codegen is the ContractCompanion.WithKey.

For example, if you define the fields { K key; ContractCompanion.WithKey<..., K> tpl; } in the scope of the type variable K, you have a single key and evidence that K is a key type, and tpl contains various details about that template. And of course you don't just have to have a single K, you can have a List<K>, or a K => WhatsIt, or a Future<K>, or a Sink<K, ...>, &c. The other type parameters on WithKey represent the contract class, payload class, and contract ID class, which can be constrained according to the goals you have with the ContractCompanion.

Here's an example using what is already available in the companion:

https://github.com/digital-asset/daml/blob/ed82926204c35ae3791d4672a74bfacdb8231963/language-support/java/codegen/src/ledger-tests/scala/com/digitalasset/TestUtil.scala#L147-L150

The companions are deliberately minimal, but any function over those types that we want to relate any of them can be added to the companion. For example, it already carries a Value => K function (decoder function) as a private implementation detail, which we could publish; we could similarly add the inverse function.

I encourage a conservative approach to what we add to this interface, of course. Since we have all four of these types of interest at hand, the companion can incorporate pretty much any function that relates those types to each other or to the Java bindings library in a sensible way.

If decoder features are of specific interest, I would also suggest including a prototype of #14313. We don't have to describe everything with decoder types right away, but this is the direction that gives us the most flexibility for abstract-with-codegen development in the future.

[S11001001]

The ContractId class is currently being used to provide access to the exercise commands. Vs for exercideByKey commands they have been added into the root of Template class.

@StephenOTT Regarding exercise specifically, this has been significantly reworked in the forthcoming Daml 2.3.x; see the changelog in #14037, and note how Exercises relates to ContractId and ByKey in the updated documentation.

[StephenOTT]

@S11001001 what does the link https://docs.daml.com/2.3.0-snapshot.20220707.10108.0.f4098846/app-dev/bindings-java/codegen.html#id7 indicate? Nothing is popping out at me.

[S11001001]

@StephenOTT It's a sample of the Java codegen output. If you prefer, when 2.3.1 is available shortly, update your Java codegen build to it and take a look at the new output.

[StephenOTT]

@S11001001 👍 yes i see the deprecated exercise by key and the new ByKey class. Would be nice to have byId(...) as well so we can just type MyTemplate.by in the ide and get the needed creator method.

[S11001001]

The companions are deliberately minimal, but any function over those types that we want to relate any of them can be added to the companion. For example, it already carries a Value => K function (decoder function) as a private implementation detail, which we could publish; we could similarly add the inverse function.

Some other proposals for inclusion in ContractCompanion.WithKey that came up in discussion:

1. Key -> ByKey. ByKey would need another type parameter on WithKey. It is perhaps of interest that Id, which is already on WithKey, and ByKey have as a LUB T.Exercises where T is the template associated with the companion.
2. ValueDecoder<Key> rather than the function
3. as mentioned, the Key -> Value encoder

We also discussed factoring key-relevant functions to a superinterface of WithKey so that users could do less wildcarding of irrelevant tparams like Ct and Data.

[StephenOTT]

@S11001001 is there going to be a Daml to Tree/AST/JSON type format that converts the Daml file(s) into some sort of navigable tree? It would be very nice if we could just have a AST-like format so the community can navigate and roll-your-own as needed for functionality. The bindings libraries seem to be getting ever more complex with intricate generics and conversions. Having an AST would have solved this issue from the start as we could use the ast and our choice of class generation.

[S11001001]

@StephenOTT If you are interested in reading LF (the compiled format of Daml) and navigating it, please see this forum comment (note that interface library is now called api-type-signature to avoid confusion with Daml interfaces).

The caveats about all those possibilities mentioned in that discussion still apply -- api-type-signature is an internal library and not subject to any stability guarantees -- but, for what it's worth, Java codegen itself uses the the api-type-signature library to load an input AST from LF.

I recommend directing further discussion along the lines of parsing LF to the forum; there are a few threads such as the above-linked with similar queries that may be informative as well.