New faces for old interfaces

[Night-walker]

"HA-HA-HA-HA" -- Fantomas

Imagine a facility aimed to be defined for broad range of data types. One which is meant to be adaptable to new scenarios. Like serialization. Plenty of things can be serialized, you can't possibly define the complete list of them in advance.

Now take the serialization facilities in serializer, json, xml -- all of them are basically only aware of primitive types. What about time::DateTime, os::Path, re::Regex or BigInt? Obviously you can manually handle their serialization should the need appear. But that's merely a workaround -- the mountain approaches Mahomet, though we know it should be the other way around. That's how the manual way looks like:

routine processStuff(stuff: list<A|B|C|...>){ # may be a map, a tree-like structure, anything.
    for (item in stuff)
        switch (item) type { # handle each type separately
        case A: handleA(item); 
        case B: handleB(item);
        case C: handleC(item);
        ...
        }
}

And each time you face similar task, this boilerplate code will emerge over and over, whereas the solution should actually resemble the following:

interface I { # which supposedly generalizes over A, B, C, ... 
    routine handle()
}

routine processStuff(stuff: list<A|B|C|...>){
    for (item in stuff)
        item.handle() # assuming I is defined for A, B, C, ...
}

Pretty evident that the latter variant, so seductive and desirable, is an unattainable dream unless the interface is actually implemented for all concerned classes beforehand. Which is so cruelly untrue for our case with wrapped types which happen to have no notion about serialization.

That is, there is currently no way to express a universal solution here, one which would encompass all similar cases. And that's sad. After all, code reuse is one of the most pursued goals in the programming. The Holy Grail of OOP, no less.

Well, it is of course possible to just add built-in serialization support for any of those types. Isn't that the simplest way to deal with the problem? Not exactly, as it

• requires you to modify existing source (and what if it's not yours to tackle with?)
• bloats the type with stuff which doesn't really belong there (dead weight most of the time)
• makes the related module's interface dependent on how serialization is done in some other module (incurring 'cascade changes' in the API)

Don't forget that serialization is just an example here -- plenty of tasks can potentially benefit from being defined on interfaces instead of strict sets of types.

The ~~salvation~~ solution actually exists and lies within our reach.

Continuing with the serialization subject, let's assume we're dealing with JSON. First, we gonna need an interface, the one to rule them all:

interface Serializable { # in json namespace
    invar routine serialize() => json::Data
}

What do we need to appease the gods and grant e.g. time::DateTime support of json::Serializable? A separate implementation of this interface for DateTime at the place where we actually require it, in the spirit of Rust:

implement json::Serializable for time::DateTime {
    # 'selfless' notation is also an option
    routine serialize(invar self: time::DateTime) => json::Data {
        self.value()
    }
}

Note that this is not supposed to change existing time::DateTime type -- serialize() is not to be injected into it like extension method or so. Implementation here just defines the way how existing type can be 'mapped' to interface. Should you cast a DateTime to Serializable, serialize() from the implementation will come into play as if it was DateTime method.

And that's not the end (but know that the end is nigh -- just few paragraphs remain). The way interface implementations are defined makes it possible to use them for any type including built-in primitive types. If that doesn't sound convincing, imagine that Dao suddenly acquires unified type system. Almost as in Scala or Rust. No more need for exhaustive variant types coupled with switch type when you can express what you need with a single interface. And then, afterwards, bind whatever you wish to that interface. At any point, without affecting the existing source.

This feature does make things substantially more complicated, I know. We already have lots of stuff in the language, and this one looks like a major feature. So I wouldn't claim that it is undoubtedly a good idea to support what might turn out to be yet another fancy trinket to toy with when one feels bored.

But I think it is worth to pay some attention to this idea. Interface implementations promise capabilities unachievable by the use of more traditional OOP idioms (at least not without potent doping). In Rust, they bring unparalleled flexibility to the language. All in all, seems like an interesting topic to discuss.

[dumblob]

This is unfortunately new stuff to me (I'm feeling ashamed I didn't study Rust so deeply :cry:). It looks really awesome, but first questions which interest me, an old performance-concerned nazi, are:

1. how much will this feature shrink (e.g. reduce structure size) the already existing types (especially primitive ones) - if not much, then I'm even more interested in the performance hit in the question No. 2

2. how much will the interface matching and routine overloading (compared to explicit swich-case statements) affect run-time performance; at first glance it seems to me, that interface matching could be done solely in compile-time and compiled very efficiently, but there might be some caveat I can't see right now

[daokoder]

This idea sounds interesting. The implementation might even be simpler than you thought. But I am concerned with introducing a new keyword just for this. I wonder if the following could be a reasonable alternative without introducing confusions:

interface json::Serializable for time::DateTime {
    # 'selfless' notation is also an option
    routine serialize(invar self: time::DateTime) => json::Data {
        self.value()
    }
}

So interface without for is abstract interface, and with for it become concrete interface for a target type. A concrete interface can be handled like a wrapper class for the target type. Casting the target type to the interface type will simply create a wrapper object for the target object. This way, the implementation should be reasonably simple.

[dumblob]

A concrete interface can be handled like a wrapper class for the target type. Casting the target type to the interface type will simply create a wrapper object for the target object.

Wrapper class sounds like mixin. How does it work with wrapper objects in Dao? Does creating of such object involve malloc() and copying of the original existing object (which is going to be wrapped) to the new bigger space?

[daokoder]

Wrapper class sounds like mixin.

Not mixin, just plain wrapper.

How does it work with wrapper objects in Dao?

It will not mess with those wrapper objects.

Does creating of such object involve malloc()

Yes, but it can be optimized to reduce allocations.

copying of the original existing object (which is going to be wrapped) to the new bigger space

All objects should be copied by pointers.

[Night-walker]

So interface without for is abstract interface, and with for it become concrete interface for a target type.

Clever workaround :) Seems OK for me.

Casting the target type to the interface type will simply create a wrapper object for the target object. This way, the implementation should be reasonably simple.

Indeed, interface implementation actually is a wrapper in its essence. Since there is room for optimization, sounds good.

[Night-walker]

2. how much will the interface matching and routine overloading (compared to explicit swich-case statements) affect run-time performance;

One can use both at the same time anyway: hard-code the handling of types known in advance with variant typing, while defining an interface to cover the rest. You are not obliged to use interface for all kinds of types, variant typing has its benefits too.

Also note that without interface implementations you have to use manual wrappers anyway -- for everything which does not conform to the interface used. And automatic wrappers here may potentially be optimized to run faster then the manual ones.

[dumblob]

Doubts vanished, let's do it.

[daokoder]

More or less done:)

[Night-walker]

Great :) I reserve some more time on this issue so that we don't miss anything, so I'll keep it opened for a while.

[daokoder]

Right. In the mean time, we should do more tests on this new feature.

[dumblob]

It seems, that currently the concrete interface implementation can be only inside of interface body. This is though inconsistent with classes as discussed in https://github.com/daokoder/dao/issues/401 . So this is currently not possible:

interface I1 {
  routine r1() => string
}
interface I1 for int {
}
routine I1<int>::r1() {
  return 'r1 ' + (string)self
}
x = (I1)5
io.writeln( std.about(x.r1()), x )

There appears to be some bugs with reference counting (caused by the code above):

[[ERROR]] in file "/home/test/del/test.dao":
  At line 4 : Invalid interface definition --- " interface I1 for int { } routine ... ";
  At line 4 : Incomplete concrete interface implementation --- " I1<int> ";
Warning: namespace/module "io" is not collected with reference count 17!
Warning: namespace/module "/home/test/del/test.dao" is not collected with reference count 4!
Warning: namespace/module "mt" is not collected with reference count 26!
Warning: namespace/module "std" is not collected with reference count 26!
Warning: namespace/module "dao" is not collected with reference count 16!

I have also a question about concrete interfaces behavior demonstrated on the following tests.

@[test()]
var k = (InterB) 0x7ffeff;
var h = (InterA) k;
io.writeln( std.about(h), h )
@[test()]
@[test()]
{{InterB<int>}}
@[test()]

I'd expect the output to be {{InterA<int>}} abc8388351 (because value of k should have been InterB<int> which has the serialize() method inherited from the InterA interface).

@[test()]
var k = (InterB) 0x7ffeff;
var m: InterA = k;
io.writeln( std.about(m), m )
@[test()]
@[test()]
{{InterB<int>}}
@[test()]

Same here, I'd expect {{InterA<int>}} abc8388351 instead of {{InterB<int>}}.

Am I misunderstanding something or is it a bug?

[Night-walker]

It seems, that currently the concrete interface implementation can be only inside of interface body.

Since the interface's methods are supposedly known at the point where concrete interface is declared, it seems unnecessary to encourage their separation.

I'd expect the output to be {{InterA}} abc8388351 (because value of k should have been InterB which has the serialize() method inherited from the InterA interface).

Most likely the inner, derived type (InterB) is extracted and printed here.

[dumblob]

Since the interface's methods are supposedly known at the point where concrete interface is declared, it seems unnecessary to encourage their separation.

Yes, it's sufficient in my opinion, but it still feels somehow inconsistent.

Most likely the inner, derived type (InterB) is extracted and printed here.

Which is certainly not what the declaration var h = (InterA) k; says. But more importantly, the missing serialized variant of the value of h on the output makes me nervous.

[Night-walker]

Which is certainly not what the declaration var h = (InterA) k; says.

Same is printed in case of inherited classes. Should be handy when debugging, no real inconsistency.

But more importantly, the missing serialized variant of the value of h on the output makes me nervous.

That's indeed strange.

[daokoder]

var k = (InterB) 0x7ffeff;
var h = (InterA) k;
var m: InterA = k;

The behaviors of these are actually expected. Note that, when you cast a type X to an abstract interface I, it will check if a concrete interface interface I for X (namely I<X>) exists for X, if yes, the result of the casting will be an object of I<X>. Otherwise, it will check if X is compatible to I, if yes, the result will be X itself, which is the case for h and m.

But more importantly, the missing serialized variant of the value of h on the output makes me nervous.

Given what I described in the previous paragraph, this is not the case here.

[daokoder]

Please note that, currently, the following also produces InterB<int>:

var s = (InterA<int>) k;

The reason is that an instance of InterB<int> has no base instance for its base type InterA<int>, because InterB<int> instance is a simple wrapper of int. And considering that, the essence of matching to interfaces types is the checking of methods, so supporting base instances (like class instance) for concrete interfaces such as InterB<int> may be completely unnecessary.

[Night-walker]

It is still puzzling for me why io.writeln() with two arguments is expected to print a single value.

[daokoder]

It is still puzzling for me why io.writeln() with two arguments is expected to print a single value.

It actually prints two values, but in the test, only the first is checked for correctness. Other tests have guaranteed the correctness of the second. The tests was directly taken from the demo, so I didn't pay much attention to remove unnecessary printing.

[Night-walker]

OK.

[dumblob]

Right, didn't know that only the first value is checked for correctness in tests.

For me the naming is though still puzzling:

interface I1 {
  routine r1() => string
}
interface I1 for int {
  routine r1() { return 'r1 ' + (string)self }
}
interface I2 : I1 {
  routine r2(x: int) => string
}
interface I2 for int : I1<int> {
  routine r2(x: int) => string { return 'r2 ' + (string)(self + x) }
}
a = (I2)5
b = (I1)a
io.writeln( std.about(b), b )
#b.r2(7)  # <-- try to uncomment this

which gives you

I2<int>[0xc4aa00] 5

Note it's talking about I2. But after uncommenting the marked line, one'll get exception about I1:

[[ERROR]] in file "/home/test/del/test.dao":
  At line 0 : Invalid function definition --- " __main__() ";
  At line 16 : Invalid virtual machine instruction --- " GETF:6,4,17 ";
In code snippet:
     16 :  MCALL       :    12 ,     3 ,    16 ;    19;   io. writeln( std. about(...
>>   17 :  GETF        :     6 ,     4 ,    17 ;    20;   b. r2
     18 :  LOAD        :     6 ,     0 ,    18 ;    20;   b. r2
  At line 20 : Member not exist --- " b. r2 for I1 ";

The exception is correct - as I expected, but the type of b returned by std.about() is really confusing. I understand and agree, that supporting base instances is a nonsense, but some solution to this confusing naming is desirable.

[Night-walker]

The exception is correct - as I expected, but the type of b returned by std.about() is really confusing. I understand and agree, that supporting base instances is a nonsense, but some solution to this confusing naming is desirable.

std.about() is not an indication of what nominative type its input is. It digs under its skin and finds out what actually lies there. Quite valuable for debugging purposes.

[dumblob]

std.about() is not an indication of what nominative type its input is. It digs under its skin and finds out what actually lies there.

My mistake.

Is there any reliable way of getting the nominative type name as string?

By the way, does ?< compares operands for partial compatibility of types or "full" compatibility? Using the example above, both io.writeln( b ?< type(I1) ) and io.writeln( b ?< type(I2) ) give me true even though, the interface is of course not the same (i.e. only partial compatibility check is performed).

[Night-walker]

Is there any reliable way of getting the nominative type name as string?

meta.nameOf(meta.typeOf(stuff)), I guess.

By the way, does ?< compares operands for partial compatibility of types or "full" compatibility? Using the example above, both io.writeln( b ?< type(I1) ) and io.writeln( b ?< type(I2) ) give me true even though, the interface is of course not the same (i.e. only partial compatibility check is performed).

I2 inherits I1, so I1 can be viewed as a part of I2 signature -- hence the result of the type checks. Same applies for classes.

[dumblob]

meta.nameOf(meta.typeOf(stuff)), I guess

I'm afraid it doesn't match reality - try e.g.

io.writeln( 'a:',
            meta.nameOf(meta.typeOf(a)),
            'b:',
            meta.nameOf(meta.typeOf(b)) )

with the example above and you'll obtain two times the same type (a: I2<int> b: I2<int>).

[daokoder]

I just changed std.about() to append the nominative type names to the returned strings. So io.writeln( std.about(b) ) will produce I2<int>[0x7fdb78405c60]:I1, when the name after : is the nominative type name.

[Night-walker]

Concrete interfaces don't want to play with C-defined interfaces:

load binary

interface bin::Encodable for array<int> {
    invar routine encode(encoder: bin::Encoder){
        # ...
    }
}

  At line 3 : Invalid interface definition --- " interface bin::Encodable for ar
ray ... ";
  At line 3 : Symbol was defined --- " bin ";

If I use const Encodable = bin::Encodable instead, the error says that Encodable is already defined.

[daokoder]

Finally a real issue related to concrete interfaces. It turns out to be a bug related to interfaces in general. Now fixed.

[Night-walker]

Duplicate concrete interface for the same type should not be allowed.

[Night-walker]

Also, I wonder how concrete interfaces behave with respect to namespaces.