Standardized class model?

[pdesaulniers]

This has been mentioned in https://github.com/teal-language/tl/issues/86#issuecomment-611241592.

I think it's a good idea (although Lua purists might not like it?)

But then, which features should Teal support? Inheritance? Access modifiers? Getter-setter properties? Interfaces?

[hishamhm]

This is a big question!

I think it's a good idea (although Lua purists might not like it?)

I tend to lean towards "good idea" but I also realize that it could turn a lot of people off to the language (and maybe another lot of people on?).

The reason why I think about it is not so much to get the full kitchen sink of OOP features into the language, but for two main reasons:

• the #1 reason to me is to give users a statically-verifiable means for dealing with metatables. By providing something that does the metatable plugging, we avoid homegrown solutions and give control for that to the Teal compiler, which can then make better assumption about the behavior of metatable-infused objects. BUT — I wonder if we could just do that by adding metatable-specifics to record definitions. You can already model methods and attributes with them, so they're almost objects. We just don't have a neat way to do inheritance, to get the OOP baseline down.
• it's my understanding (hope I'm not getting the history backwards — @mascarenhas, I hear you're the TS expert now :) , do you know?) that TypeScript adding classes helped to standardize the JavaScript class model. So perhaps by pushing one model, Teal could help promote "Teal-compatible classes" (not in syntax but in behavior) even in pure Lua code. BUT — that's bluesky wishful thinking and we don't have the strength to make that push at this point. Anyway, I don't think one of Teal's goals should be to push Lua one way or another; it could be just a nice side-effect.

I remember @mascarenhas suggested once adding a class model to Typed Lua (or Titan?) with "pluggable backends" so that they could emit code for different Lua OOP models — while the idea is intriguing I think it is overkill for Teal.

[mascarenhas]

TypeScript did get classes years before ES6 was finalized and classes officially got into JavaScript, but they had the fig leaf that the proposal was already going through standardization, ES6 just took a loooong time. :smile:

I have always been in favor of baking in classes and interfaces in the language definition, and Typed Lua would have them if we kept going; all the alternatives PL people have come up with to model the same kinds of concepts that OO models are more complex. :smile:

I was looking at the docs, does this work, and what does it compile to:

local Point = record
   x: number
   y: number
end

function Point:move(dx: number, dy: number)
   self.x = self.x + dx
   self.y = self.y + dy
end

local p: Point = { x = 100, y = 200 }

Is the move method defined inside the p table, or is it in a separate table and the assignment does a setmetatable behind the scenes?

Also, does the special-casing of casting a table constructor to a record also work for return types? Because if that is the case this gives a way to do constructors without special syntax:

function Point.fromPolar(rho: number, theta: number): Point
  return { x = rho * math.cos(theta), y = rho * math.sin(theta) }
end

Implementation inheritance is overrated, and given that with the standard way of doing inheritance with Lua metatables method resolution is O(length of inheritance chain) I don't believe inheritance was ever popular in Lua. :smile:

[mascarenhas]

By the way, modeling the self parameter in methods as having the type of the record does not scale when you want to do interfaces:

local Plus = record
  e1: Exp
  e2: Exp
end

function Plus:eval()
  return self.e1:eval() + self.e2:eval()
end

local Minus = record
  e1: Exp
  e2: Exp
end

function Minus:eval()
  return self.e1:eval() - self.e2:eval()
end

local Num = record
  val: number
end

function Num:eval()
  return self.val
end

local Exp = Plus | Minus | Num -- what is the type of eval?

The type of eval in Exp is essentially function (self: Exp): number, but if it is just a plain function you can do:

local e: Exp = <something that returns Sum>
local eval = e.eval
eval(<something that returns Num>) -- typechecks but is unsound

In Typed Lua we had a self type and (too) complicated rules around it, in Titan we just made functions and methods different things...

[hishamhm]

I was looking at the docs, does this work, and what does it compile to:

To this:

local Point = {}

function Point:move(dx: number, dy: number)
   self.x = self.x + dx
   self.y = self.y + dy
end

local p = { x = 100, y = 200 }

So for now you need to add the metatable plumbing by hand for the above so that it works. (Right now there is a regression in the code but it's a silly bug which should be fixed quick.)

Also, does the special-casing of casting a table constructor to a record also work for return types?

Yes, it does!

By the way, modeling the self parameter in methods as having the type of the record does not scale when you want to do interfaces

Noted! The above doesn't even work because we can't do T1 | T2 for two different table types (the union is matched at runtime using type() so the compiler restricts the union to one table type max, currently). My plan is to allow T1 | T2 once I add some sort of construct for metatable-backed tables, so we can check against them at runtime.

in Titan we just made functions and methods different things...

Yeah, I'm currently parsing functions and methods ("record functions") as different things (i.e., I chose to not desugar function r:f() into function r.f(r) at parse time), to keep my options open on how to handle these things.

[mascarenhas]

A simple encoding is to use the "class" as the metatable for the instances, and have __index be a circular link:

-- local Point = record ... end compiles to the following two statements
local Point = {} 
Point.__index = Point

-- compiling a method is just type erasure
function Point:move(dx, dy)
   self.x = self.x + dx
   self.y = self.y + dy
end

function Point.new(x: number, y: number)
  -- coercion of a table constructor to Point compiles to a setmetatable call
  return setmetatable({ x = 100, y = 200 }, Point)
end

The wart with this encoding is that the same namespace you use for your methods is polluted with your constructors and with your metamethods, but that is the price to pay for having the generated code be more like what a programmer would write by hand...

I am sure people would complain about the extra cost of record instantiation, but I would just make all records have the reified type as a metatable, as that will give you runtime type checks. Of course use of as <record> would not mess with the metatable.

[hishamhm]

A simple encoding is to use the "class" as the metatable for the instances, and have __index be a circular link:

This is sort of what users do by hand already, but not something I'd plug in every record, of course.

I would just make all records have the reified type as a metatable

I don't think I can because of interop with Lua (existing Lua libraries returns tables which I want to type as records and those don't set the metatable), but when thinking about adding explicit OOP this was the direction I was thinking about (so records would not always get an automatic metatable, but objects would).

[Mehgugs]

I dont know if this is a good idea.

OOP in lua is so ad hoc and dynamic that I think this would easily feel restrictive to some people. I'd rather see some more general metatable/metamethod stuff first. I do see where a simple circular __index "class" could be useful though since it's probably the common denominator among all the "class" implementations out there.

The Typescript to JS situation with classes was different since -- if I recall correctly -- there was already a standard pattern with function prototypes and then the new keyword. There was then a proposal to add the class keyword to wrap up this pattern in a familar syntax. Typescript usually implements these "features to be" earlier since they can transpile it all down to whatever version of js you've configured it to aim for. Lua has no proposals to implement classes nor does it have a standardized class pattern itself.

[hishamhm]

OOP in lua is so ad hoc and dynamic that I think this would easily feel restrictive to some people.

I would never prevent people to manipulate metatables explicitly like they do today. The only issue is that fully-manual OOP is very dynamic, which means the compiler has no information about it — so when you do the metatable plumbing by hand you also need to add typecast plumbing using as in order to tell Teal what you're doing, with things such as Point.__index = Point as Point and local self: Point = setmetatable({}, Point as METATABLE).

(In other words: if doing OOP by hand in Lua is verbose, doing it by hand in Teal is currently even more verbose.)

I'd rather see some more general metatable/metamethod stuff first.

This is the general direction I've been leaning lately (not discarding OOP in the roadmap but thinking about adding something to records first). This thread has a lot of food for thought, lots of great input!

Lua has no proposals to implement classes nor does it have a standardized class pattern itself.

Well, yes, not as a standard part of Lua but, as you said...

I do see where a simple circular __index "class" could be useful though since it's probably the common denominator among all the "class" implementations out there.

...there are de facto patterns that are popular out there, which have some common baseline. Even the PiL book teaches these patterns. So, just like self is a bit of syntactic sugar added to Lua to ease OOP, then maybe adding another extra bit with some bare minimum for instantiation time would help the compiler make better assumptions (and reduce a bit the verbosity as a bonus).

[Mehgugs]

then maybe adding another extra bit with some bare minimum for instantiation time would help the compiler make better assumptions (and reduce a bit the verbosity as a bonus).

Yeah I don't think this would do any harm either 👍

[mascarenhas]

I would never prevent people to manipulate metatables explicitly like they do today. The only issue is that fully-manual OOP is very dynamic, which means the compiler has no information about it — so when you do the metatable plumbing by hand you also need to add typecast plumbing using as in order to tell Teal what you're doing, with things such as Point.__index = Point as Point and local self: Point = setmetatable({}, Point as METATABLE).

(In other words: if doing OOP by hand in Lua is verbose, doing it by hand in Teal is currently even more verbose.)

Trying to make the compiler understand and type simple metatable patterns for OO was also something we tried with Typed Lua, the problem is that, while in Lua you can abstract away the metatable boilerplate (and this is what some of the "class systems" do) you cannot do that with a static type system.

In Lua you can easily do:

local function class()
  local k = {}
  k.__index = k
  return k, function (obj) return setmetatable(obj, class) end
end

local Point, newPoint = class()

But in Teal (as in Typed Lua) with setmetatable hacks you are stuck with the boilerplate, unless you decide to go the macro route and open another huge can of worms. :smile:

[hishamhm]

Trying to make the compiler understand and type simple metatable patterns for OO was also something we tried with Typed Lua

Ah, I did not mean that. The compiler won't try to understand it. To use metatables you have to put typecasts by hand everywhere (that's why I said doing it in Teal it's even more boilerplate than in Lua).

The path I'm considering is to just wrap one common metatable pattern as a language construct ("class", "object", whatever), which the compiler will be able to understand and that's it. If you want to use metatables explicitly in other creative ways, then you're on your own.

[dnchu]

What about instead of OOP, we had something like embedding in Go? For example:

type Discount struct {
    percent float32
    startTime uint64
    endTime uint64
}

type PremiumDiscount struct {
    Discount // Embedded
    additional float32
}

[cloudfreexiao]

is tl will support interface grammar like c# ?

[euclidianAce]

I've hacked together a branch that allows for record embedding. While this isn't a fully integrated solution to OOP (as it doesn't hook into metatables at all), it does allow for composition of types similar to Go with its embedding or the feel of C with anonymous structs inside of structs.

Currently the branch type checks the basics of embedding:

local record A
   x: number
   y: number
end
local record B
   embed A
   z: number
end
local function a(var: A)
   print(var.x)
   print(var.y)
end
local function b(var: B)
   a(var)
   print(var.z)
end

But has some trouble resolving generics For example an embedding like

local record Foo
   embed Bar<string>
end

works fine but

local record Foo<T>
   embed Bar<T>
end

doesn't.

Internally, this basically just lets the record with the embedding be substituted for any of its embeddings

More complex things like the semantics of how nesting and embedding records can be worked out later, but I'd like feedback before working further.

Additionally, I think this could replace how array-records work and introduce map-records both as just records with arrays and maps embedded.

This was partially inspired by attempting to write some type defs for luv, where all the handle types inherit some base methods and a simple copy-paste sort of method like embedding works nice for those definitions.

[hishamhm]

@euclidianAce This is a really interesting experiment! This is really another step towards intersection types (as were function overloads in records and array-records). I'm starting to wonder if the compiler should just adopt intersection types internally for real, and expose special syntax for common cases^* for better ergonomics (or PX (programmer experience), if we want to be cool :) ).

(^* and possibly restrict the ones that are hard/unfeasible to deal with given runtime constraints, as we've done with other features such as unions)

[euclidianAce]

While intersection types are not a direct solution to integrating metatables, I do think it would be super helpful for annotating existing OOP implementations. For example, the embedding branch only allows records to be embedded but that's just an if check. Embedding a function type could help emulate a __call metamethod, and embedding certain types of maps could help with __index and __newindex methods.

[arnoson]

is there any news on this?

[vitiral]

I was pointed in this direction to discuss the "metatype model" of Lua.

I've written a pure-lua library inspired by teal to encapsulate what I think this could look like. IMO it would be great if teal (and others) used something similar to this, enabling runtime type checking for tests and also getting rid of many of the warts of lua (primarily debug formatting and table equality)

https://github.com/civboot/civlua/blob/main/metaty/README.md

My suggestion in https://github.com/teal-language/tl/issues/697 was that Teal could compile something like this:

local record Point
  x: number = 0 -- default=0
  y: number = 0
  tag: string?
end 

Into Lua code using metaty (a.k.a Teal would depend on metaty or similar):

local Point = metaty.record('Point')
  :field('x', 'number, 0)
  :field('y', 'number, 0)
  :fieldMaybe('tag', 'string')

Note: metaty allows representing types in pure lua and enables (optional) type checking which is zero-cost when disabled.

This would allow for incremental transition from Lua to teal. Also, enabling test-time type checking at runtime means that Lua code importing Teal could still get fail-fast type errors if/when they use the incorrect type: something that is very valuable IMO for the interop story.