sinclairzx81
`Type.Evaluate` doesn't respect `Type.Base` during `Type.Intersect`
When I try to intersect a type extended from base and then evaluate it, it turns into never
import { Type, type StaticDecode } from 'typebox';
class TId extends Type.Base<string> {}
const schemaNonEvaluated = Type.Intersect([
new TId(),
new TId(),
]);
type TNonEvaluated = StaticDecode<typeof schemaNonEvaluated>;
/**
* const schemaNonEvaluated: Type.TIntersect<[TId, TId]>;
* type TNonEvaluated = string;
*/
const schemaEvaluated = Type.Evaluate(schemaNonEvaluated);
type TEvaluated = StaticDecode<typeof schemaEvaluated>;
/**
* const schemaEvaluated: Type.TNever;
* type TEvaluated = never;
*
* instead of
*
* const schemaEvaluated: TId;
* type TEvaluated = string;
*/
Showcase: https://play.vuejs.org/#eNqNlMty2jAUhl9Fo00IQ+1Fu6IOnaZlQRe003gXZaHYB+PEljTSMU6G4d1zJAMxt0w2HnSu3y9+e81/GhOtGuBjnrjMlgaZA2wMq6QqbgRHJ/hEqLI22iJbs/TVwIghPdkdSiyz35DpHNiGLayu2ZXPPOqXq+9CCZVV0jmWznIGLwgqp9+Ujm6lg8ShLVUxYetNqNTK0eZsCbWcazVdyaqRCDm76VpmCsE6yHBwLxRjClo/dnA9Oj49XNPmgJcezenjJr5CL84snATweDikwUPGLnGNO6x0z5XcE8HIYzz4Eb73LEUnO1TEx8JPVO8Cg1OAd5mf0/gZgSfq5rACe6Cmv0z1st2zpFkgc6YX/eilFbP84ujDW0rizpnkQzog1KaiOjolce/AR7xz6ZdamujJaUWeXnt/iG2CrDxmIeJjW6f6mOBLROPGcdy2bdQo81xEma7jbcmPWudNBYL7VrLrhlahI1mLsjhaRF2mrMD+NViS7IOFsqp0+yfE0DYQrBt6lpA9n4k/uS3cPwsO7IoA9jmUtgDs0tO7Ob1eveQO94Pkf3C6ajxjV3bbqJywe3WBdhbujf6K1E39G+x2ojxouI1QLzh9QX59IP0d92v0bX+LmzeR+4iI
The most straightforward solution I see is to add Equals method to Base type and use it within Narrow function in /src/type/engine/evaluate/narrow.ts
@orimay Hi! Thanks for reporting! (good to see users diving into some of TB's internals!)
The most straightforward solution I see is to add Equals method to Base type and use it within Narrow function in /src/type/engine/evaluate/narrow.ts
At this stage, the Base types are somewhat at odds with Schema types and I haven't quite figured out how to reconcile them with the rest of the type system. At this stage, evaluating two intersected Base types will result in never because TB can't derive structural information from class instances (they are effectively nominal)
But you're on the right track with Equals tho! There's actually two functions that need to be implemented (ExtendsLeft, and ExtendsRight) where the extends logic may need to perform the check in reverse. The checks are modelling this aspect of TypeScript.
type R = Left extends Right ? true : false
//
// where: Left/Right is substituted with Base instances
The issue tho is that TB needs a symmetric (runtime and type-level) implementation and I haven't figured out an API design that would achieve the type-level check, but I suspect HKT implementations may be required.
With regards to API | HKT design, I have been exploring HKT implementations on another (albeit unrelated) project. Where the parser design for mapping AST structures has been working quite well, but where the callbacks "could" be repurposed for dynamic (or type level dynamic) Base type extends checks.
TypeScript Link | Compilers in the Type System
type Math = Wat<`(module
(func private_function (param $x f32) (param $y f32) (result f32) (result f32)
local.get $x
local.get $y
)
(func (export swap) (param $x f32) (param $y f32) (result f32) (result f32)
local.get $y
local.get $x
)
(func (export add) (param $x f32) (param $y f32) (result f32)
local.get $x
local.get $y
f32.add
)
(func (export sub) (param $x f32) (param $y f32) (result f32)
local.get $x
local.get $y
f32.sub
)
(func (export mul) (param $x f32) (param $y f32) (result f32)
local.get $x
local.get $y
f32.mul
)
(func (export div) (param $x f32) (param $y f32) (result f32)
local.get $x
local.get $y
f32.div
)
)`>
Unfortunately, there's no established patterns for handling these kinds of things in the type system (this is on the upper end complex design work, so progress here will be slow). There won't be any significant changes to the TB type system before the end of the year (just stabilization work mostly) so will pick this one up next year .... but experimentation from the community is always welcome! Design suggestions too!
I will drop a research tag on it for now. Cheers! S
@sinclairzx81, just FYI, this is not a breaking change (unless someone relies on getting TNever for Base, but I don't think it's very much probable and/or limiting. Equals method is not required and has a default implementation, and, most likely, is only needed for generic variants (as shown in tests with Bar class). Would highly appreciate it if you get some time to merge it, as it will make my complex project much easier. Please let me know if any other adjustments are needed. Thank you!
@orimay Hi, thanks for the PR and sorry for the delay (have been busy wrapping up work for the year)
So have had a look through the PR tonight (thanks for this), but not too sure about the current interpretation of Base. The implementation provided performs a structural Equal comparison check on the Base type itself (which gets around Never), but I'm not sure if it's accurate to perform the Equal check on the Base type instances themselves... but rather the structure Base is trying to represent ....
Base Represents Structural Types
This is a more concrete example of what Base Extends would need to model for Reference Link. The goal would be to have Base behave the same as schematic based types, but where the mechanisms to achieve this are not clear (as of writing)
import Type from 'typebox'
// -----------------------------------------------------------------
// (1) TypeScript: Structural Extends
// -----------------------------------------------------------------
{
type Vector3 = { x: number, y: number, z: number }
type Vector2 = { x: number, y: number }
type A = Vector3 extends Vector2 ? true : false // true
type B = Vector2 extends Vector3 ? true : false // false
}
// -----------------------------------------------------------------
// (2) TypeBox: Structural Extends
// -----------------------------------------------------------------
{
const { A, B } = Type.Script(`
type Vector3 = { x: number, y: number, z: number }
type Vector2 = { x: number, y: number }
type A = Vector3 extends Vector2 ? true : false
type B = Vector2 extends Vector3 ? true : false
`)
type A = Type.Static<typeof A> // true
type B = Type.Static<typeof B> // false
}
// -----------------------------------------------------------------
// (3) TypeBox: Base Extends Base
//
// Below Vector3 and Vector2 express a structural shape, but how
// should the Type.Base<...> describe Left / Right assignability?
//
// -----------------------------------------------------------------
{
class Vector3 extends Type.Base<{ x: number, y: number, z: number }> { /* ??? */ }
class Vector2 extends Type.Base<{ x: number, y: number }> { /* ??? */ }
const A = Type.Extends({}, new Vector2, new Vector3) // expect: TExtendsTrue
const B = Type.Extends({}, new Vector3, new Vector2) // expect: TExtendsFalse
}
// -----------------------------------------------------------------
// (4) TypeBox: Base Extends Schematic
// -----------------------------------------------------------------
{
class Vector3 extends Type.Base<{ x: number, y: number, z: number }> { /* ??? */ }
const Vector2 = Type.Script(`{ x: number, y: number }`)
const A = Type.Extends({}, Vector2, new Vector3) // expect: TExtendsTrue
const B = Type.Extends({}, new Vector3, Vector2) // expect: TExtendsFalse
}
So in cases (3) and (4), we wouldn't perform a check against the class Vector3 or class Vector2 instances, but rather the generic parameter { x: number, y: number, ... } as this is what Base is structurally representing. The Base type should provide a mechanism to allow it to perform cases (3) and (4) but where there is no obvious mechanism to achieve this.
Note: The closest thing I have to derive a structural type / schema from TS type is ReverseStatic but only solves half the problem (there is still a runtime mapping aspect to this). This needs a lot of work.
Equal
Looking at the current Equals check, I'm not too sure this would work quite right without nominal typing support in TS. For example, the following Base types are structurally Equal.
Ref: Nominal Typing in TypeScript (from 2014) https://github.com/Microsoft/TypeScript/issues/202 Ref: Nominal Emulation using Structural LR Checks Reference Code
// These instances would be considered structurally Equal
class A extends Type.Base {}
class B extends Type.Base {}
This said, there could be some utility in performing instance level extends checks (as per PR) but this would be somewhat orthogonal to the generic parameter structural requirement for Base itself. It is worth considering tho.
ExtendsLeft / ExtendsRight
I do think there needs to be a ExtendsLeft(...) and ExtendsRight(...) override for Base, and where the Left / Right implementation would need to try and derive associativity corresponding to the Base generic type <T>, but it's not clear how to achieve this, or what the API for such a thing would look like (HKT's spring to mind, perhaps calling to ReverseStatic or user defined implementation)
... but given the challenges, the current implementation is basically saying ....
"We can't see into the Base structure, so we can't accurately tell if Left extends Right (or vice versa), so the safest thing is to say neither type extends, even if Left and Right are the same instance of Base"
The default of Never is somewhat unfortunate, but is a stand-in until the structural representational aspects of Base can be resolved (it's a really difficult design problem!!!)
Unfortunately, I don't think I can take on the PR as it's only solving instance level Equals comparison on Base, but still happy to discuss more.
Like most things in TypeBox, these kinds of designs tend to move at glacial speeds and there needs to be a lot of experimentation and design work done first. If I get a implementation wrong, it usually means I need to carry the implementation around for years (and there is some uncertainty if Base will be a long term type in the library given the reconciling challenges with structural types)
But lets discuss more. Cheers! S
@sinclairzx81, thank you for the detailed response! I can see an easier option that may make everyone's lives much easier. What if Base constructor accepted an optional schema of type T extends ReverseStatic<Value>? Then we could have used that type for such checks and type merges. And some types, like Date, could have had that omitted:
class Vector3 extends Type.Base<{ x: number, y: number, z: number }> {
public constructor() {
super(Type.Object({
x: Type.Number(),
y: Type.Number(),
z: Type.Number(),
}));
}
}
@orimay Hiya, again, sorry for the delay.
thank you for the detailed response! I can see an easier option that may make everyone's lives much easier. What if Base constructor accepted an optional schema of type T extends ReverseStatic<Value>? Then we could have used that type for such checks and type merges. And some types, like Date, could have had that omitted:
Also, it's not a breaking change and is a progressive enhancement
Hmmm, perhaps. I have been mulling over this issue, and the more I look at the Base type, the more I think the introduction of it in V1 was a mistake. TypeBox really wants to operate in a structural sense, and the requirement to add additional "things" to Base to make it integrate correctly seems to be an indication the type is not fitting the TBV1 design (at least not in the way I had hoped I could get it to work)
Let's consider that ReverseStatic + interior Schema suggestion, and just to follow through on a train a thought.
Base
This could technically work where ReverseStatic<T> could be used to constrain the Base constructor (super)
class Vector3 extends Type.Base<{ x: number, y: number, z: number }> {
public constructor() {
super(Type.Object({
x: Type.Number(),
y: Type.Number(),
z: Type.Number(),
}));
}
}
... but the natural extension of this would be the following....
class DateType extends Type.Base<Date> {
public constructor() {
super(Type.Object({
toString: Type.Function([], Type.String()),
getTime: Type.Function([], Type.Number()),
toISOString: Type.Function([], Type.String()),
toUTCString: Type.Function([], Type.String()),
// ...
}));
}
}
... which just makes me wonder if just this would be reasonable ...
const DateType = Type.Object({
toString: Type.Function([], Type.String()),
getTime: Type.Function([], Type.Number()),
toISOString: Type.Function([], Type.String()),
toUTCString: Type.Function([], Type.String()),
// ...
})
... because it naturally integrates with the TB type system with no additional complexity Reference Link
import Type from 'typebox'
const DateType = Type.Object({
toString: Type.Function([], Type.String()),
getTime: Type.Function([], Type.Number()),
toISOString: Type.Function([], Type.String()),
toUTCString: Type.Function([], Type.String()),
// ...
})
// Evaluation Works | Intersected Property
const Entry = Type.Evaluate(Type.Intersect([ // const Entry: Type.TObject<{
Type.Object({ created: DateType }), // created: Type.TObject<{
Type.Object({ created: DateType }), // toString: Type.TFunction<[], Type.TString>;
])) // getTime: Type.TFunction<[], Type.TNumber>;
// toISOString: Type.TFunction<[], Type.TString>;
// toUTCString: Type.TFunction<[], Type.TString>;
// }>;
// }>
function test(entry: Type.Static<typeof Entry>) {
const iso = entry.created.toISOString() // appears like a globalThis.Date
}
The above would be the natural way to express arbitrary structures using the TB V1 type system .... and much of the ground work has been setup to move the library in this direction. I guess the main issue with Base is it requires bolting on all this intrinsic functionality (Check, Clone, Clean, Default, etc) just to be able to compensate for the fact it isn't a schematic,. My general thinking is that the additional complexity to support Extends / Evaluate probably indicates the Base type is probably the wrong fit.
What are your thoughts on this latter example?
Again, sorry for the delay (busy end to the year), but still very keen to discuss this further. I'd like to get some thoughts on this latter approach as it would be technically feasible to validate for Date (or other) instances using structural checks (while also letting types naturally evaluate under the TBV1 type system)
It's a bit of tangent from Base (we can return to this), but would be keen to hear your thoughts on the above all the same. S
@orimay Hi, just a quick additional update here.
I have just pushed a small update on 1.0.64 to enables the following.
const Uint8ArrayType = Type.Object({
BYTES_PER_ELEMENT: Type.Literal(1),
byteLength: Type.Number(),
byteOffset: Type.Number(),
// ... additional
})
const DateType = Type.Object({
getTime: Type.Function([], Type.Number()),
toDateString: Type.Function([], Type.String()),
toISOString: Type.Function([], Type.String()),
// ... additional
})
console.log('Uint8Array', Value.Check(Uint8ArrayType, new Uint8Array())) // true
console.log('Date', Value.Check(DateType, new Date())) // true
.. and for TId
import { ObjectId } from 'npm:mongodb' // ???
const TId = Type.Object({
_bsontype: Type.Literal('ObjectId'),
getTimestamp: Type.Function([], Type.Unknown()),
toHexString: Type.Function([], Type.Unknown()),
// ... additional
})
console.log('TId', Value.Check(TId, new ObjectId())) // true
... In the case of Date, it would be possible wrap the schema in Unsafe to infer as a globalThis.Date Reference Link
import Type from 'typebox'
const DateType = Type.Unsafe<globalThis.Date>(Type.Object({
getTime: Type.Function([], Type.Number()),
toDateString: Type.Function([], Type.String()),
toISOString: Type.Function([], Type.String()),
// ... additional
}))
type T = Type.Static<typeof DateType> // type T = Date
Again, open to thoughts here. The function checks are not perfectly ideal as it's not possible to derive parameter and return type information from JavaScript functions (the best TB can do tell a function exists at a instance key), but this approach does achieve some level of structural duck typing which would naturally integrate with the V1 Extends and Evaluate infrastructure.
This is worth some consideration as an alternative to Base.
One more reference link for consideration Reference Link
// ------------------------------------------------------------------------
// type derived from lib.d.ts
// ------------------------------------------------------------------------
export const DateType = () => Type.Object({
[Symbol.toPrimitive]: Type.Any(),
/** Returns a string representation of a date. The format of the string depends on the locale. */
toString: Type.Function([], Type.String()),
/** Returns a date as a string value. */
toDateString: Type.Function([], Type.String()),
/** Returns a time as a string value. */
toTimeString: Type.Function([], Type.String()),
/** Returns a value as a string value appropriate to the host environment's current locale. */
toLocaleString: Type.Function([], Type.String()),
/** Returns a date as a string value appropriate to the host environment's current locale. */
toLocaleDateString: Type.Function([], Type.String()),
/** Returns a time as a string value appropriate to the host environment's current locale. */
toLocaleTimeString: Type.Function([], Type.String()),
/** Returns the stored time value in milliseconds since midnight, January 1, 1970 UTC. */
valueOf: Type.Function([], Type.Number()),
/** Returns the stored time value in milliseconds since midnight, January 1, 1970 UTC. */
getTime: Type.Function([], Type.Number()),
/** Gets the year, using local time. */
getFullYear: Type.Function([], Type.Number()),
/** Gets the year using Universal Coordinated Time (UTC). */
getUTCFullYear: Type.Function([], Type.Number()),
/** Gets the month, using local time. */
getMonth: Type.Function([], Type.Number()),
/** Gets the month of a Date object using Universal Coordinated Time (UTC). */
getUTCMonth: Type.Function([], Type.Number()),
/** Gets the day-of-the-month, using local time. */
getDate: Type.Function([], Type.Number()),
// ...
})
@sinclairzx81, sorry for late response as well, things are busy here. I was hoping to not express Date as a set of its functions (or any other class, especially imported, that may change over time). My use cases include, for instance, RecordId that I get from the database, and I convert it to a branded type. I tried dealing with Type.Unsafe, but it was not a very pleasant experience so far. Base type seems to work well with cases like RecordId, because the check is not based on a set of properties, but rather on the simple instanceof check, and the same perfectly works for Date, or any Temporal type, that I use. Most likely, my codebase will be out of reach for until after the 4th of January, but my project involves some complex type transformations, and that's exactly why I report so many issues :D
- I load a structure from SurrealDB, and it has some classes, such as
RecordIdmentioned above - I transform it into plain data that I can work with (e.g. I transform
RecordIdinto a brandedstring, and somestringtoTemporalDate/Time/DateTimeinstances) - I store it inside a class, that allows reactive access to the data properties via Vue and tracking of the properties that change deep. I use yet another class, that converts between plain data and the class, because my reactivity-tracking class uses TypeBox schema, so it can't be a part of
Type.Codec, as it would cause self-reference:c - Once the data gets changed, I encode everything back into SurrealDB representation (where I use
RecordIdin place of the brandedstringandstringin place ofTemporalinstances again) and send it to the DB.