Typescript Conversion

[ogrady]

Migrate to Typescript. Fixes https://github.com/queicherius/gw2api-client/issues/64

⚠ This is still a draft and far from done. Some parts are currently not even in a compileable state.

Most parts of the migration are rather straightforward, thanks to the well-structured nature of the current state of gw2-api-client. The problematic part is the variety of schemas that not always follow a linear inheritance, but sometimes change the structure of the old schema entirely.

Accomodating the Various Schemas

My best guess as to how to solve this is currently the following: Each endpoint has their various responses defined in the endpoints/schemas/responses directory. If a response can vary based on the selected schema, then that is reflected by exposing the type in various schema versions through namespaces[^1]. See Category, which has changed significantly on 2022-03-23.

Schemas are then bundled in the endpoints/schemas directory.

Each schema files exports an interface Schema which provides the type information for all endpoints (or the encapsulating client) for that specific schema version. Calls to get, all, etc. change their result type accordingly. Take the Cats response, which has changed from the original schema to the 2019-03-22 schema:

import { Schema as Schema_1970_01_01} from './endpoints/schemas/schema_1970_01_01'
import { Schema as Schema_2022_03_09} from './endpoints/schemas/schema_2022_03_09'
import { CatsEndpoint } from './endpoints/cats'

let oldEndpoint: CatsEndpoint<Schema_1970_01_01>
let newEndpoint: CatsEndpoint<Schema_2022_03_09>

oldEndpoint.get(…).then(res => res.hint)
//                        ↳ { hint, id }
newEndpoint.get(…).then(res => res)
//                        ↳ number

I believe this makes the most sense, but requires a bit of voodoo and probably a change in gw2api-clients API, as users would have to explicitely give a schema version while instantiating the client (see below).

As this requires quite a bit of manual work, could you please give your opinion on this to make sure this doesn't go in a completely unmaintainable direction, @queicherius?

EDIT: shoehorning a demonstration of how the instantiation with different schemas would look like and what issues would arise from it:

Different Schema Versions in Action

import { Schema as S1970 } from './endpoints/schemas/schema_1970_01_01'
import { Schema as S2022 } from './endpoints/schemas/schema_2022_03_09'

class Client<S extends Schema> {
    // overloads to enable the user to pass in exact timestamps to automatically select the matching schema
    schema (schema: '1970-01-01T00:00Z'): Client<S1970>;
    schema (schema: '2022-03-09T00:00Z'): Client<S2022>;
    // catchall
    schema <S extends Schema>(schema: string): Client<S>;
    schema <S extends Schema>(schema: string): Client<S> {
      // can't change generic parameter of existing instance → create new instance and return that instead.
      // Obvious drawback: references to old client become stale
      const result = new Client<S>()
      result.schemaVersion = schema
      //return this
      return result      
    }
}

const clientWithOldSchema = new Client<S1970>()
// for exact matches, the generic parameter is inferred from the overloads 👍
const clientWithNewSchema = clientWithOldSchema.schema('2022-03-09T00:00Z')
// for everything else (mind the hour which is off by one), the user has to explicitly pass a schema. 😕
// Note: the user is responsible for matching their string parameter with the schema version to receive correct types. See below. 
const clientWithManuallySetSchema = clientWithOldSchema.schema<S2022>('2022-03-09T01:00Z')
// user didn't match their string with their schema. Suggested types will match the 1970 schema, actual responses will match the S2022 schema 👎
const clientWithFaultySchema = clientWithOldSchema.schema<S1970>('2022-03-09T01:00Z')

clientWithOldSchema.cats().get(42).then(cat => cat)  // {id, hint} 👍
clientWithNewSchema.cats().get(42).then(cat => cat)  // number 👍
clientWithManuallySetSchema.cats().get(42).then(cat => cat)  // number 👍
clientWithFaultySchema.cats().get(42).then(cat => cat)  // {id, hint} 👎 is actually number at runtime

[^1]: although namespaces are kind of deprecated, I believe they make the most sense here.

[queicherius]

Thank you for this! I'll read through it on the weekend to give you proper feedback 🙂

[queicherius]

The problematic part is the variety of schemas that not always follow a linear inheritance, but sometimes change the structure of the old schema entirely.

Yeah, I agree that that's the tricky part, and also the part that I was most thinking about reading through your changes.

Each endpoint has their various responses defined in the endpoints/schemas/responses directory. If a response can vary based on the selected schema, then that is reflected by exposing the type in various schema versions through namespaces[^1]. See Category, which has changed significantly on 2022-03-23.

Like mentioned above, I do like the separation into a different file for the schemas, and I do like the idea of having the history there. I don't have any opinion on namespaces, what you did seemed reasonable to me. (Another option entirely is to just provide "latest" types, which would be OK for me too, but kind of sad to loose the information)

Schemas are then bundled in the endpoints/schemas directory. Each schema files exports an interface Schema which provides the type information for all endpoints (or the encapsulating client) for that specific schema version.

I am worried about how the compose together (and how many types that will generate) once there are a few dozen different schema versions, each with all of the endpoints that come before + at current time.

It might be a good idea to compose them "one by one" (e.g. Schema-2 inherits from Schema-1 and not from BaseSchema) to avoid duplication?

import { Schema as S1970 } from './endpoints/schemas/schema_1970_01_01'
import { Schema as S2022 } from './endpoints/schemas/schema_2022_03_09'

I would explicitly name the schema types: Schema_2022_03_09 (so you can just import, without aliasing)

    // overloads to enable the user to pass in exact timestamps to automatically select the matching schema
    schema (schema: '1970-01-01T00:00Z'): Client<S1970>;
    schema (schema: '2022-03-09T00:00Z'): Client<S2022>;

I'm OK with just enforcing the day, which would make this a bit less "voodoo": schema (schema: '2022-03-09'): Client<Sceham_2022_03_09>;

      // can't change generic parameter of existing instance → create new instance and return that instead.
      // Obvious drawback: references to old client become stale

I would also be OK with the schema having to be explicitly provided via the generic (and this function not switching) or a new API entirely that returns a new client with the specific schema.

// Note: the user is responsible for matching their string parameter with the schema version to receive correct types. See below. 

That'd be fine IMO - we just have to document it propperly.

[queicherius]

I had another idea previously with "global type configuration" via Type Augmentation and Merging Interfaces (stolen from here).

https://www.typescriptlang.org/play?#code/C4TwDgpgBAygxgCwgWwIYDUICcDOBLAewDsoBeKAcgEYBOAdgAYBaBqlqiqAH0oCZWaLAMxNevCgCgJAemlQAqjmxQANhABuEFRLxFg2AGao40AMIBXHMALIAKuAgB5MMEJEcUAN4SoUHIhQMbHxiAC4+AWFRcShZKFMEVCIAc2hgBDwcADoJAF8pOIAZPAAjLFQsEFUNLQlQSCgAJQgcAhVNAB5bAD4yKFsoCAAPfSIAEw8AMXMiOFdiKAB+fqhwzwBtAGkoXSgAawgQAgN+gF1w2y3T3IBuKXqzVGAAfXgkNGfaRmfWH6o+zw7MbhIjmZAlbA3KAZPThKxYXTJKD5B7xJ6vAIffi0H5CZ5iPqg8HYKTDMAELDAKCo0zot6BPrNVrtCAdHzxSzWOwOZzzdzrCj+d5BXBuCinQYjCDjDz0tCYUXEdm+ZYWKw2eyQXluHACoWBBUhIjiyWjCYRWhRMSSXy2pZol5y1D4gS4-G8ZW28K0x2Y51fBh-P6e8IUWxIPx+qCaRUkI7mKBgLAEdR4MYQMY7Dy6dSoFRpyTdKRwYhWKBDb10qM8UEqFSE8x1mRyW0APUWEiAA

But I actually like your approach a lot more, because I know that people (including myself) are using different clients with different schemas across a single application.

[ogrady]

Thank you for the very detailed feedback and insights!

I am worried about how the compose together (and how many types that will generate) once there are a few dozen different schema versions, each with all of the endpoints that come before + at current time.

It might be a good idea to compose them "one by one" (e.g. Schema-2 inherits from Schema-1 and not from BaseSchema) to avoid duplication?

I absolutely agree. That is a wrinkle that annoyed me too. I have since updated them to a linear inheritance structure that only requires the delta in each new schema.

I would explicitly name the schema types: Schema_2022_03_09 (so you can just import, without aliasing)

Can absolutely do. I streamlined the names to Schema to avoid possible typos between the file name and the name of the schema, but I do see the benefits when using multiple schemas in the same consuming file.

I'm OK with just enforcing the day, which would make this a bit less "voodoo": schema (schema: '2022-03-09'): Client<Sceham_2022_03_09>;

I think that's a good compromise 👍 As you already mentioned, we'd have to make sure it is very explicitly documented why this argument differs from the arguments mentioned in the official API (I really like that your current implementation stayed very close to what is documented in the wiki).

Regarding your suggestions with Type Augmentation and Interface Merging:

That's actually a very cool feature I didn't know of! Will read a bit more into and consider as well.

[queicherius]

I absolutely agree. That is a wrinkle that annoyed me too. I have since updated them to a linear inheritance structure that only requires the delta in each new schema.

Yeah that sounds great, thank you :100:

Can absolutely do. I streamlined the names to Schema to avoid possible typos between the file name and the name of the schema, but I do see the benefits when using multiple schemas in the same consuming file.

Thanks, perfect!

I think that's a good compromise 👍 As you already mentioned, we'd have to make sure it is very explicitly documented why this argument differs from the arguments mentioned in the official API (I really like that your current implementation stayed very close to what is documented in the wiki).

:+1:

Regarding your suggestions with Type Augmentation and Interface Merging: That's actually a very cool feature I didn't know of! Will read a bit more into and consider as well.

Thanks! I still think your version is better, but its very possible I'm missing something.

[ogrady]

Okay, great! I think I am set to keep working on the PR then. Full disclosure: it will probably take quite some time to get everything up and running, since I can only put in an hour or two a week. So don't worry if you don't hear back for a while.

You are probably aware already, but this will be a huge PR (touching basically every file, breaking changes, introduction of ESLint[^1], probably introduction of rudimentary testing of types against actual API responses). Feels a bit uncomfortable and monolithic, but I hope that's fine.

[^1]: thanks for the baseline! Will gladly use.

[queicherius]

it will probably take quite some time to get everything up and running, since I can only put in an hour or two a week. So don't worry if you don't hear back for a while.

No worries at all, I'm very much in the same boat. Appreciate you taking this on!

You are probably aware already, but this will be a huge PR [...]

Yeah that's totally fine - then once it's in it's final stages I can carve out a good chunk of time to review :)

probably introduction of rudimentary testing of types against actual API responses

That's a good idea, I like it!