Support/Query: Complex/Unnamed types makes Typescript use tricky

[asquithea]

New to electrodb, so apologies if this is obvious or just fundamentally can't be improved.

I'm finding that the types created by electrodb are quite hard to work with in practice in standard Typescript with strict mode enabled and all the recommended eslint rules turned on:

• type signatures are complex and unpredictable
• named types really needed for function signatures, but hard to get

Simplest possible example

I have a class implementing a service interface, with the DynamoDB client passed into the constructor. I want to create an Entity (or a service) and store it as a member variable for use in methods. What type signature do I use for the member variable?

It's an Entity<something, something, something, something>.

I've found two acceptable approaches:

1: Observe and accept

Notice that while the type params are not obvious, the schema type is at least recognizable and the whole signature is manageable.

Create a type alias for the schema as just declare the entity using the IDE-provided signature:

   private readonly integrations: Entity<string, string, string, SomeSchema>;

2: The factory function gambit

Notice that Typescript will infer function return types and also provides ReturnType<...>

Make a factory function and exploit that feature:

export function SomeEntity(client: electrodb.DocumentClient, table: string) {
  return new electrodb.Entity(
    {
      // ... schema
    },
    {
      client,
      table,
    },
  );
}

  private readonly entity: ReturnType<typeof SomeEntity>;

  constructor(client: ddb.DynamoDBClient, table: string) {
    this.entity = SomeEntity(client, table);
  }

Commentary

This is obviously a trivial example but basically illustrates the issue I'm having:

electrodb generates complex type signatures with lots of parameters which presents difficulties when you want to reference them by name, as in this member variable.

Paging Example

So for the next example, let's say we want to expose a simple paging system, which might look like:

export interface ListSomethingOptions {
  nextPageToken?: string;
}

export interface SomethingStore {
  listSomethingAsync(tenantName: string, options: ListSomethingOptions): Promise<SomethingList>;
  // ...
}

The obvious thing for me to do is implement this using the page method, perhaps by encoding the returned page object into the nextPageToken (let's say as base64-encoded JSON for this example):

    const [next, items] = await this.entity.query
      .byIds({ tenantName })
      .page(decodePageToken(options.nextPageToken));

I need to implement decodePageToken, but what type does it return?
It's not obvious. The easiest workaround is to start turning off ESLint:

// eslint-disable-next-line @typescript-eslint/no-explicit-any
function decodePageToken(token: string | undefined): any {
  return token ? JSON.parse(Buffer.from(token, "base64").toString()) : undefined;
}

But this is probably better though it's equivalent:

function decodePageToken<T>(token: string | undefined): T | undefined {
  return token ? (JSON.parse(Buffer.from(token, "base64").toString()) as T) : undefined;
}

Item Mapping Example

Continuing on from the previous example, let's say we now want to map the stored items onto the SomethingList to be returned. Happily the type signatures mostly match, so writing a reusable map function should be easy... right?

Not so much - because again I've no type name for the fields I'm mapping from.

Doing an inline map is dead easy:

    const [next, items] = await this.entity.query
      .byIds({ tenantName })
      .page(decodePageToken(options.nextPageToken));

    return {
      nextPageToken: encodePageToken(next),
      items: items.map(function (item) {
        return {
          tenantName: item.tenantName,
          id: item.fooId,
          // ... about 10 more
        };
      }),
    };

Making it reusable looks much harder though.

Question

I think this is mostly happening because instead of starting with a user-constructed type and then applying magic - as with most popular ORMs, or the standard DocumentClient - we start with a schema and get generated types back.

Do you have some recommendations on the best way to tackle this?

Would it be possible to add more practical Typescript examples into the docs illustrating good usage patterns that minimise this sort of pain - I feel like there's a decent possibility I'm just "holding it wrong" :-)

Many thanks for your efforts on electrodb.

[asquithea]

Subsequently found EntityItem and friends, which helps quite a bit - eliminates the mapping issue.

I missed this in the docs - found it indirectly through the repo examples.

Leaving this ticket open anyway for the moment in case you have any advice or thoughts on fleshing out the typescript examples in the docs - might just be a case of taking the existing example typescript code in the repo and framing it up as a tutorial.

[tywalch]

Hey @asquithea I appreciate the feedback, I agree the library has graduated to the point where a more robust documentation site is in order. I want to say thank you for taking the time to write this out, this is very detailed and helpful to for me to gain the perspective of you and other potentially new users.

On the subject of types, this library aims to own most of the complex typing required to match the underlying runtime validations the library provides. These validations (like required attributes not being able to be deleted, or the nuance of partially provided keys) are based on a user defined model but could become a difficult burden for users to reason about, update, and maintain. In lieu of requiring user maintained generics the library extensively uses type inferences, though this removes the ability for a user to pass their own types.

I see you found EntityItem and the others, do you feel it was helpful to see how those types were constructed to see how you may make other types yourself? I would like to export as many times as can be useful from the library itself. It sounds like the type for the cursor/pager returned by .page() is a good candidate?

Let me know your thoughts, and if you've had any more since your last post. I'm interested in expanding the types that are exported so any additional types you'd like to see would be good to know 👍

[asquithea]

The Pager would be another good candidate - probably along with anything that is likely to need to be passed between methods or stored in a variable.

I looked at the definition for EntityItem. At my current level of Typescript experience (only about 6 months) I don't think I'd have been able to come up with that definition on my own, nor easily use the pattern to add more. I think your use of Typescript is fairly advanced compared to what many devs will have encountered unless they're making a similarly complicated library :-)

I will run through the rest of my short project and update the ticket with any similar missing types if something sticks out.

So far the main issue other issue I've run into is ESLint throwing a fit at those places where any is required. This is not a Typescript issue or a library bug, but might be worth calling out in any tutorials.

[asquithea]

Closing - I'll probably stick with plain Document Client for this project.

electrodb looks really cool regardless

[jonnedeprez]

@tywalch To pick up on your proposal: yes, I think exported types for .page() would help. I'm having the same issue. I now defined the pager as any. In any case, I find this library elegantly written and a great improvement compared to working with the native client!

[tywalch]

@asquithea @jonnedeprez i am wrapping up changes to a breaking change (will become 2.0.0) to the return structure of go(). The purpose of the breaking change is to unify/simplify pagination across all areas of ElectroDB. This change will:

• deprecate page() in favor of extending go()
• type the pagination cursor as a string (escape hatch to get returned LastEvaluatedKey will remain)