pinecone-ts-client icon indicating copy to clipboard operation
pinecone-ts-client copied to clipboard

Remove Typebox and AVJ to work better with Edge runtimes

Open aulorbe opened this issue 1 year ago • 0 comments

Problem

There have been a litany of issues related to the incompatibility of our current Typescript client and Edge runtimes (e.g. issue 164, issue 212, this community forum post).

The long and the short of it is that we use two libraries: sinclair/typebox and ajv that to do not work in Edge runtimes. The result is that users need to downgrade their client version in order to use our client. This has the major disadvantage of meaning that these users don't get access to our client's newest features (e.g. inference).

Asana ticket: https://app.asana.com/0/1203260648987893/1208002049018710/f

Solution

Remove typebox and ajv!

Approach

  • Previously we had custom typebox schema objects declared in types.ts files under control and data. This PR removes those typebox schemas. We used those typebox schemas to do things like ensure the correct properties were sent with requests like Create Index, which needs a particular Spec obj with particular fields, etc. This PR maintains that behavior, but through custom validation functions embedded within the client's native objects (e.g. QueryCommand, createIndex, etc.) themselves.

  • Importantly, these internal changes do not result in any downstream changes the user should be made aware of -- all exported Pinecone types remain untouched in files like index.ts under data and control.

  • This PR also introduces new unit and integration tests to ensure the above-mentioned functionality. It also removes unit and integration tests that simply test whether a particular field's value is of a particular data type.

    • I chose to remove these tests since they're redundant in Typescript. The thinking here is that: if we're going to write a Typescript client and glean the benefits of using Typescript (rather than Javascript), we need to rely on Typescript's typing power. Otherwise, we should just write a Javascript client. I know this is unideal for non-Typescript users of our client, but the backend API will throw errors at runtime if non-TS users pass in the incorrect datatypes, and this is enough for now.
  • Finally, this PR introduces checks for unknown field properties/names. If a user passes in diemnsion instead of dimension when creating an index, an error will be thrown that tells the user 1) what field was identified as unknown and 2) a list of all the acceptable fields for that object.

Type of Change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] This change requires a documentation update
  • [ ] Infrastructure change (CI configs, etc)
  • [ ] Non-code change (docs, etc)
  • [ ] None of the above: (explain here)

Test Plan

CI passes.

aulorbe avatar Aug 13 '24 20:08 aulorbe