Bug: Key size is larger than the maximum key size (1978)

[corgrath]

Problem

Gatsby builds fail due to a Key size is larger than the maximum key size (1978) error.

Background

This project imports the datocms-client project in its package.json: https://github.com/datocms/gatsby-source-datocms/blob/master/package.json#L41

In the js-datocms-client project, it tries to cache datocms data by creating a key

https://github.com/datocms/js-datocms-client/blob/5ea5073a3ee7285ccd800523f0dec4e80c058172/src/local/Loader.js#L24-L31

  cacheKey() {
    return `datocms-loader-data-${JSON.stringify({
      clientArgs: this.clientArgs,
      environment: this.environment,
      previewMode: this.previewMode,
      pageSize: this.pageSize,
    })}`;
  }

Full stack-trace

success loading DatoCMS schema - 2.381s

 ERROR #11321  API.NODE.EXECUTION

"gatsby-source-datocms" threw an error while running the createSchemaCustomization lifecycle:

Key size is larger than the maximum key size (1978)



  Error: Key size is larger than the maximum key size (1978)

  - write.js:159 writeInstructions
    [md_gatsby5]/[lmdb]/write.js:159:8

  - write.js:604 LMDBStore.put
    [md_gatsby5]/[lmdb]/write.js:604:8

  - cache-lmdb.ts:90 GatsbyCacheLmdb.set
    [md_gatsby5]/[gatsby]/src/utils/cache-lmdb.ts:90:24

  - EntitiesRepo.js:116 EntitiesRepo._callee$
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:116:30

  - regeneratorRuntime.js:44 tryCatch
    [md_gatsby5]/[@babel]/runtime/helpers/regeneratorRuntime.js:44:17

  - regeneratorRuntime.js:125 Generator.<anonymous>
    [md_gatsby5]/[@babel]/runtime/helpers/regeneratorRuntime.js:125:22

  - regeneratorRuntime.js:69 Generator.next
    [md_gatsby5]/[@babel]/runtime/helpers/regeneratorRuntime.js:69:21

  - EntitiesRepo.js:28 asyncGeneratorStep
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:28:103

  - EntitiesRepo.js:30 _next
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:30:194

  - EntitiesRepo.js:30 
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:30:364

  - new Promise

  - EntitiesRepo.js:30 EntitiesRepo.<anonymous>
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:30:97

  - EntitiesRepo.js:143 EntitiesRepo.saveStateToCache
    [md_gatsby5]/[datocms-client]/lib/local/EntitiesRepo.js:143:34

  - Loader.js:90 Loader.saveStateToCache
    [md_gatsby5]/[datocms-client]/lib/local/Loader.js:90:32

  - index.js:102 _callee$
    [md_gatsby5]/[gatsby-source-datocms]/hooks/createSchemaCustomization/index.js:102:27

  - regeneratorRuntime.js:44 tryCatch
    [md_gatsby5]/[@babel]/runtime/helpers/regeneratorRuntime.js:44:17

Comments

I previously thought this was due to the key is using JSON.stringify JSON-stringify of multiple properties.

And, according to the official Gatsby recommendation for plugins, keys should be created with createNodeId, as mentioned here:

https://github.com/gatsbyjs/gatsby/discussions/33887#discussioncomment-1714468

  cacheKey() {
    return createNodeId( `datocms-loader-data-${JSON.stringify({
      clientArgs: this.clientArgs,
      environment: this.environment,
      previewMode: this.previewMode,
      pageSize: this.pageSize,
    })}`);
  }

So the key becomes an UUIDv5 and not a JSON.stringify of multiple properties.

But it seems like its not the because of the key length itself, but maybe another length? Or duplication of keys, not sure what the original problem is referring to.

[arcataroger]

@corgrath,

Are you able to make a reproduction for this, please (like using a simple demo project and example code) or at least share with us some detailed context (your project URL, which queries have this error, etc.)?

We can triage and investigate this in more detail then!

(However, please also be aware that this repo is in maintenance mode, and we will only be providing critical bug fixes. If it's at all feasible for your team, you may wish to consider migrating to gatsby-source-graphql eventually, or moving away from Gatsby altogether...)

[corgrath]

Hey, I don't think its possible to create an mini project - its probably a combination of a lot of things.

Not sure what to do here, or if this is already a dead end.

[arcataroger]

What about just the snippet of code around this error, like whatever function you're calling and the data you're sending into it? We just need enough to be able to reproduce the error. Or is even that too complicated?

[matjack1]

hey @corgrath we have checked internally a bit more and we have never seen this problem before, so we are wondering if it's maybe some model/env/record name very long that you might be able to change? Can you help us debugging a bit by adding some console.log statements?

Otherwise you need to share with us your project and code. For that please write directly at [email protected]