payload icon indicating copy to clipboard operation
payload copied to clipboard

Field "id" is invalid when updating search records

Open TimLanzi opened this issue 3 months ago • 3 comments

Describe the Bug

In experimenting with the website template for 3.0, I found that the following code in /src/search/beforeSync.ts doesn't work as expected:


  if (categories && Array.isArray(categories) && categories.length > 0) {
    // get full categories and keep a flattened copy of their most important properties
    try {
      const mappedCategories = categories.map((category) => {
        const { id, title } = category

        return {
          relationTo: 'categories',
          id,
          title,
        }
      })

      modifiedDoc.categories = mappedCategories
    } catch (err) {
      console.error(
        `Failed. Category not found when syncing collection '${collection}' with id: '${id}' to search.`,
      )
    }
  }

After updating a document, the search result categories will look like this:

Image

I'm using postgres for my database, and the id looks like a mongo id. Plus, the title isn't saved. Logging the categories from the originalDoc yields an array of numbers (i.e. [3, 1]). So I figured I'd need to brute force the titles and correct ids into the record. So I updated the code as follows:

  if (categories && Array.isArray(categories) && categories.length > 0) {
    // get full categories and keep a flattened copy of their most important properties
    try {
      const populatedCategories = await payload.find({
        collection: "categories",
        where: { id: { in: categories } },
        pagination: false,
      });

      const mappedCategories = populatedCategories.docs.map((category) => {
        const { id, title } = category

        return {
          relationTo: 'categories',
          id,
          title,
        }
      })

      modifiedDoc.categories = mappedCategories
    } catch (err) {
      console.error(
        `Failed. Category not found when syncing collection '${collection}' with id: '${id}' to search.`,
      )
    }
  }

While not ideal, it did the job. I couldn't find a better way in the docs. However, this came with a new issue. If 2 posts use the same category, I get the following error:

[13:11:02] ERROR: Error updating search document.
    err: {
      "type": "ValidationError",
      "message": "The following field is invalid: id",
      "stack":
          ValidationError: The following field is invalid: id
              at upsertRow (webpack-internal:///(rsc)/./node_modules/@payloadcms/drizzle/dist/upsertRow/index.js:345:19)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async Object.updateOne (webpack-internal:///(rsc)/./node_modules/@payloadcms/drizzle/dist/update.js:49:20)
              at async updateByIDOperation (webpack-internal:///(rsc)/./node_modules/payload/dist/collections/operations/updateByID.js:279:22)
              at async syncWithSearch (webpack-internal:///(rsc)/./node_modules/@payloadcms/plugin-search/dist/Search/hooks/syncWithSearch.js:108:29)
              at async hooks.afterChange (webpack-internal:///(rsc)/./node_modules/@payloadcms/plugin-search/dist/index.js:37:37)
              at async eval (webpack-internal:///(rsc)/./node_modules/payload/dist/collections/operations/updateByID.js:353:22)
              at async updateByIDOperation (webpack-internal:///(rsc)/./node_modules/payload/dist/collections/operations/updateByID.js:351:9)
              at async Object.updateByID (webpack-internal:///(rsc)/./node_modules/@payloadcms/next/dist/routes/rest/collections/updateByID.js:37:15)
              at async eval (webpack-internal:///(rsc)/./node_modules/@payloadcms/next/dist/routes/rest/index.js:760:19)
              at async AppRouteRouteModule.do (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:10:33313)
              at async AppRouteRouteModule.handle (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:10:40382)
              at async doRender (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:1455:42)
              at async responseGenerator (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:1814:28)
              at async DevServer.renderToResponseWithComponentsImpl (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:1824:28)
              at async DevServer.renderPageComponent (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:2240:24)
              at async DevServer.renderToResponseImpl (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:2278:32)
              at async DevServer.pipeImpl (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:960:25)
              at async NextNodeServer.handleCatchallRenderRequest (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/next-server.js:281:17)
              at async DevServer.handleRequestImpl (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/base-server.js:853:17)
              at async /home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/dev/next-dev-server.js:373:20
              at async Span.traceAsyncFn (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/trace/trace.js:153:20)
              at async DevServer.handleRequest (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/dev/next-dev-server.js:370:24)
              at async invokeRender (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/lib/router-server.js:183:21)
              at async handleRequest (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/lib/router-server.js:360:24)
              at async requestHandlerImpl (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/lib/router-server.js:384:13)
              at async Server.requestListener (/home/tlanzi/Projects/Test/search-delete-bug-postgres/node_modules/next/dist/server/lib/start-server.js:142:13)
      "data": {
        "id": 4,
        "errors": [
          {
            "message": "Value must be unique",
            "path": "id"
          }
        ]
      },
      "isOperational": true,
      "isPublic": false,
      "status": 400,
      "name": "ValidationError"
    }

I've tried this in a fresh instance of the website template using postgres as well as another project where I have a "Resource" collection with "Category", "Tag", and "Source" related collections used in the same way. This bug appears if there is ever any overlap on relations. Sometimes I'll need to add the category, save, make another edit, and save again before the error shows up.

Link to the code that reproduces this issue

https://github.com/TimLanzi/payload-search-relationship-id-bug

Reproduction Steps

  1. Use create-payload-app choose website template and postgres database
  2. Modify /src/search/beforeSync.ts as follows:
  if (categories && Array.isArray(categories) && categories.length > 0) {
    // get full categories and keep a flattened copy of their most important properties
    try {
      const populatedCategories = await payload.find({
        collection: "categories",
        where: { id: { in: categories } },
        pagination: false,
      });

      const mappedCategories = populatedCategories.docs.map((category) => {
        const { id, title } = category

        return {
          relationTo: 'categories',
          id,
          title,
        }
      })

      modifiedDoc.categories = mappedCategories
    } catch (err) {
      console.error(
        `Failed. Category not found when syncing collection '${collection}' with id: '${id}' to search.`,
      )
    }
  }

  1. Create account and seed database
  2. Give post "Dollar and Sense" a category (I chose Technology)
  3. Give post "Global Gaze" the same category. You should see the error. If not, make another arbitrary edit to the document, save, and you should see it now.

Which area(s) are affected? (Select all that apply)

plugin: search

Environment Info

Binaries:
  Node: 22.7.0
  npm: 10.8.2
  Yarn: 1.22.22
  pnpm: 9.8.0
Relevant Packages:
  payload: 3.1.0
  next: 15.0.3
  @payloadcms/db-postgres: 3.1.0
  @payloadcms/email-nodemailer: 3.1.0
  @payloadcms/graphql: 3.1.0
  @payloadcms/live-preview: 3.1.0
  @payloadcms/live-preview-react: 3.1.0
  @payloadcms/next/utilities: 3.1.0
  @payloadcms/payload-cloud: 3.1.0
  @payloadcms/plugin-form-builder: 3.1.0
  @payloadcms/plugin-nested-docs: 3.1.0
  @payloadcms/plugin-redirects: 3.1.0
  @payloadcms/plugin-search: 3.1.0
  @payloadcms/plugin-seo: 3.1.0
  @payloadcms/richtext-lexical: 3.1.0
  @payloadcms/translations: 3.1.0
  @payloadcms/ui/shared: 3.1.0
  react: 19.0.0-rc-65a56d0e-20241020
  react-dom: 19.0.0-rc-65a56d0e-20241020
Operating System:
  Platform: linux
  Arch: x64
  Version: #202405300957~1732141768~22.04~f2697e1 SMP PREEMPT_DYNAMIC Wed N
  Available memory (MB): 31971
  Available CPU cores: 8

TimLanzi avatar Nov 22 '24 18:11 TimLanzi