feat: add path field to nested docs plugin

[Livog]

Description

I propose the introduction of a new 'path' field for the nested docs plugin, that enables developers to query documents across multiple collections using a single field. This field can be configured as either unique or non-unique across collections and is opt-in to suit specialized needs.

This feature emerged from the need for a simpler way to query documents, as well as the limitations encountered with the 'breadcrumbs' field, particularly when trying to query paths with repeated segments like /pages/pages/page and when querying against the breadcrumbs array proved challenging. To address this, the 'path' field allows to query directly against 1 collection using 1 field or creating a custom endpoint that query two collections simultaneously. This feature is especially useful when the same 'content' field, such as blocks or content (Rich Text), is used across multiple collections to display content.

Here is an integration example with a Next.js project using the App Router:

// app/[[...path]]/page.tsx
const { docs } = await payload.find({
    collection: 'pages',
    where: { path: { equals: path } },
    depth: 3
});
const page = docs?.at(0) || null

This setup highlights the ease of fetching your content in a single dynamic route, adding onto a simpler developer experience.

For context, here is an example of the current method of querying documents using the 'breadcrumbs' field:

const results = await payload.db.collections['pages'].aggregate([
    {
      $match: {
        'breadcrumbs.url': path
      }
    },
    {
      $addFields: {
        lastBreadcrumb: { $arrayElemAt: ['$breadcrumbs', -1] }
      }
    },
    {
      $match: {
        'lastBreadcrumb.url': path
      }
    }
]);

While effective, the current method using 'breadcrumbs' does not allow for preemptive blocking of the save action if the prospective path conflicts with or already exists. The new 'path' field addresses this issue by enabling conflict checks before saving, ensuring unique navigation paths. Additionally, if you use the breadcrumbs way of querying the developers often need to query twice to utilize Payload's built-in depth feature for fetching relational fields.

The introduction of the 'path' field not only simplifies querying across collections but should also be enhancing link management with the lexical LinkFeature as you can use the path field.

• [x] I have read and understand the CONTRIBUTING.md document in this repository.

Type of change

• [x] New feature (non-breaking change which adds functionality)
• [x] This change requires a documentation update

Checklist:

• [ ] I have added tests that prove my fix is effective or that my feature works
• [x] Existing test suite passes locally with my changes
• [x] I have made corresponding changes to the documentation

[jmikrut]

Hey @Livog we'll take a look at this ASAP and get back to you!

[stefvw93]

Any updates on this?

[magicspon]

@jmikrut any news on this? I'm struggling to work out how we're meant to query nested docs. Thanks, having a great time playing with payload.

[Livog]

I believe this solution is quite outdated at the moment, and I cannot speak to whether there is interest in adding it to the Payload core if I were to fix it. Still, I'd like to share my current approach for anyone who needs a straightforward way to query their documents.

Here’s my pathField implementation for anyone who wants to include it in their project:
https://gist.github.com/Livog/1c4673b264158d367ab2b423aacd7f4e

The main drawback is that I must query the database for each collection in the PATH_UNIQUE_AGINST_COLLECTIONS constant. Even so, I personally find path more appealing than using collection + slug because it offers a flexible way to set up fully custom routes.

[magicspon]

thanks @Livog. IMHO, nested docs is kinda unusable if you can't query for nested docs!

@jmikrut if i have two pages a/demo and b/demo how am i meant to target these paths?

I presumed i could do something like:

where: {
  slug: {
    equals: 'demo'
  },
  and: {
    parent: {
      slug: {
        equals: 'a'
      }
    }
  }
}

or even better, what @Livog has illustrated with this PR

[notflip]

My code seems to be working, or am I missing something? (I've only just installed the nested-docs-plugin. Every page now has the breadcrumbs array, which I check based on the slug array (catch-all route in nextjs)

[...slug].tsx

  const slug = (await params).slug;
  const slugArray = Array.isArray(slug) ? slug : [slug];

  const payload = await getPayload({
    config,
  });

  const result = await payload.find({
    collection: "pages",
    where: {
      "breadcrumbs.url": { equals: `/${slugArray.join("/")}` },
    },
  });

  const docs = result.docs;

To fetch a full link in the frontend, use

  const href =
    type === "reference" &&
    typeof reference?.value === "object" &&
    Array.isArray(reference.value.breadcrumbs) &&
    reference.value.breadcrumbs.length > 0
      ? reference.value.breadcrumbs[reference.value.breadcrumbs.length - 1].url // Use the last breadcrumb URL
      : null;

[magicspon]

thanks @notflip i'll have to try this... if this works, it needs to be documented.

[Livog]

Hey @notflip, I actually tried your approach before moving to a path-based field. The main issue I ran into was keeping things unique and double and more-nested docs. For example, I need to allow both /contact/legal and /about/legal with the same slug, but if someone tries creating another /about/legal again, it should conflict and throw. Another potential issue is: if you have 100 nested pages under /blog then rendering the /blog page your query would in theory also return all of the nested 100 pages under /blog as well and then you would need to loop over them to find where breadcrumbs last index is /blog.

Or am I missing something here?

[notflip]

Hey @notflip, I actually tried your approach before moving to a path-based field. The main issue I ran into was keeping things unique and double and more-nested docs. For example, I need to allow both /contact/legal and /about/legal with the same slug, but if someone tries creating another /about/legal again, it should conflict and throw. Another potential issue is: if you have 100 nested pages under /blog then rendering the /blog page your query would in theory also return all of the nested 100 pages under /blog as well and then you would need to loop over them to find where breadcrumbs last index is /blog.

Or am I missing something here?

@Livog Hm you're right, when fetching /blog I'm getting 3 results (so each subpage is also included). I'm not sure if this is a big performance impact or not? Regarding the unique validation, I hadn't thought of that as well. Does your PR work with validation/unique and not querying too deep?

[Livog]

Hey @notflip, I actually tried your approach before moving to a path-based field. The main issue I ran into was keeping things unique and double and more-nested docs. For example, I need to allow both /contact/legal and /about/legal with the same slug, but if someone tries creating another /about/legal again, it should conflict and throw. Another potential issue is: if you have 100 nested pages under /blog then rendering the /blog page your query would in theory also return all of the nested 100 pages under /blog as well and then you would need to loop over them to find where breadcrumbs last index is /blog. Or am I missing something here?

@Livog Hm you're right, when fetching /blog I'm getting 3 results (so each subpage is also included). I'm not sure if this is a big performance impact or not? Regarding the unique validation, I hadn't thought of that as well. Does your PR work with validation/unique and not querying too deep?

I don’t see it as a performance concern; however, exceeding 1000 posts without setting the limit property above 1000 can trigger an unexpected 404 error. The path field is meant to be unique across collections, serving as a publicly queryable identifier. It should throw an error if you attempt to create a nested path that already exists.