sitemap-module icon indicating copy to clipboard operation
sitemap-module copied to clipboard

npm run build sitemap.xml create not found

Open todaynogada opened this issue 2 years ago • 25 comments

In npm run dev state, sitemap.xml is created, but in npm run start state after npm run build, sitemap.xml is not created.

Is /sitemap.xml normal after npm run dev command? After npm run build and after npm run start, sitemap.xml 404 error is returned. I hope it gets patched soon.

Currently, it is temporarily used as https://github.com/benoitdemaegdt/nuxt3-sitemap.

npm 8.5.0 Node 16.14.2 nuxtjs3 v3.0.0-rc.1

todaynogada avatar Apr 26 '22 03:04 todaynogada

Nuxt 3 removed the addServerMiddleware function (but for whatever reason only on production) see https://v3.nuxtjs.org/api/advanced/kit
The problem is "sitemap-module" uses this function from Nuxt 2 to intercept any incoming requests on /sitemap.xml etc. and returning the xml file.

I am currently working on getting this sitemap-module running on a project with Nuxt 3. If it works out , I may post it here or open a PR.

Basicly you need to use the new server middleware which doesn't have access to the nuxt instance which makes things a bit more complicated but possible.

d3xter-dev avatar May 17 '22 17:05 d3xter-dev

also see https://github.com/nuxt/nuxt.js/issues/13559

d3xter-dev avatar May 17 '22 17:05 d3xter-dev

So I kinda did it :partying_face:, at least a simple version supporting a single sitemap.xml (no sitemapindex.xml etc)

If there are enough people who need a full working version, I would offer to create a Nuxt 3 Version of this module.

I hope that helps someone :smile:

nuxt.config.ts

modules: [
    // ....
    '@nuxtjs/sitemap',
    '~/modules/sitemap/module.js',
  ],

/modules/sitemap/module.js

import { defineNuxtModule } from '@nuxt/kit'

export default defineNuxtModule({
  setup(options, nuxt) {
    /**
     * remove default middleware so our works on dev as well
     */
    nuxt.options.serverMiddleware = nuxt.options.serverMiddleware.filter((mw) => !mw.path.includes('sitemap.xml'))

    /**
     * Use addServerHandler to dynamicly add server middlewares in Nuxt 3
     */
    // addServerHandler({
    //   handler: '/path/to/handler',
    // })

    /**
     * pass routes to nitro runtime config
     */
    nuxt.hook('nitro:build:before', (nitro) => {
      nitro.options.runtimeConfig.sitemap = nuxt.options.runtimeConfig.sitemap
    })

    /**
     * Get all static routes and remove dynamic ones
     */
    nuxt.hook('pages:extend', (routes) => {
      nuxt.options.sitemap.staticRoutes = routes
        .map((r) => r.path)
        .filter((path) => ![':', '*'].some((c) => path.includes(c)))
      nuxt.options.runtimeConfig.sitemap = nuxt.options.sitemap
    })
  },
})

/server/middleware/sitemap.ts

import { defineEventHandler } from 'h3'
import { excludeRoutes } from '@nuxtjs/sitemap/lib/routes.js'
import { createRoutesCache } from '@nuxtjs/sitemap/lib/cache.js'
import { createSitemap } from '@nuxtjs/sitemap/lib/builder.js'
import generateETag from 'etag'
import fresh from 'fresh'

export const globalCache = { staticRoutes: null, routes: null }

const defaults = {
  path: '/sitemap.xml',
  hostname: undefined,
  exclude: [],
  routes: [],
  cacheTime: 1000 * 60 * 15,
  etag: false,
  filter: undefined,
  gzip: false,
  xmlNs: undefined,
  xslUrl: undefined,
  trailingSlash: false,
  lastmod: undefined,
  i18n: undefined,
  defaults: {},
}

export default defineEventHandler(async (event) => {
  const config = useRuntimeConfig()
  const sitemap = { ...defaults, ...config.sitemap }

  if ([sitemap.pathGzip, sitemap.path].includes(event.req.url)) {
    const generateDynamicRoutes = async (slug) => {
      console.log('[vue-sitemap] generate dynamic routes')

      const routes = []
      // make requests to an API or something and build an array of routes

      return routes
    }

    /**
     * Init cache if not active yet
     */
    if (!globalCache.staticRoutes) {
      /**
       * add dynamic routes
       */
      sitemap.routes = async () => [
        ...(await generateDynamicRoutes('news')),
      ]

      globalCache.staticRoutes = () => excludeRoutes(sitemap.exclude, sitemap.staticRoutes)
      globalCache.routes = createRoutesCache(globalCache, sitemap)
    }

    if (sitemap.gzip) {
      try {
        // Init sitemap
        const routes = await globalCache.routes.get('routes')
        const gzip = await createSitemap(sitemap, routes, '/', event.req).toGzip()
        // Check cache headers
        if (validHttpCache(gzip, sitemap.etag, event.req, event.res)) {
          return
        }
        // Send http response
        event.res.setHeader('Content-Type', 'application/gzip')
        return event.res.end(gzip)
      } catch (err) {
        return err
      }
    }

    try {
      // Init sitemap
      const routes = await globalCache.routes.get('routes')
      const xml = await createSitemap(sitemap, routes, '/', event.req).toXML()
      // Check cache headers
      if (validHttpCache(xml, sitemap.etag, event.req, event.res)) {
        return
      }

      // Send http response
      event.res.setHeader('Content-Type', 'application/xml')
      return xml
    } catch (err) {
      return err
    }
  }
})

function validHttpCache(entity, options, req, res) {
  if (!options) {
    return false
  }
  const { hash } = options
  const etag = hash ? hash(entity, options) : generateETag(entity, options)
  if (fresh(req.headers, { etag })) {
    res.statusCode = 304
    res.end()
    return true
  }
  // Add ETag header
  res.setHeader('ETag', etag)
  return false
}

d3xter-dev avatar May 18 '22 11:05 d3xter-dev

I guess I'm missing something. I get the following error when going to /sitemap.xml:

[nuxt] [request error] Cannot read properties of undefined (reading 'map')
  at createError (./node_modules/h3/dist/index.mjs:185:15)  
  at ./node_modules/h3/dist/index.mjs:430:17  
  at processTicksAndRejections (node:internal/process/task_queues:96:5)  
  at async Server.nodeHandler (./node_modules/h3/dist/index.mjs:367:7)
[nitro] Error while generating error response FetchError: 404  (/__nuxt_error?url=/sitemap.xml&statusCode=500&statusMessage=H3Error&message=Cannot+read+p
roperties+of+undefined+(reading+'map')&description=%3Cpre%3E%3Cspan+class=%22stack+internal%22%3Eat+createError+(./node_modules/h3/dist/index.mjs:185:15)
%3C/span%3E%0A%3Cspan+class=%22stack+internal%22%3Eat+./node_modules/h3/dist/index.mjs:430:17%3C/span%3E%0A%3Cspan+class=%22stack+internal%22%3Eat+proces
sTicksAndRejections+(node:internal/process/task_queues:96:5)%3C/span%3E%0A%3Cspan+class=%22stack+internal%22%3Eat+async+Server.nodeHandler+(./node_modules/h3/dist/index.mjs:367:7)%3C/span%3E%3C/pre%3E)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.errorhandler [as onError] (file:///home/ljansen/......./.nuxt/dev/index.mjs:311:16)
    at async Server.nodeHandler (file:///home/ljansen/........../node_modules/h3/dist/index.mjs:370:9)

What am I doing wrong?

I added this to nuxt.config.ts (in addition to the things you posted above):

sitemap: {
        hostname: 'https://www.ljpc.nl',
        gzip: true,
        path: '/sitemap.xml',
        routes: [],
        defaults: {
            changefreq: 'daily',
            priority: 1,
            lastmod: new Date()
        }
    }

Lars- avatar May 19 '22 15:05 Lars-

sitemap.staticRoutes doesn't seem to be set in the sitemap.ts?
is the nuxt module loaded correctly ?

try console.log in the sitemap.ts

  const config = useRuntimeConfig()
  const sitemap = { ...defaults, ...config.sitemap }
  console.log(sitemap)

and in the module.js

    nuxt.hook('pages:extend', (routes) => { 
        console.log('adding static routes to nitro config....')
        ......
     })

static routes are only set once during build time so if you add a new static page you need to restart the server e.g. yarn dev / yarn build && yarn start

d3xter-dev avatar May 20 '22 12:05 d3xter-dev

@d3xter-dev

Followed your steps but getting undefined errors on nuxt.options.serverMiddleware?

Edit: Error appears to no longer be occurring.

scottingle avatar May 23 '22 12:05 scottingle

@d3xter-dev I do see the "adding static routes to nitro config" console log. And on every page load I see this logged:

{
  path: '/sitemap.xml',
  hostname: undefined,
  exclude: [],
  routes: [],
  cacheTime: 900000,
  etag: false,
  filter: undefined,
  gzip: false,
  xmlNs: undefined,
  xslUrl: undefined,
  trailingSlash: false,
  lastmod: undefined,
  i18n: undefined,
  defaults: {}
}

Lars- avatar May 23 '22 12:05 Lars-

@Lars- yeah the staticRotues are missing so, something seems to be wrong in the modules.js does routes in pages:extend contain all your routes? and does the nuxt.options.runtimeConfig.sitemap in nuxt.hook('nitro:build:before', (nitro) include a prop staticRoutes with all your routes?

@scottingle you could try nuxt.options.serverMiddleware = nuxt.options.serverMiddleware.filter((mw) => mw?.path && !mw.path.includes('sitemap.xml')) ?

d3xter-dev avatar May 23 '22 12:05 d3xter-dev

I'll check later. It takes a bit too much time for now unfortunately. But I really do appreciate the replies. If I know more, I'll let you know.

Lars- avatar May 23 '22 13:05 Lars-

Would this work with dynamic routes?

scottingle avatar May 23 '22 16:05 scottingle

@scottingle nope, they should be excluded thats what's the

 nuxt.options.sitemap.staticRoutes = routes
        .map((r) => r.path)
        .filter((path) => ![':', '*'].some((c) => path.includes(c)))

is for, dynamic routes need to be added in the sitemap.ts if we take product for example I would do a request to my endpoint where I get the products from and take alle the ids I get to build an array of routes which we can cache then see

      /**
       * add dynamic routes
       */
      sitemap.routes = async () => [
        ...(await generateDynamicRoutes('products')),
      ]
      ....

d3xter-dev avatar May 23 '22 16:05 d3xter-dev

Hey @d3xter-dev, thanks for providing a solution for Nuxt 3!

Does this support Nuxt Content 2?

heychazza avatar May 23 '22 20:05 heychazza

@heychazza not sure what you mean by that

d3xter-dev avatar May 24 '22 09:05 d3xter-dev

@heychazza not sure what you mean by that

Does this support the new version of Nuxt Content?

heychazza avatar May 24 '22 09:05 heychazza

Thank you @d3xter-dev for helping make this plugin work with V3 ! Can you confirm me that this fix doesn't work with a sitemapindex ? I'm using an index of sitemaps on my app and waiting to switch to V3.

desaintflorent avatar May 24 '22 10:05 desaintflorent

@heychazza not sure what you mean by that

Does this support the new version of Nuxt Content?

@heychazza ah that's another nuxt module I see, should be fine as long as Nuxt Content loads before the module.js I posted above and pages:extends contains all routes added by Nuxt Content (just add a console.log to check, only runs once per build)

@desaintflorent yeah you would have to add support for a sitemapindex.xml in the middleware or create another one by copying + adapting following code: https://github.com/nuxt-community/sitemap-module/blob/0bc0ebdd6b560d288cc1e0e259298ddbffc47ef3/lib/middleware.js#L124-L167

d3xter-dev avatar May 24 '22 11:05 d3xter-dev

I would greatly appreciate a nuxt3 module for this :d, and sooner than later, thousands of devs.

fskarmeta avatar Jun 11 '22 21:06 fskarmeta

@d3xter-dev Pog. Could you provide a Repository to contribute on? Seems the next-sitemap guys are kinda inactive

ROBJkE avatar Jun 13 '22 22:06 ROBJkE

is this still not available in nuxt 3?

jmdmacapagal avatar Jun 14 '22 14:06 jmdmacapagal

We decided that we'll start working on porting the whole module to Nuxt 3 in the next few days :smile:
Link to our repo https://github.com/funken-studio/sitemap-module-nuxt-3

d3xter-dev avatar Jun 16 '22 07:06 d3xter-dev

wow thank you @d3xter-dev looking forward to it!

jmdmacapagal avatar Jun 17 '22 09:06 jmdmacapagal

Damn seems nuxt content isn't yet supported by this :(

@heychazza not sure what you mean by that

Does this support the new version of Nuxt Content?

@heychazza ah that's another nuxt module I see, should be fine as long as Nuxt Content loads before the module.js I posted above and pages:extends contains all routes added by Nuxt Content (just add a console.log to check, only runs once per build)

@desaintflorent yeah you would have to add support for a sitemapindex.xml in the middleware or create another one by copying + adapting following code:

https://github.com/nuxt-community/sitemap-module/blob/0bc0ebdd6b560d288cc1e0e259298ddbffc47ef3/lib/middleware.js#L124-L167 CleanShot 2022-06-18 at 9 59 30@2x

heychazza avatar Jun 18 '22 08:06 heychazza

I am happy to announce :partying_face: We finally finished a first version for Nuxt 3, feel free to check it out. :tada:

yarn add @funken-studio/sitemap-nuxt-3

https://github.com/funken-studio/sitemap-module-nuxt-3/tree/experimental https://www.npmjs.com/package/@funken-studio/sitemap-nuxt-3

Please give me feedback :)

d3xter-dev avatar Jun 28 '22 18:06 d3xter-dev

@d3xter-dev thanks for your module '~/modules/sitemap/module.js', it works fine with server build, but with 'nuxt generate' map file is not generated in output dir, what could be the problem?

mak33v avatar Aug 14 '22 19:08 mak33v

Damn seems nuxt content isn't yet supported by this :(

@heychazza not sure what you mean by that

Does this support the new version of Nuxt Content?

@heychazza ah that's another nuxt module I see, should be fine as long as Nuxt Content loads before the module.js I posted above and pages:extends contains all routes added by Nuxt Content (just add a console.log to check, only runs once per build) @desaintflorent yeah you would have to add support for a sitemapindex.xml in the middleware or create another one by copying + adapting following code: https://github.com/nuxt-community/sitemap-module/blob/0bc0ebdd6b560d288cc1e0e259298ddbffc47ef3/lib/middleware.js#L124-L167

CleanShot 2022-06-18 at 9 59 30@2x

Yeah, you're correct. I'm also facing the same issue for nuxt content :(

Neilblaze avatar Apr 01 '23 08:04 Neilblaze