nuxt-generate-cluster icon indicating copy to clipboard operation
nuxt-generate-cluster copied to clipboard

Assertion failed: len < MAX_ENV_VAR_LENGTH

Open pimlie opened this issue 5 years ago • 11 comments

The master sends the nuxt.config through an env var to the worker. On windows there appears to be a limit of passing env vars of 32KB (possibly since node v12.10), if you exceed that limit libuv will throw an error.

Possible fixes:

  • zip the env
  • use disk
This question is available on Nuxt community (#c13)

pimlie avatar Jan 16 '20 23:01 pimlie

serverMiddleware serialization will cause generate workers to fail. as solution -> exclude config key customization

aldarund avatar Jan 17 '20 00:01 aldarund

v2.7.0 includes a beforeWorkers hook you can use to exclude keys

pimlie avatar Jan 17 '20 11:01 pimlie

how many urls can nuxt-generate-cluster process at a time? breaks over 700 for me

lxsedov avatar Jan 28 '20 05:01 lxsedov

@lxsedov Are you on Windows? The limit talked about in this issue is caused by the nuxt.config that is passed from the master to worker. If your nuxt.config is big then you could experience this issue. The solution is to use the beforeWorkers hook to remove properties from your nuxt.config that the workers dont need. See the docs

pimlie avatar Jan 28 '20 09:01 pimlie

@pimlie on Windows it dies with 700 URLs, on Ubuntu after 5000. Is there a way to bypass the limitation of env var size? beforeWorkers doesn't really help when you need to use nuxt-generate for large amount of dynamic URLs.

lxsedov avatar Jan 29 '20 00:01 lxsedov

@lxsedov Are you listing 5000 urls directly in your nuxt.config's generate.routes? If so, you can remove that property in beforeWorkers, the workers dont need that. The master needs it and uses it to distribute smaller route chunks to workers. But really, give more information about what you are trying to do because what you are saying doesnt make that much sense to me so now I have to guess what you are trying to do

pimlie avatar Jan 29 '20 09:01 pimlie

basically i want to generate 2-3M pages based on dynamic routes; removing routes in beforeWorkers helped but now I get 'No routes so not starting workers' passing 125000 routes

lxsedov avatar Feb 21 '20 01:02 lxsedov

@lxsedov Could you please share your full nuxt config? If thats not possible, at least the generate part?

pimlie avatar Feb 23 '20 12:02 pimlie

https://github.com/lxsedov/test/blob/master/nuxt-config.js; tried with completely fresh project no modules no plugins, passing 150k dynamic routes it fails with 'No routes so not starting workers'; win10 node10.16.0

lxsedov avatar Feb 24 '20 00:02 lxsedov

Maybe try to clone the options first? You are probably now deleting the reference on the options in the master too

pimlie avatar Feb 24 '20 09:02 pimlie

It exits with 'No routes...' even before getting to 'beforeWorkers'

lxsedov avatar Feb 24 '20 19:02 lxsedov