Bug: runSeeders throws with: The "path" argument is undefined

[snebjorn]

Versions

• Node: Node 18
• OS: AWS lambda

Reproduction

When executing runSeeders inside an AWS lambda function the following error is thrown:

{
  "errorType": "TypeError",
  "errorMessage": "The \"path\" argument must be of type string or an instance of URL. Received undefined",
  "trace": [
    "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined",
    "    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)",
    "    at new NodeError (node:internal/errors:405:5)",
    "    at Object.fileURLToPath (node:internal/url:1401:11)",
    "    at Object.<anonymous> (/node_modules/locter/node_modules/glob/src/index.ts:218:13)",
    "    at Module._compile (node:internal/modules/cjs/loader:1256:14)",
    "    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)",
    "    at Module.load (node:internal/modules/cjs/loader:1119:32)",
    "    at Module._load (node:internal/modules/cjs/loader:960:12)",
    "    at Module.require (node:internal/modules/cjs/loader:1143:19)",
    "    at require (node:internal/modules/cjs/helpers:121:18)",
    "    at _tryRequire (file:///var/runtime/index.mjs:1039:14)",
    "    at async _loadUserApp (file:///var/runtime/index.mjs:1055:16)",
    "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:21)",
    "    at async start (file:///var/runtime/index.mjs:1256:23)",
    "    at async file:///var/runtime/index.mjs:1262:1"
  ]
}

I must admit I'm at a loss here. I have no idea why this is failing in an AWS lambda context.

I can run the seed just fine via typeorm-extension/bin/cli.cjs (not using the lambdaHandler)

Additional Details

Lambda function:

import { runSeeders } from 'typeorm-extension';
import { AppDataSource } from './myDataSource';

export const lambdaHandler = async () => {
    if (!AppDataSource.isInitialized) {
        await AppDataSource.initialize();
    }

    await runSeeders(AppDataSource);

    return { statusCode: 200, body: 'Seeding completed successfully.' };
};

// myDataSource
export const AppDataSourceOptions: DataSourceOptions & SeederOptions = {
    type: 'postgres',
    ...
    // the error persists with or without seeds and factories props, so I'm thinking it's unrelated
}

export const AppDataSource = new DataSource(AppDataSourceOptions);

Steps to reproduce

1. Clone https://github.com/snebjorn/typeorm-ext-seed
2. Install Node, SAM CLI and Docker as described in the README
3. in the root of the repo run:
   • sam build
   • docker compose up
   • sam local invoke HelloWorldFunction
4. Observe the "path argument" error

What is Expected?

runSeeders should work.

What is actually happening?

$ sam local invoke HelloWorldFunction

Invoking app.lambdaHandler (nodejs18.x)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

Mounting ~\sam-app\.aws-sam\build\HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: d36f07db-c785-4381-af32-f6bb168c4730 Version: $LATEST
2023-08-24T08:26:25.873Z        undefined       ERROR   Uncaught Exception      {"errorType":"TypeError","errorMessage":"The \"path\" argument must be of type string or an instance of URL. Received undefined","code":"ERR_INVALID_ARG_TYPE","stack":["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined","    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)","    at new NodeError (node:internal/errors:405:5)","    at Object.fileURLToPath (node:internal/url:1401:11)","    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)","    at Module._compile (node:internal/modules/cjs/loader:1256:14)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at Module.require (node:internal/modules/cjs/loader:1143:19)","    at require (node:internal/modules/cjs/helpers:121:18)","    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)","    at _tryRequire (file:///var/runtime/index.mjs:1026:25)","    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)","    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)","    at start (file:///var/runtime/index.mjs:1256:42)","    at file:///var/runtime/index.mjs:1262:7","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}
24 Aug 2023 08:26:25,891 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 129 InvokeID=
2023-08-24T08:26:26.666Z        undefined       ERROR   Uncaught Exception      {"errorType":"TypeError","errorMessage":"The \"path\" argument must be of type string or an instance of URL. Received undefined","code":"ERR_INVALID_ARG_TYPE","stack":["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined","    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)","    at new NodeError (node:internal/errors:405:5)","    at Object.fileURLToPath (node:internal/url:1401:11)","    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)","    at Module._compile (node:internal/modules/cjs/loader:1256:14)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at Module.require (node:internal/modules/cjs/loader:1143:19)","    at require (node:internal/modules/cjs/helpers:121:18)","    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)","    at _tryRequire (file:///var/runtime/index.mjs:1026:25)","    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)","    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)","    at start (file:///var/runtime/index.mjs:1256:42)","    at file:///var/runtime/index.mjs:1262:7","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}
END RequestId: 3f1cbd2e-9f2e-4dbc-85e2-ce908b03c63e
REPORT RequestId: 3f1cbd2e-9f2e-4dbc-85e2-ce908b03c63e  Init Duration: 0.12 ms  Duration: 1606.12 ms    Billed Duration: 1607 ms        Memory Size: 512 MB     Max Memory Used: 512 MB
{"errorType": "TypeError", "errorMessage": "The \"path\" argument must be of type string or an instance of URL. Received undefined", "trace": ["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined", "    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)", "    at new NodeError (node:internal/errors:405:5)", "    at Object.fileURLToPath (node:internal/url:1401:11)", "    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)", "    at Module._compile (node:internal/modules/cjs/loader:1256:14)", "    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)", "    at Module.load (node:internal/modules/cjs/loader:1119:32)", "    at Module._load (node:internal/modules/cjs/loader:960:12)", "    at Module.require (node:internal/modules/cjs/loader:1143:19)", "    at require (node:internal/modules/cjs/helpers:121:18)", "    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)", "    at _tryRequire (file:///var/runtime/index.mjs:1026:25)", "    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)", "    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)", "    at start (file:///var/runtime/index.mjs:1256:42)", "    at file:///var/runtime/index.mjs:1262:7", "    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}

[snebjorn]

I managed to isolate the issue in a repo. Updated issue with details.

[tada5hi]

I think the problem is that, the import.meta.url property is not supported in AWS Lambda functions running in a Node.js environment. It also looks like the output conforms to esm syntax and not cjs syntax. Could you set module to CommonJS in your tsconfig and try again?

[snebjorn]

I set all the commonjs flags I could find and now I'm getting this

{
  "errorType": "TypeError",
  "errorMessage": "The \"path\" argument must be of type string or an instance of URL. Received undefined",
  "trace": [
    "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined",
    "    at new NodeError (node:internal/errors:405:5)",
    "    at Object.fileURLToPath (node:internal/url:1401:11)",
    "    at Object.<anonymous> (/var/task/app.js:164771:38)",
    "    at Module._compile (node:internal/modules/cjs/loader:1256:14)",
    "    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)",
    "    at Module.load (node:internal/modules/cjs/loader:1119:32)",
    "    at Module._load (node:internal/modules/cjs/loader:960:12)",
    "    at Module.require (node:internal/modules/cjs/loader:1143:19)",
    "    at require (node:internal/modules/cjs/helpers:121:18)",
    "    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)"
  ]
}

looks the same expect for the last async lines.

I unminified the compiled code and looked at app.js:164771. Which is this line of code:

// node_modules/locter/dist/index.mjs
var import_url3 = __toESM(require("url"), 1);
var import_meta = {};
var __filename = import_url3.default.fileURLToPath(import_meta.url);
// ----------------------------------^

Seems something is not right with importing meta. I'm guessing here. But I think it's trying to do import.meta.url in a commonjs context which isn't supported. There is a node_modules/locter/dist/index.cjs file so not sure why it (esbuild I assume) is taking the .mjs file.

[snebjorn]

Turns out that the package locter (used by typeorm-extension) doesn't work when built using esbuild.

locter uses import.meta.url which esbuild doesn't support.

There are many node-specific features that esbuild doesn't support while bundling such as __dirname, import.meta.url, fs.readFileSync, and *.node native binary modules. https://esbuild.github.io/getting-started/#bundling-for-node

There are many ways to "fix" this. Like use another package then locter . Not sure what is easiest.

https://github.com/evanw/esbuild/issues/3365

[tada5hi]

The problem is that even if we map the functionality in this library, import.meta.url is still used because we provide a bundle for cjs and esm. Esbuild should be able to resolve to the cjs file. But it kind of looks like based on your configuration an esm bundle is generated anyway and cjs imports are wrapped with toEsm.

[snebjorn]

I'm still waiting for a full explanation from esbuild. They told me that I have to use require to get the cjs file. That probably makes sense if writing raw JS, but I'm writing TS and I'm expecting my import to be converted to require if I'm compiling to commonjs. So I'm not sure why that isn't happening.

[tada5hi]

@snebjorn do you have found any solution in the meantime?

[snebjorn]

I'm still waiting for an explanation from esbuild or a discussion about this behavior.

I didn't test this. But I think a workaround would be to use require instead of import in the .ts files to get esbuild to import the commonjs code.

[snebjorn]

Closing this as there is no progress in https://github.com/evanw/esbuild/issues/3365

This is a niche issue that is becoming less and less relevant as we're all moving to ESM