antlr4 icon indicating copy to clipboard operation
antlr4 copied to clipboard

Esbuild fail to bundle from 4.10 to 4.12

Open richard-julien opened this issue 2 years ago • 18 comments

  • [x] I have reproduced my issue using the latest version of ANTLR
  • [x] I have done a search of the existing issues to make sure I'm not sending in a duplicate

Please include the following information:

  • target information: Javascript
  • smallest possible grammar and code that reproduces the behavior: N/A related only to code import
  • description of the expected behavior and actual behavior: No error bundling code with esbuild

Hi Antlr team,

I will try to expose my problem in a clearer way but sorry if information looks like partial. Difficult to explain correctly my problem with my current knowledge of ANTLR.

I try to upgrade ANTLR in my project (nodejs) from 4.10 to 4.12. My project use esbuild to bundle the source code. After upgrading to 4.12, the bundled code refuse to start with the error.

node:internal/modules/cjs/loader:1367
    throw new ERR_INVALID_ARG_VALUE('filename', filename, createRequireError);
    ^

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'filename' must be a file URL object, file URL string, or absolute path string. Received undefined

Looking into the bundled code, its looks like the esbuild transform the file and i have this line.

var ha = (0, import_module.createRequire)(import_meta.url)("fs");

And as the error said, the import_meta is empty initialized so url is undefined.

Looking into esbuild, the documentation said that import.meta.url is not supported when bundling:

You also may not want to bundle your dependencies with esbuild. There are many node-specific features that esbuild doesn't support while bundling such as __dirname, import.meta.url, fs.readFileSync, and *.node native binary modules. You can exclude all of your dependencies from the bundle by setting [packages](https://esbuild.github.io/api/#packages) to external:

So my question is, do you change something in ANTLR that now use the meta import feature that break the compatibility with esbuild bundling?

Thanks in advance.

richard-julien avatar Mar 28 '23 09:03 richard-julien

Yes the packaging changed a bit in 4.12 for TypeScript to work.

ericvergnaud avatar Mar 28 '23 14:03 ericvergnaud

Any solution/workaround available @ericvergnaud? Problem is still present in 4.13 and i dont know where to start to upgrade.

richard-julien avatar May 25 '23 23:05 richard-julien

@richard-julien How you build your app? If you could provide some small repo with your setup that reproduces this error I can assist you with that.

Tarjei400 avatar Aug 30 '23 11:08 Tarjei400

This might already be fixed, a release is planned this week-end, let's see if it happens and does fix the problem.

ericvergnaud avatar Aug 30 '23 11:08 ericvergnaud

@ericvergnaud Cool! Just in case let me know guys if there are issues, would gladly help :+1:

Tarjei400 avatar Aug 30 '23 12:08 Tarjei400

@ericvergnaud Hey, has this been released yet?

ShaharCider avatar Sep 03 '23 10:09 ShaharCider

i'm seeing the same issue using esbuild - so am also curious about release status/timeline

ZenMagus avatar Sep 19 '23 19:09 ZenMagus

npm install antlr4 --save-dev npm install esbuild --save-dev

[email protected] [email protected]

npx esbuild target.js --bundle --platform=node --format=cjs --outfile=out.cjs

---> node out.cjs inFile

<--- TypeError [ERR-INVALID_ARG_VALUE]: The argument 'fliename' must be a file URL object, file URL string, or absolute path string. Received undefined

ZenMagus avatar Sep 19 '23 19:09 ZenMagus

Can you try manually replacing antlr4's package.json with latest from 'development' branch and see of it fixes the problem ?

ericvergnaud avatar Sep 20 '23 07:09 ericvergnaud

Hey! thanks for the idea to try! ---> OK here is what i found:

looks like the package.json installed via [ npm install antlr4 ] is from master and has the following:

"browser": "dist/antlr4.web.js", "main": "dist/antlr4.node.mjs",

and the package.json from dev has the following changes:

c "browser": "dist/antlr4.web.mjs", c "main": "dist/antlr4.node.cjs",

  • "module": "dist/antlr4.node.mjs",

-- when i place the updated 'dev' package.json in /node_modules/antlr4 and re-run esbuild it still results in the undefined filename error

-- i am willing and able to work on this -- do you have ideas for further adjustments to experiment with?

maybe i can install the entire dev branch? but not exactly sure how to do that without NPM -- i'm search up how to do this and see if that will fix it.

ZenMagus avatar Sep 20 '23 15:09 ZenMagus

The dev file seems to work for many people, I have never used esbuild so I encourage you to try out things Maybe what's missing is a esbuild cli option ? also not sure what target.js is (filename error raises all sorts of suspicions)

ericvergnaud avatar Sep 20 '23 15:09 ericvergnaud

is my first time trying to pack javascript -- tried and failed with rollup and esbuild so far. target.js name was just to obscure actual file name. i believe the error i'm experiencing is the same error that is described in detail at the beginning of this thread by the OP. basically i have a working javascript ANTLR4 parser - but it breaks when i try to pack it up. want to move it into a VSCode extension without requiring ANTLR4 being required by the extension - but may have to go that route if i can't pack it up.

ZenMagus avatar Sep 20 '23 16:09 ZenMagus

The generated parser requires the antlr runtime, regardless of packaging

ericvergnaud avatar Sep 20 '23 16:09 ericvergnaud

ah OK - i thought i could pack it all up -- is probably better form just to make antlr4 npm install part of the VSCode dependency/environment so will go that route -- thank you for your thoughts and your help! very much appreciated <3

ZenMagus avatar Sep 20 '23 16:09 ZenMagus

@ZenMagus i met the same problem, it seems to be caused by the import.meta.url, You can see if this solution can help you https://github.com/evanw/esbuild/issues/1492#issuecomment-893144483

eirueirufu avatar Oct 06 '23 12:10 eirueirufu

All, please try 4.13.1-patch-1, so we can possibly close this.

ericvergnaud avatar Nov 10 '23 08:11 ericvergnaud

Hi @ericvergnaud @eirueirufu

Thank you for your responses and input!

My dev environment is no longer in context with the OPs issue so i don't have follow up.

Since Ericvergnaud's reply above I have gone the route of

  • using the node runtime
  • generating to typescript
  • manually fixing generated typescript errors - (currently it is generating nulls as values for the linteralNames string which typescript complains about)

ANTLR4 is amazing.
I am grateful to be able to have this tool - thank you.

ZenMagus avatar Nov 10 '23 18:11 ZenMagus

The bug is still in this repo but I found a way to inject a fix at the command level in this comment :

https://github.com/evanw/esbuild/issues/2441#issuecomment-1315341698

Janther avatar Dec 19 '23 17:12 Janther