'baml-cli generate' supports ES Module

[lanluu]

Currently, baml-cli generate creates .ts files such as globals.ts containg the following import:

import { getBamlFiles } from './inlinedbaml'

Compilation in module type can cause error, as the resolve without file extensions .js cannot executed.

How can I force baml-cli generate to create the import with file extensions i.e. import { getBamlFiles } from './inlinedbaml.js'?

[hellovai]

oh that's a great request. We should really just support raw js as a generation type and that would solve this problem i think. I'll take a look at when we can do this (it should be low effort)

Meanwhile, I think your workaround would be:

1. have a seperate local package that in your node project that contains your baml code
2. import your code from that local package

[lanluu]

I think i did it but it does not work out.

So, i have the following struture: (1) Package P1: contains baml_client and my code calls to functions given by baml_client -> I build it with tsc and pack dist folder.

import { b } from '../baml_client/async_client.js';
...

(2) Main Project (NestJs) -> import .js file from Package P1 -> By running via nest start, the mentioned error still appears:

Error [ERR_MODULE_NOT_FOUND]: ....
baml_client/globals imported from .... /async_client.js
at finalizeResolution (node:internal/modules/esm/resolve:257:11)
    at moduleResolve (node:internal/modules/esm/resolve:914:10)
    at defaultResolve (node:internal/modules/esm/resolve:1039:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:554:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:523:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:246:38)
    at ModuleJob._link (node:internal/modules/esm/module_job:126:49)

Or did you mean that at (2) I should import the whole Package P1?

[aaronvg]

yes at (2) you should import the whole package. Basically you can just build a standalone package with your baml client and then add it to the package dependencies in package #2. You can use something like npm or pnpm workspaces. The problem is just hot-reloading, and the development cycle is going to be longer. Perhaps we can add a flag to the generator block to add .js to the file imports.

[lanluu]

Thank you @aaronvg @hellovai for your tips. I will give it a try tonight. I wish you guys will add such configuration for generation soon. :-) (y). Really love BAML.

[lanluu]

I confirm it worked.

[jkonowitch]

oh that's a great request. We should really just support raw js as a generation type and that would solve this problem i think. I'll take a look at when we can do this (it should be low effort)

Meanwhile, I think your workaround would be:

1. have a seperate local package that in your node project that contains your baml code
2. import your code from that local package

ESM is especially important for Deno. At the moment, you must deno run --unstable-sloppy-imports in order for the modules to be resolved.

What would raw js generation achieve? And I'm assuming type definitions would still be generated?

(FYI workaround for now, ensure

{
  "deno.unstable": ["sloppy-imports"]
}

is in your .vscode/settings.json to get IDE type completions )

[jkonowitch]

One other complicating consideration here: Deno enforces the correct file extension, so in the case of a typescript file it expects:

import type { Checked, Check } from "./types.ts"

Whereas in node, you would use

import type { Checked, Check } from "./types.js"

Even for typescript files. (Because when using node.js everything is ultimately transpiled to js, whereas that is not the case in Deno)

One alternative, instead of making baml deal with this:

deno lint --unstable-sloppy-imports . --fix will fix all of the file extension issues - this could easily be run after baml-cli generate in Deno projects. Just a thought to minimize complexity of the library.

[lanluu]

@jkonowitch Have you had any update for me? :-) The latest version 0.73.4 still generates the following code:

export { b } from "./async_client"

export * from "./types"
export * from "./tracing"
export { resetBamlEnvVars } from "./globals"
export { BamlValidationError } from "@boundaryml/baml"

If I use node instead of tsx to run my code, the original error reported above appears again. :-(

Thank you for your info!

[lanluu]

@jkonowitch Have you had any update for me? :-) The latest version 0.73.4 still generates the following code:

export { b } from "./async_client"

export * from "./types" export * from "./tracing" export { resetBamlEnvVars } from "./globals" export { BamlValidationError } from "@boundaryml/baml" If I use node instead of tsx to run my code, the original error reported above appears again. :-(

Thank you for your info!

Currently, the following workaround works for me: node --loader ts-node/esm dist/src/main.js

[lanluu]

Hi, any update :-) ? Thanks!

[jkonowitch]

@hellovai @aaronvg Reproduction here: https://github.com/jkonowitch/baml-esm-repro

At the moment I can confirm that BAML only works with a bundler. It does not work with node ESM resolution.

The fix would be to configure the generate command to add file extensions (.js) and ensure that type only imports are prefixed with import type.

[adamsuskin]

I am also blocked because of this issue :( I wish this was something flagged in the readme before trying to use baml for the project

[aaronvg]

@adamsuskin let me see if I can fix this for tomorrow's release

[aaronvg]

Ok I took @jkonowitch repro, for this test and it works now: https://github.com/BoundaryML/baml/pull/1831

Should be out tomorrow,

The change would be to add module_format esm to the generator

[aaronvg]

LMK if I'm missing something here, but it seems the only change needed was to change all relative imports in baml_client to use .js extension.

and now folks must import baml_client more explicitly:

import {b } from "./baml_client/index.js"
...

[adamsuskin]

Thanks for being so responsive! Happy to confirm with tomorrow's release.

[aaronvg]

Can you try 0.85.0 version of the extension and @boundaryml/baml package?

generator tyepscript {
  ....
  module_format "esm"
}

[adamsuskin]

That did the trick for me. Thanks!

[Kilowon]

I'm still encountering the ReferenceError: require is not defined error when running the production build (npm run start) of my SolidStart/Vinxi application, even after updating to 0.85.0 and using module_format "esm" in my generators.baml.

/generators.baml

generator typescript {
    output_type "typescript"
    output_dir "../"
    version "0.85.0"
    default_client_mode async
    module_format "esm"
}

/baml_resume.tsx

import { b } from "~/../baml_client/index.js"

I updated my Repo from #1690 to reflect the new changes.

The error occurs during server startup and originates from the requireNative function within the bundled server code (.output/server/chunks/_/index.mjs), specifically when it tries to load the native .node addon using require():

[unhandledRejection] Error: Failed to load native binding
    at file:///E:/JavaScriptPlusReact/BAML_Server/solid-baml/.output/server/chunks/_/index.mjs:374:11
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    // ... (rest of stack trace)
  [cause]: [
    ReferenceError: require is not defined
        at requireNative (file:///E:/JavaScriptPlusReact/BAML_Server/solid-baml/.output/server/chunks/_/index.mjs:102:9) // or similar line number
        at file:///E:/JavaScriptPlusReact/BAML_Server/solid-baml/.output/server/chunks/_/index.mjs:347:17 // or similar line number
        // ... (rest of stack trace)
  ]

Unless its something I'm missing here?

It seems the issue reported previously #1690 persists because the native.js file within the @boundaryml/baml package still uses direct require() calls for the .node files, which fails in the bundled ESM server environment.

Could you please investigate using an ESM-compatible method like createRequire within native.js to load the native bindings?

https://nodejs.org/api/module.html#modulecreaterequirefilename

Thanks!

[aaronvg]

ok so it seems we fixed the esm compatibility in baml_client, but we still need to fix it in @boundaryml/baml package. For my own visibility, this issue is likely related: https://github.com/napi-rs/napi-rs/pull/2284 which uses createRequire to load native bindings.

I will use https://github.com/Kilowon/solid-baml as a reference to fix the remaining issue, aiming for next weeks release (next Tuesday, sorry for the delay!). It will depend on how easy it is to configure our NAPI-RS dependency to support both cjs and esm

[Kilowon]

Sounds good, thanks

[lukeramsden]

I notice that the following two imports do not have extensions added correctly in the module_format esm mode

import { FieldType } from '@boundaryml/baml/native'
import { TypeBuilder as _TypeBuilder, EnumBuilder, EnumViewer, ClassBuilder, ClassViewer } from '@boundaryml/baml/type_builder'

Changing them to this fixes some issues for me:

import { FieldType } from '@boundaryml/baml/native.js'
import { TypeBuilder as _TypeBuilder, EnumBuilder, EnumViewer, ClassBuilder, ClassViewer } from '@boundaryml/baml/type_builder.js'

[aaronvg]

this seems like the actual remaining fix here https://github.com/BoundaryML/baml/pull/2489/files but requires more testing