Can not load language due to _EditorSimpleWorker.loadForeignModule error

[glebcha]

Using Vite and standard configuration like yours https://github.com/DTStack/monaco-sql-languages/blob/main/website/vite.config.ts

languageWorker.ts

self.MonacoEnvironment = {
  async  getWorker(_, label) {
    let worker;
    
    switch (label) {
      case 'trinosql':
        worker = await import('monaco-sql-languages/out/esm/trinosql/trinosql.worker?worker');
        break;
      default:
        worker = await import('monaco-editor/esm/vs/editor/editor.worker?worker');
    }

    return new worker.default();
  },
};

monacoInit.ts

import { loader } from '@monaco-editor/react';
import * as monaco from 'monaco-editor';

import 'monaco-sql-languages/out/esm/trinosql/trinosql.contribution';
import {
  setupLanguageFeatures,
  LanguageIdEnum,
} from 'monaco-sql-languages';

import './language-worker.ts';
import { theme } from './theme';

setupLanguageFeatures({
  languageId: LanguageIdEnum.TRINO,
  completionItems: false
});

loader.config({ monaco });

loader.init().then((monaco) => {
  monaco.editor.defineTheme('theme', theme);
});

Editor.tsx

import MonacoEditor from '@monaco-editor/react';
import { LanguageIdEnum } from 'monaco-sql-languages';

export const Editor = (editorOptions) => {
 return (
    <MonacoEditor
      height="100%"
      width="100%"
      defaultLanguage={LanguageIdEnum.TRINO}
      options={editorOptions}
    />
  );
}

[HaydenOrz]

You need to make sure that the Monaco Editor version is 0.31.0， and the languageWorker.ts should be done with reference to this Monaco doc integrate-esm.md#using-vite

[glebcha]

You need to make sure that the Monaco Editor version is 0.31.0， and the languageWorker.ts should be done with reference to this Monaco doc integrate-esm.md#using-vite

It won't help, because the root cause is import from dt-sql-parser/dist/parser inside ex. TrinoSQLWorker. Such imports won't resolve in workers, but they will if I move workers to my project source directory and change import paths like this:

src/workers/TrinoSQLWorker.js

import TrinoSQL from 'dt-sql-parser/dist/parser/trinosql';
import { BaseSQLWorker } from 'monaco-sql-languages/out/esm/baseSQLWorker';
export class TrinoSQLWorker extends BaseSQLWorker {
    constructor(ctx) {
        super();
        this._ctx = ctx;
        this.parser = new TrinoSQL();
    }
}
export function create(ctx, createData) {
    return new TrinoSQLWorker(ctx, createData);
}

src/workers/trinosql.worker.js

import * as EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker';
import { TrinoSQLWorker } from './TrinoSQLWorker.js';

self.onmessage = () => {
    EditorWorker.initialize((ctx, createData) => {
        return new TrinoSQLWorker(ctx, createData);
    });
};

Also I'd like to mention that language support works fine with latest [email protected] (initial setup and solution found with version 0.31.0 recommended by your library).

[HaydenOrz]

Can you provide the versions of the dependencies associated with this problem in your project? Includes vite, monaco-sql-languages and React Monaco Editor.

Of course, the best-case scenario is that you create a minimal demo that reproduces the problem, if you will.

This will help us resolve the issue faster

[glebcha]

Can you provide the versions of the dependencies associated with this problem in your project? Includes vite, monaco-sql-languages and React Monaco Editor.

Sure, here they are (changing monaco version to 0.31.0 won't change anything):

{
    "monaco-editor": "^0.44.0",
    "@monaco-editor/react": "^4.6.0",
    "monaco-sql-languages": "^0.12.0-beta.7",
    "vite": "^5.0.6",
}

Of course, the best-case scenario is that you create a minimal demo that reproduces the problem, if you will.

I'll try to provide a sandbox.

[HaydenOrz]

Also I'd like to mention that language support works fine with latest [email protected] (initial setup and solution found with version 0.31.0 recommended by your library).

Yes, monaco-sql-languagues works with the latest version of monaco-editor in most cases. However, this depends on whether the latest version of monaco-editor changes to the handling of languageWorkers.

[HaydenOrz]

Sure, here they are (changing monaco version to 0.31.0 won't change anything):

{
    "monaco-editor": "^0.44.0",
    "@monaco-editor/react": "^4.6.0",
    "monaco-sql-languages": "^0.12.0-beta.7",
    "vite": "^5.0.6",
}

I'll try to provide a sandbox.

I've reproduced this issue. When I start Vite, I get an error like this printed in my terminal.

And found the error you reported in the browser.

It's a weird question for me. I'm not proficient at using Vite and I don't know how to fix this at the moment, but I'll keep trying. If you have any ideas on how to fix this, please let me know, it will be very helpful for me, thanks.

[glebcha]

And found the error you reported in the browser.

It's a weird question for me. I'm not proficient at using Vite and I don't know how to fix this at the moment, but I'll keep trying. If you have any ideas on how to fix this, please let me know, it will be very helpful for me, thanks.

Sorry for being late with repro. Great that you've found this bug. Now I try to understand why this happen in Vite and seems that it is because of import resolution strategy, but it is only a possibility. Also sometimes Vite's builds are failing with registering language worker while codebase stay same (that's why import resolution strategy may be the reason). Also I'll try rspack, because it has compatibility layer with webpack api and webpack seems to work great with your library.

[resetsix]

Indeed, I'm experiencing the same problem. The codebase registers the syntax correctly, but using the yarn add monaco-sql-languages dependency causes the registration to fail. Even if I build with webpack and install the correct dependencies, it still prompts an assertion error message. I also can't resolve the syntax error message caused by using vite

webpack build error message

/* config-overrides.js */

/*
...
Consistent with monaco-sql-languages#using-the-monaco-editor-webpack-plugin configuration
...
*/

module.exports = function override(config, env) {
	
	config.plugins = [...config.plugins, monacoWebpackPlugin];

	return config;
};

webpack - package.json

"@dtinsight/molecule": "^1.3.0",
"monaco-editor-webpack-plugin": "^7.1.0",
"monaco-sql-languages": "^0.12.0-beta.7",
"reflect-metadata": "^0.1.13",
"rollup-plugin-node-polyfills": "^0.2.1",
"rollup-plugin-polyfill-node": "^0.12.0",
"util": "link:rollup-plugin-node-polyfills/polyfills/util",
"assert": "link:rollup-plugin-node-polyfills/polyfills/assert",
"buffer": "^6.0.3"

[HaydenOrz]

webpack build error message

@resetsix You need to install the assert package to fix this.

see webpack doc about this problem .

[resetsix]

webpack build error message

@resetsix You need to install the assert package to fix this.

see webpack doc about this problem .

Glad to receive your reply. I also found out that the reason for the error report is that webpack no longer provides Node.js built-in modules like assert, utils, etc. after webpack5

However, I have a new problem, after registering the syntax with setupLanguageFeatures, the console tells me that webpack fails when trying to parse the source map of the dt-sql-parser module

Although monaco-sql-languages already dependency on dt-sql-parser, I assumed the error was telling me that I needed to install the dt-sql-parser dependency again locally, but it turns out that the same error message applies after installation

The error reported seems to be related to dt-sql-parser

my package.json

"@dtinsight/molecule": "^1.3.0",
"monaco-editor-webpack-plugin": "^7.1.0",
"monaco-sql-languages": "^0.12.0-beta.7",
"dt-sql-parser": "^4.0.0-beta.4.8",
"assert": "^2.1.0",
"util": "^0.12.5"

It's a real pain in the ass.

[HaydenOrz]

@resetsix @glebcha I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

[glebcha]

@resetsix @glebcha I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

Thank you so much, really appreciate your help! As a proposal, can you include all needed NodeJs polyfills (process, Buffer etc.) required by dt-sql-parser?

Please ping me to test new implementations, will be happy to help you.

[resetsix]

@resetsix @glebcha I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

I'm also more than willing to help with testing in any way I can if I can

[HaydenOrz]

Correlation issue vite #15359

[HaydenOrz]

Referring to monaco-sql-languages-vite, this issue can be bypassed. This may not be the ultimate solution, I'll keep track of the issue.

The relevant integration documentation will be updated at a later date.

[HaydenOrz]

The error reported seems to be related to dt-sql-parser

@resetsix Version 0.12.0-beta.9 has been released. With the removal of the dt-sql-parser sourceMap in this release, you will no longer receive this error

[resetsix]

The error reported seems to be related to dt-sql-parser

@resetsix Version 0.12.0-beta.9 has been released. With the removal of the dt-sql-parser sourceMap in this release, you will no longer receive this error@resetsix 版本 0.12.0-beta.9 已发布。在此版本中删除 dt-sql-parser sourceMap 后，您将不再收到此错误

I see，thank u for all your efforts

[HaydenOrz]

@resetsix @glebcha

Integrating samples and integrating doc are all done now.

Integration in Vite is still not supported at this time.

[glebcha]

@resetsix @glebcha

Integrating samples and integrating doc are all done now.

Integration in Vite is still not supported at this time.

Can I count on your support with Vite integration?

[HaydenOrz]

Can I count on your support with Vite integration?

I'm following up on this issue, but there's no more progress at the moment.

The Monaco SQL Languages website is currently being built with vite, but it requires vite-plugin-commonjs, This is the only solution at the moment, but I don't think it's suitable for most projects.

[ankit-shopflo]

Is there any progress on this issue? We are planning to move to Monaco but this is a blocker.

[glebcha]

@HaydenOrz Monaco SQL Languages is broken with [email protected], but only in production build.

UPDATE:

This setup works fine

import 'monaco-editor/esm/vs/editor/browser/coreCommands.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/comment.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/blockCommentCommand.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/lineCommentCommand.js';
import 'monaco-editor/esm/vs/editor/contrib/find/browser/findController.js';
import 'monaco-editor/esm/vs/editor/contrib/parameterHints/browser/parameterHints.js';
import 'monaco-editor/esm/vs/editor/contrib/snippet/browser/snippetController2.js';
import 'monaco-editor/esm/vs/editor/contrib/suggest/browser/suggestController.js';

import 'monaco-sql-languages/out/esm/trinosql/trinosql.contribution';

import {
  setupLanguageFeatures,
  LanguageIdEnum,
} from 'monaco-sql-languages';
import { initMonacoEnvironment } from './language-worker.ts';

export function initMonacoEditor() {
  initMonacoEnvironment();

  setupLanguageFeatures({
    languageId: LanguageIdEnum.TRINO,
  });
}

[HaydenOrz]

Is there any progress on this issue? We are planning to move to Monaco but this is a blocker.

Sorry, there is no further progress at this time. We plan to replace the antlr4 runtime of dt-sql-parser with antlr4ng in the future, which should fix the issue, but it will take a lot of time.

[HaydenOrz]

@HaydenOrz Monaco SQL Languages is broken with [email protected], but only in production build.

Thanks for your efforts,

There are currently ways to work around this issue using some unofficial plugins, such as the approach taken by the monaco-sql-languagues website. However, as mentioned in the documentation, this is not a final solution and we hope that monaco-sql-languages can be easily integrated into vite projects without introducing any additional overhead.

[HaydenOrz]

@glebcha @ankit-shopflo @resetsix

Hi, here's the happy news. I've done my preliminary work on the antlr4 runtime migration of dt-sql-parser https://github.com/DTStack/dt-sql-parser/pull/261, and it's been much smoother than expected.

Now you can install [email protected] to experience. While this feature hasn't been merged into the main branch yet, I've done quite a bit of testing and it's working well so far.

In this release, the issue of integration with Vite has been resolved. In addition , dependencies such as assert and util and the global variable process.env mentioned in the previous documentation are no longer needed. Now, integrating monaco-sql-languages in Vite no longer requires any additional dependencies or any redundant configurations.

You can see the latest integration documentation and the latest integration examples on this branch https://github.com/HaydenOrz/monaco-sql-languages/tree/feat/migrate_ng

[HaydenOrz]

@glebcha @ankit-shopflo @resetsix

monaco-sql-languages 0.12.0-beta.11 has been released： https://www.npmjs.com/package/monaco-sql-languages/v/0.12.0-beta.11

We have confirmed that the issue is fixed on this version, please install this version to fix this issue.

In addition to this, there are some breaking changes on this release, if the upgrade causes an exception, please check the latest documentation.

This issue will be closed within a week if it is not active.

[glebcha]

@glebcha @ankit-shopflo @resetsix

monaco-sql-languages 0.12.0-beta.11 has been released： https://www.npmjs.com/package/monaco-sql-languages/v/0.12.0-beta.11

We have confirmed that the issue is fixed on this version, please install this version to fix this issue.

In addition to this, there are some breaking changes on this release, if the upgrade causes an exception, please check the latest documentation.

This issue will be closed within a week if it is not active.

@HaydenOrz thank you so much for efforts! Changes are easy to support, did it within a few minutes.

P.S.: do you have plans to support the latest monaco version?

[HaydenOrz]

P.S.: do you have plans to support the latest monaco version?

At the moment it looks like moanco-sql-lanuages already supports the latest version of monaco. I didn't see any issues when monaco-sql-languages was running on the latest monaco. But this needs more testing.

If you find that monaco-sql-languages is running unexpectedly on the latest version of monaco, please let us know by opening a new issue.