Can not load language due to _EditorSimpleWorker.loadForeignModule error #87

glebcha · 2023-12-06T15:26:31Z

Using Vite and standard configuration like yours https://github.com/DTStack/monaco-sql-languages/blob/main/website/vite.config.ts

languageWorker.ts

self.MonacoEnvironment = {
  async  getWorker(_, label) {
    let worker;
    
    switch (label) {
      case 'trinosql':
        worker = await import('monaco-sql-languages/out/esm/trinosql/trinosql.worker?worker');
        break;
      default:
        worker = await import('monaco-editor/esm/vs/editor/editor.worker?worker');
    }

    return new worker.default();
  },
};

monacoInit.ts

import { loader } from '@monaco-editor/react';
import * as monaco from 'monaco-editor';

import 'monaco-sql-languages/out/esm/trinosql/trinosql.contribution';
import {
  setupLanguageFeatures,
  LanguageIdEnum,
} from 'monaco-sql-languages';

import './language-worker.ts';
import { theme } from './theme';

setupLanguageFeatures({
  languageId: LanguageIdEnum.TRINO,
  completionItems: false
});

loader.config({ monaco });

loader.init().then((monaco) => {
  monaco.editor.defineTheme('theme', theme);
});

Editor.tsx

import MonacoEditor from '@monaco-editor/react';
import { LanguageIdEnum } from 'monaco-sql-languages';

export const Editor = (editorOptions) => {
 return (
    <MonacoEditor
      height="100%"
      width="100%"
      defaultLanguage={LanguageIdEnum.TRINO}
      options={editorOptions}
    />
  );
}

HaydenOrz · 2023-12-07T02:01:46Z

You need to make sure that the Monaco Editor version is 0.31.0， and the languageWorker.ts should be done with reference to this Monaco doc integrate-esm.md#using-vite

glebcha · 2023-12-07T08:04:47Z

You need to make sure that the Monaco Editor version is 0.31.0， and the languageWorker.ts should be done with reference to this Monaco doc integrate-esm.md#using-vite

It won't help, because the root cause is import from dt-sql-parser/dist/parser inside ex. TrinoSQLWorker.
Such imports won't resolve in workers, but they will if I move workers to my project source directory and change import paths like this:

src/workers/TrinoSQLWorker.js

import TrinoSQL from 'dt-sql-parser/dist/parser/trinosql';
import { BaseSQLWorker } from 'monaco-sql-languages/out/esm/baseSQLWorker';
export class TrinoSQLWorker extends BaseSQLWorker {
    constructor(ctx) {
        super();
        this._ctx = ctx;
        this.parser = new TrinoSQL();
    }
}
export function create(ctx, createData) {
    return new TrinoSQLWorker(ctx, createData);
}

src/workers/trinosql.worker.js

import * as EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker';
import { TrinoSQLWorker } from './TrinoSQLWorker.js';

self.onmessage = () => {
    EditorWorker.initialize((ctx, createData) => {
        return new TrinoSQLWorker(ctx, createData);
    });
};

Also I'd like to mention that language support works fine with latest [email protected] (initial setup and solution found with version 0.31.0 recommended by your library).

HaydenOrz · 2023-12-07T08:28:57Z

Can you provide the versions of the dependencies associated with this problem in your project? Includes vite, monaco-sql-languages and React Monaco Editor.

Of course, the best-case scenario is that you create a minimal demo that reproduces the problem, if you will.

This will help us resolve the issue faster

glebcha · 2023-12-07T08:35:01Z

Can you provide the versions of the dependencies associated with this problem in your project? Includes vite, monaco-sql-languages and React Monaco Editor.

Sure, here they are (changing monaco version to 0.31.0 won't change anything):

{
    "monaco-editor": "^0.44.0",
    "@monaco-editor/react": "^4.6.0",
    "monaco-sql-languages": "^0.12.0-beta.7",
    "vite": "^5.0.6",
}

Of course, the best-case scenario is that you create a minimal demo that reproduces the problem, if you will.

I'll try to provide a sandbox.

HaydenOrz · 2023-12-07T08:35:37Z

Also I'd like to mention that language support works fine with latest [email protected] (initial setup and solution found with version 0.31.0 recommended by your library).

Yes, monaco-sql-languagues works with the latest version of monaco-editor in most cases. However, this depends on whether the latest version of monaco-editor changes to the handling of languageWorkers.

HaydenOrz · 2023-12-08T01:45:38Z

Sure, here they are (changing monaco version to 0.31.0 won't change anything):
{
    "monaco-editor": "^0.44.0",
    "@monaco-editor/react": "^4.6.0",
    "monaco-sql-languages": "^0.12.0-beta.7",
    "vite": "^5.0.6",
}
I'll try to provide a sandbox.

I've reproduced this issue. When I start Vite, I get an error like this printed in my terminal.

And found the error you reported in the browser.

It's a weird question for me. I'm not proficient at using Vite and I don't know how to fix this at the moment, but I'll keep trying. If you have any ideas on how to fix this, please let me know, it will be very helpful for me, thanks.

glebcha · 2023-12-08T15:02:45Z

And found the error you reported in the browser.

It's a weird question for me. I'm not proficient at using Vite and I don't know how to fix this at the moment, but I'll keep trying. If you have any ideas on how to fix this, please let me know, it will be very helpful for me, thanks.

Sorry for being late with repro. Great that you've found this bug.
Now I try to understand why this happen in Vite and seems that it is because of import resolution strategy, but it is only a possibility.
Also sometimes Vite's builds are failing with registering language worker while codebase stay same (that's why import resolution strategy may be the reason).
Also I'll try rspack, because it has compatibility layer with webpack api and webpack seems to work great with your library.

resetsix · 2023-12-09T01:25:11Z

Indeed, I'm experiencing the same problem. The codebase registers the syntax correctly, but using the yarn add monaco-sql-languages dependency causes the registration to fail. Even if I build with webpack and install the correct dependencies, it still prompts an assertion error message. I also can't resolve the syntax error message caused by using vite

webpack build error message

/* config-overrides.js */

/*
...
Consistent with monaco-sql-languages#using-the-monaco-editor-webpack-plugin configuration
...
*/

module.exports = function override(config, env) {
	
	config.plugins = [...config.plugins, monacoWebpackPlugin];

	return config;
};

webpack - package.json

"@dtinsight/molecule": "^1.3.0",
"monaco-editor-webpack-plugin": "^7.1.0",
"monaco-sql-languages": "^0.12.0-beta.7",
"reflect-metadata": "^0.1.13",
"rollup-plugin-node-polyfills": "^0.2.1",
"rollup-plugin-polyfill-node": "^0.12.0",
"util": "link:rollup-plugin-node-polyfills/polyfills/util",
"assert": "link:rollup-plugin-node-polyfills/polyfills/assert",
"buffer": "^6.0.3"

HaydenOrz · 2023-12-09T12:12:49Z

webpack build error message

@resetsix You need to install the assert package to fix this.

see webpack doc about this problem .

resetsix · 2023-12-10T02:19:01Z

webpack build error message

@resetsix You need to install the assert package to fix this.

see webpack doc about this problem .

Glad to receive your reply. I also found out that the reason for the error report is that webpack no longer provides Node.js built-in modules like assert, utils, etc. after webpack5

However, I have a new problem, after registering the syntax with setupLanguageFeatures, the console tells me that webpack fails when trying to parse the source map of the dt-sql-parser module

Although monaco-sql-languages already dependency on dt-sql-parser, I assumed the error was telling me that I needed to install the dt-sql-parser dependency again locally, but it turns out that the same error message applies after installation

The error reported seems to be related to dt-sql-parser

my package.json

"@dtinsight/molecule": "^1.3.0",
"monaco-editor-webpack-plugin": "^7.1.0",
"monaco-sql-languages": "^0.12.0-beta.7",
"dt-sql-parser": "^4.0.0-beta.4.8",
"assert": "^2.1.0",
"util": "^0.12.5"

It's a real pain in the ass.

HaydenOrz · 2023-12-10T15:05:50Z

@resetsix @glebcha
I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

glebcha · 2023-12-10T16:05:54Z

@resetsix @glebcha
I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

Thank you so much, really appreciate your help!
As a proposal, can you include all needed NodeJs polyfills (process, Buffer etc.) required by dt-sql-parser?

Please ping me to test new implementations, will be happy to help you.

resetsix · 2023-12-11T00:59:37Z

@resetsix @glebcha I have to admit that we didn't do enough testing on the monaco-sql-langauges integration. Because monaco-sql-languages works well in webpack and umi projects,

Regarding the sourceMap of dt-sql-parser, we will release a new package to remove the sourceMap in the near future. In fact, dt-sql-parser is compiled using tsc only, and sourceMap is not required.

Previously, our main focus was on supporting more sql-languages, and I'm sorry that these issues have caused you so much trouble. We'll resolve integration issues as soon as possible and provide examples of integrations.

I'm also more than willing to help with testing in any way I can if I can

HaydenOrz · 2023-12-15T10:46:26Z

Correlation issue vite #15359

HaydenOrz · 2023-12-15T10:50:36Z

Referring to monaco-sql-languages-vite, this issue can be bypassed. This may not be the ultimate solution, I'll keep track of the issue.

The relevant integration documentation will be updated at a later date.

HaydenOrz · 2023-12-15T12:01:50Z

The error reported seems to be related to dt-sql-parser

@resetsix Version 0.12.0-beta.9 has been released. With the removal of the dt-sql-parser sourceMap in this release, you will no longer receive this error

resetsix · 2023-12-15T12:06:51Z

The error reported seems to be related to dt-sql-parser

@resetsix Version 0.12.0-beta.9 has been released. With the removal of the dt-sql-parser sourceMap in this release, you will no longer receive this error@resetsix 版本 0.12.0-beta.9 已发布。在此版本中删除 dt-sql-parser sourceMap 后，您将不再收到此错误

I see，thank u for all your efforts

HaydenOrz · 2023-12-20T12:05:21Z

@resetsix @glebcha

Integrating samples and integrating doc are all done now.

Integration in Vite is still not supported at this time.

glebcha · 2023-12-25T19:29:33Z

@resetsix @glebcha

Integrating samples and integrating doc are all done now.

Integration in Vite is still not supported at this time.

Can I count on your support with Vite integration?

HaydenOrz · 2023-12-26T01:58:37Z

Can I count on your support with Vite integration?

I'm following up on this issue, but there's no more progress at the moment.

The Monaco SQL Languages website is currently being built with vite, but it requires vite-plugin-commonjs, This is the only solution at the moment, but I don't think it's suitable for most projects.

ankit-shopflo · 2024-02-08T09:48:02Z

Is there any progress on this issue? We are planning to move to Monaco but this is a blocker.

glebcha · 2024-02-12T12:45:31Z

@HaydenOrz Monaco SQL Languages is broken with [email protected], but only in production build.

UPDATE:

This setup works fine

import 'monaco-editor/esm/vs/editor/browser/coreCommands.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/comment.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/blockCommentCommand.js';
import 'monaco-editor/esm/vs/editor/contrib/comment/browser/lineCommentCommand.js';
import 'monaco-editor/esm/vs/editor/contrib/find/browser/findController.js';
import 'monaco-editor/esm/vs/editor/contrib/parameterHints/browser/parameterHints.js';
import 'monaco-editor/esm/vs/editor/contrib/snippet/browser/snippetController2.js';
import 'monaco-editor/esm/vs/editor/contrib/suggest/browser/suggestController.js';

import 'monaco-sql-languages/out/esm/trinosql/trinosql.contribution';

import {
  setupLanguageFeatures,
  LanguageIdEnum,
} from 'monaco-sql-languages';
import { initMonacoEnvironment } from './language-worker.ts';

export function initMonacoEditor() {
  initMonacoEnvironment();

  setupLanguageFeatures({
    languageId: LanguageIdEnum.TRINO,
  });
}

HaydenOrz · 2024-02-19T03:04:17Z

Is there any progress on this issue? We are planning to move to Monaco but this is a blocker.

Sorry, there is no further progress at this time. We plan to replace the antlr4 runtime of dt-sql-parser with antlr4ng in the future, which should fix the issue, but it will take a lot of time.

HaydenOrz · 2024-02-19T03:11:38Z

@HaydenOrz Monaco SQL Languages is broken with [email protected], but only in production build.

Thanks for your efforts,

There are currently ways to work around this issue using some unofficial plugins, such as the approach taken by the monaco-sql-languagues website. However, as mentioned in the documentation, this is not a final solution and we hope that monaco-sql-languages can be easily integrated into vite projects without introducing any additional overhead.

HaydenOrz · 2024-02-19T12:38:53Z

@glebcha @ankit-shopflo @resetsix

Hi, here's the happy news. I've done my preliminary work on the antlr4 runtime migration of dt-sql-parser DTStack/dt-sql-parser#261, and it's been much smoother than expected.

Now you can install [email protected] to experience. While this feature hasn't been merged into the main branch yet, I've done quite a bit of testing and it's working well so far.

In this release, the issue of integration with Vite has been resolved. In addition , dependencies such as assert and util and the global variable process.env mentioned in the previous documentation are no longer needed. Now, integrating monaco-sql-languages in Vite no longer requires any additional dependencies or any redundant configurations.

You can see the latest integration documentation and the latest integration examples on this branch
https://github.com/HaydenOrz/monaco-sql-languages/tree/feat/migrate_ng

HaydenOrz · 2024-04-01T12:51:19Z

@glebcha @ankit-shopflo @resetsix

monaco-sql-languages 0.12.0-beta.11 has been released：
https://www.npmjs.com/package/monaco-sql-languages/v/0.12.0-beta.11

We have confirmed that the issue is fixed on this version, please install this version to fix this issue.

In addition to this, there are some breaking changes on this release, if the upgrade causes an exception, please check the latest documentation.

This issue will be closed within a week if it is not active.

glebcha · 2024-04-03T16:16:59Z

@glebcha @ankit-shopflo @resetsix

monaco-sql-languages 0.12.0-beta.11 has been released： https://www.npmjs.com/package/monaco-sql-languages/v/0.12.0-beta.11

We have confirmed that the issue is fixed on this version, please install this version to fix this issue.

In addition to this, there are some breaking changes on this release, if the upgrade causes an exception, please check the latest documentation.

This issue will be closed within a week if it is not active.

@HaydenOrz thank you so much for efforts!
Changes are easy to support, did it within a few minutes.

P.S.: do you have plans to support the latest monaco version?

HaydenOrz · 2024-04-17T06:13:56Z

P.S.: do you have plans to support the latest monaco version?

At the moment it looks like moanco-sql-lanuages already supports the latest version of monaco. I didn't see any issues when monaco-sql-languages was running on the latest monaco. But this needs more testing.

If you find that monaco-sql-languages is running unexpectedly on the latest version of monaco, please let us know by opening a new issue.

HaydenOrz added the bug Something isn't working label Dec 10, 2023

HaydenOrz mentioned this issue Dec 15, 2023

Vue3 + Vite4 + monaco-editor 加载 monaco-sql-languages package 报错 #39

Closed

magnew mentioned this issue Dec 27, 2023

Use Monaco editor for SQL runner lightdash/lightdash#8287

Closed

HaydenOrz closed this as completed Apr 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can not load language due to _EditorSimpleWorker.loadForeignModule error #87

Can not load language due to _EditorSimpleWorker.loadForeignModule error #87

glebcha commented Dec 6, 2023 •

edited

Loading

HaydenOrz commented Dec 7, 2023

glebcha commented Dec 7, 2023 •

edited

Loading

HaydenOrz commented Dec 7, 2023

glebcha commented Dec 7, 2023 •

edited

Loading

HaydenOrz commented Dec 7, 2023

HaydenOrz commented Dec 8, 2023

glebcha commented Dec 8, 2023

resetsix commented Dec 9, 2023

HaydenOrz commented Dec 9, 2023 •

edited

Loading

resetsix commented Dec 10, 2023

HaydenOrz commented Dec 10, 2023 •

edited

Loading

glebcha commented Dec 10, 2023

resetsix commented Dec 11, 2023

HaydenOrz commented Dec 15, 2023

HaydenOrz commented Dec 15, 2023

HaydenOrz commented Dec 15, 2023

resetsix commented Dec 15, 2023

HaydenOrz commented Dec 20, 2023

glebcha commented Dec 25, 2023

HaydenOrz commented Dec 26, 2023

ankit-shopflo commented Feb 8, 2024

glebcha commented Feb 12, 2024 •

edited

Loading

HaydenOrz commented Feb 19, 2024

HaydenOrz commented Feb 19, 2024

HaydenOrz commented Feb 19, 2024 •

edited

Loading

HaydenOrz commented Apr 1, 2024

glebcha commented Apr 3, 2024

HaydenOrz commented Apr 17, 2024

Can not load language due to _EditorSimpleWorker.loadForeignModule error #87

Can not load language due to _EditorSimpleWorker.loadForeignModule error #87

Comments

glebcha commented Dec 6, 2023 • edited Loading

HaydenOrz commented Dec 7, 2023

glebcha commented Dec 7, 2023 • edited Loading

HaydenOrz commented Dec 7, 2023

glebcha commented Dec 7, 2023 • edited Loading

HaydenOrz commented Dec 7, 2023

HaydenOrz commented Dec 8, 2023

glebcha commented Dec 8, 2023

resetsix commented Dec 9, 2023

HaydenOrz commented Dec 9, 2023 • edited Loading

resetsix commented Dec 10, 2023

HaydenOrz commented Dec 10, 2023 • edited Loading

glebcha commented Dec 10, 2023

resetsix commented Dec 11, 2023

HaydenOrz commented Dec 15, 2023

HaydenOrz commented Dec 15, 2023

HaydenOrz commented Dec 15, 2023

resetsix commented Dec 15, 2023

HaydenOrz commented Dec 20, 2023

glebcha commented Dec 25, 2023

HaydenOrz commented Dec 26, 2023

ankit-shopflo commented Feb 8, 2024

glebcha commented Feb 12, 2024 • edited Loading

HaydenOrz commented Feb 19, 2024

HaydenOrz commented Feb 19, 2024

HaydenOrz commented Feb 19, 2024 • edited Loading

HaydenOrz commented Apr 1, 2024

glebcha commented Apr 3, 2024

HaydenOrz commented Apr 17, 2024

glebcha commented Dec 6, 2023 •

edited

Loading

glebcha commented Dec 7, 2023 •

edited

Loading

glebcha commented Dec 7, 2023 •

edited

Loading

HaydenOrz commented Dec 9, 2023 •

edited

Loading

HaydenOrz commented Dec 10, 2023 •

edited

Loading

glebcha commented Feb 12, 2024 •

edited

Loading

HaydenOrz commented Feb 19, 2024 •

edited

Loading