Error: No grammar provided for <insert-lang-here>

[stevebauman]

I'm trying to get async loading working properly for all languages supported by Shiki in showcode.app, but it seems some languages have issues and others do not. The issue is completely sporadic, even on the Shiki playground I receive this issue:

If you start clicking languages randomly, you'll eventually receive this error.

I haven't been able to track it down properly. My shiki highlighter instance is a singleton shared throughout my application, and when the languages load properly, they do stay loaded.

Can anyone point me in the direction of a potential fix? I've been digging through the source and trying to find potential issues, but my JS knowledge is certainly lacking.

Any help is greatly appreciated! ❤️ Thank you so much for this amazing tool.

[stevebauman]

This is my implementation (as a nuxt Vue plugin by the way):

import { setOnigasmWASM, setCDN, getHighlighter } from 'shiki';

setCDN('/shiki/');
setOnigasmWASM('/shiki/dist/onigasm.wasm');

const preloadedThemes  = ['github-light'];
const preloadedLangs = ['css', 'html', 'python', 'c', 'go', 'ruby', 'java', 'tsx'];
const customLangs = [
    {
        id: 'antlers',
        scopeName: 'text.html.statamic',
        path: 'languages/antlers.tmLanguage.json',
        embeddedLangs: ['html', 'php'],
    },
    {
        id: 'blade',
        scopeName: 'text.html.php.blade',
        path: 'languages/blade.tmLanguage.json',
        embeddedLangs: ['html', 'php'],
    },
];

export default async (context, inject) => {
    const highlighter = await getHighlighter({
        themes: preloadedThemes,
        langs: preloadedLangs
    });

    const shiki = {
        async loadLanguage(lang) {
            const custom = customLangs.find(({id}) => id === lang);

            return await highlighter.loadLanguage(custom ?? lang);
        },
    
        async loadLanguages(langs) {
            return await Promise.all(
                langs.map(async (lang) => await this.loadLanguage(lang))
            );
        },
    
        async loadTheme(theme) {
            return await highlighter.loadTheme(theme);       
        },
    
        getTheme(theme) {
            return highlighter.getTheme(theme);
        },
    
        tokens(code, lang, theme) {
            return highlighter.codeToThemedTokens(
                code,
                lang,
                theme,
            );
        }
    }

    inject('shiki', shiki);
}

[stevebauman]

I was able to produce a simple failing test:

https://github.com/stevebauman/shiki/runs/4244049762?check_suite_focus=true

[lnfel]

I also got the same error when using highlighter.codeToHtml(el.innerText, { lang: "bat" });. It seems the syntax changed, cause this is the one I followed in the docs.

Changed the code to highlighter.codeToHtml(el.innerText, 'bat') and it worked. Final code on my project:

shiki.getHighlighter({
    theme: 'material-palenight',
}).then(highlighter => {
    var code = highlighter.codeToHtml(el.innerText, 'bat');

    insertAfter(el, htmlToElement(code));
    el.parentNode.removeChild(el);
});

[octref]

Hey @stevebauman, it seems you have it fixed already in your fork. Did you figure our the issue?

[stevebauman]

Hey @octref, yup I was able to resolve this issue by adding the loaded grammar to the base TextMateRegistry implementation via the addGrammar() method:

https://github.com/stevebauman/shiki/blob/321a1d37c5d8777ed3bfd33a9970917f2a5c36ba/packages/shiki/src/registry.ts#L54

public async loadLanguage(lang: ILanguageRegistration) {
    const g = await this.addGrammar(await this._resolver.loadGrammar(lang.scopeName))

    // ...
}

Let me know if you'd like a PR submitted or if this is enough 👍

[octref]

That doesn't look like the fix for the issue – the original error is probably caused by loading a language that embeds source.objc and you don't have source.objc loaded. For example, markdown grammar.

[stevebauman]

Whoops -- forgot to mention this fix as well: https://github.com/stevebauman/shiki/commit/02542df16789730231b6f5db9501c55f99b41198

I know the above patch was also required though.

[kevinmarrec]

@octref Indeed I've ben able to reproduce the issue by loading vue language then typescript (which is embedded into vue)

Loading in this order vue => typescript does not work but typescript => vue works.

[atinux]

Do you need any help for adding a fix @octref ?