Add code_syntax_highlight extension

[DavG25]

An extension that highlights code syntax inside code snippets using highlight.js

Main features:

• Uses highlight.js with common languages and "GitHub" / "GitHub Dark" highlight theme
• Automatically switches highlight.js CSS theme to match the current Gradio theme in real time (dark and light)
• Can also highlight inline code snippets (<code> blocks without <pre>) if configured to do so
• Has a performance mode to reduce CPU usage (highlight is deferred until text generation stops) or live mode (highlight is applied as code appears)
• Can be configured from the UI and through global params

Preview

Standard highlight in chat mode

Inline highlight in instruct mode

Standard highlight in default / notebook mode

Standard highlight with light theme in instruct mode

Extension settings

[oobabooga]

Two questions:

1. What is the difference between this and the current markdown support?
2. What would be the cons of making this a default instead of an extension?

[da3dsoul]

Nice! I can tell that it's better than the current implementation.

Two questions:

1. What is the difference between this and the current markdown support?
2. What would be the cons of making this a default instead of an extension?

1. Markdown is only available for certain models, right? This works in instruct and chat mode
2. It seems to be better than the current method of detecting code, so it may be better as a default. Realistically, because of the false positives or just incorrect matching, I would prefer this be an extension while removing the default. The default is to not even try, and the extension tries.

[DavG25]

Two questions:

1. What is the difference between this and the current markdown support?
2. What would be the cons of making this a default instead of an extension?

1 - To my understanding, the current markdown support does not highlight code snippets based on the specified language, for example if we write the following in the chat

```python
paths = (x for x in Path(path).iterdir() if x.suffix in ('.json', '.yaml', '.yml'))
```

it will appear like a code snippet, but not highlighted based on the Python syntax. It will just appear as non-colored text. With the extension, the code inside the snippet is highlighted similarly to how GitHub does.

This is how it looks now (just markdown support):

paths = (x for x in Path(path).iterdir() if x.suffix in ('.json', '.yaml', '.yml'))

This is how it would look with the extension (markdown support + syntax highlighting):

paths = (x for x in Path(path).iterdir() if x.suffix in ('.json', '.yaml', '.yml'))

2 - The cons would be that even when the user is not using syntax highlighting:

• We would still inject and execute all the scripts
• The MutationObserver used to watch for new <code> elements could impact performance, depending on how many messages are present in the chat

Both cons can be fixed by dynamically injecting the scripts (for example if the user checks the box 'Use code syntax highlighting') and by disconnecting (basically turning off) the MutationObserver when not in use, although Gradio makes it a pain to work with the HTML DOM and JS.

I think having this as an extension provides more control for the user, and we can later add support for multiple highlight.js languages and themes, similarly to how characters can be added to the gallery extension.

[DavG25]

Closing this in favor of https://github.com/oobabooga/text-generation-webui-extensions/pull/3

The submitted standalone version should be easier to maintain and update, it also has some improvements:

• Changes to settings apply immediately, for example disabling the syntax highlighting will immediately turn off the highlight in all code snippets
• The MutationObserver is turned off when unused to not impact performance

[oobabooga]

Thank you @DavG25, I had been meaning to review the extension but got lost. I agree that it's best for you to maintain it in its own repository, and I'll be your user :)

[DavG25]

Thank you @DavG25, I had been meaning to review the extension but got lost. I agree that it's best for you to maintain it in its own repository, and I'll be your user :)

I'm happy to hear it, thank you!