Highlighter Languages no longer configurable and target `highlighter` in `make.js` broken?

[tbfe-de]

In comparison to earlier versions I observed that the size of remark-min.js grew by 200% (i.e. it has tripled).

This seems to be due to syntax highlighting which since v0.13 includes a huge number of languages.

The Wiki still describes how the highlighter can be configured to include only some languages:

---

Excerpt from the Wiki-Text:

Building remark highlighter

In addition to building remark itself, there's an additional highlighter target that will bundle Highlight.js into the src/remark/highlighter.js file.

node make highlighter

This will bundle Highlight.js itself, its styles (with a few exceptions), and the languages specified in the package.json file:

{
  ...
  "config": {
    "highlighter": {
       "languages": [
         "javascript",
         "ruby",
         ...
       ]
    }
  }
}

---

Back to the problem analysis:

It appears v0.12 is the last version which actually supported this with the following code in make.js:

      , HIGHLIGHTER_LANGUAGES:
          config.highlighter.languages.map(function (language) {
            return '{name:"' + language + '",create:' +
              cat(highlightjs + 'languages/' + language + '.js') + '}';
          }).join(',')

The above fragment has been replaced in v0.13 with the following, which seems to unconditionally include all languages for which a .js-file is present in vendor/highlight.js/src/languages:

      , HIGHLIGHTER_LANGUAGES:
          ls(highlightjs + 'languages/*.js').map(function (file) {
            var language = path.basename(file, path.extname(file))
            return '{name:"' + language + '",create:' + cat(file) + '}';
          }).join(',')

Then v0.14 of added the following code with a call to sort with the purpose (as stated in the comment) to bring cpp.js to the front of the list:

      , HIGHLIGHTER_LANGUAGES:
          ls(highlightjs + 'languages/*.js')
            .sort(function (a, b) {
              // Other languages depend on cpp, so put it first
              return a.indexOf('cpp.js') !== -1 ? -1 : 0;
            })
            .map(function (file) {
              var language = path.basename(file, path.extname(file))
              return '{name:"' + language + '",create:' + cat(file) + '}';
            }).join(',')

This seems to break bundling the highlighter as

make highlighter

now ends with the following error:

remark/node_modules/shelljs/src/common.js:360
      if (config.fatal) throw e;
                        ^

Error: sort: no such file or directory: function (a, b) {
              // Other languages depend on cpp, so put it first
              return a.indexOf('cpp.js') !== -1 ? -1 : 0;
            }
    at Object.error (remark/node_modules/shelljs/src/common.js:71:27)
    at remark/node_modules/shelljs/src/sort.js:74:14
    at Array.forEach (native)
    at Array._sort (remark/node_modules/shelljs/src/sort.js:71:9)
    at Array.<anonymous> (remark/node_modules/shelljs/src/common.js:344:25)
    at bundleHighlighter (remark/make.js:156:14)
    at Function.target.highlighter (remark/make.js:26:3)
    at Object.global.target.(anonymous function) [as highlighter] (remark/node_modules/shelljs/make.js:36:40)
    at remark/node_modules/shelljs/make.js:48:27
    at Array.forEach (native)

I suppose the shelljs/sort function is modeled after the OS sort command, expecting a file name as argument (and not in a programming language library style, expecting an "order defining" argument to compare elements).

(Note that I'm neither a JavaScript developer nor do I have any deeper knowledge wrt. to node/shelljs, but glancing over https://github.com/shelljs/shelljs#sortoptions-file--file- and https://github.com/shelljs/shelljs/blob/master/src/sort.js seems to confirm my analysis.)

If I am right and the code of the highlighter target in make.js needs to be fixed anyway, I'm strongly in favor to keep the option to configure the highlighter languages, as most any presentation will not need all the supported languages. Selecting only the ones that are used cuts down the size of remark-min.js substantially.

Currently - with all languages enabled - the highlighter contributes twice as much JavaScript code as remark itself.

[abelards]

This looks fixed, right?