Add Luau language

[robloxiandemo]

Description

Resolving this issue, this PR aims to finally add Luau support to the platform.

Checklist:

• [x] I am adding a new language.
  • [x] The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • .luau 3.9k files
  • [x] I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s) [MODIFIED]:
      • createProcessor.luau
      • createSignal.luau
      • equals.luau
      • isEmpty.luau
      • none.luau
      • replaceMacros.luau
      • timeStepper.luau
      • toSet.luau
      • ~signal.luau~
      • ~vector.luau~
    • Sample license(s) [MODIFIED]:
      • createProcessor.luau - (https://github.com/vinum-team/Vinum/blob/b80a6c194e6901f9d400968293607218598e1386/LICENSE) MIT
      • createSignal.luau - (https://github.com/Floral-Abyss/recs/blob/master/LICENSE.md) MIT
      • equals.luau - (https://github.com/vinum-team/Vinum/blob/b80a6c194e6901f9d400968293607218598e1386/LICENSE) MIT
      • isEmpty.luau - (https://github.com/duckarmor/Freeze/blob/670bb6593d8cc54cdcbb96cd94d1273ee0420abb/LICENSE) MIT
      • none.luau - (https://github.com/duckarmor/Freeze/blob/670bb6593d8cc54cdcbb96cd94d1273ee0420abb/LICENSE) MIT
      • replaceMacros.luau - (https://github.com/grilme99/Flow/blob/973ac39fe52de9ec9407d7ae3d9ab62867b9e982/LICENSE-Brooke) MIT
      • timeStepper.luau - (https://github.com/Floral-Abyss/recs/blob/master/LICENSE.md) MIT
      • toSet.luau - (https://github.com/duckarmor/Freeze/blob/670bb6593d8cc54cdcbb96cd94d1273ee0420abb/LICENSE) MIT
      • ~signal.luau - (https://github.com/highlightjs/highlightjs-luau) MIT~
      • ~vector.luau - (https://github.com/RobloxianDemo/Roblox-Modules) MIT~
  • [x] I have included a syntax highlighting grammar:
    • https://github.com/JohnnyMorganz/Luau.tmLanguage – MIT
  • [x] I have added a color:
    • Hex value: #14a5ff
    • Rationale: Logo color from https://luau-lang.org/ at the time of writing.
  • [x] I have updated the heuristics to distinguish my language from others using the same extension.

[robloxiandemo]

Hey! I'd like to ask if a CI could be ran for this, as I wasn't able to check it myself under the fork.

[moo1210]

I strongly support including support for Luau, which for context is an open-source Lua 5.1 fork, with many backward-compatible features (and therefore syntax) changes built onto it over the years.

With that being said, I do think it's worth noting that Vector3 (or any other Roblox types/globals) probably shouldn't be included in this. Standalone Luau has no built-in Vector3 type or globals such as game, this seems largely out of scope.

[robloxiandemo]

I'll be going through to fix up my commits with your suggested changes, thank you!

[robloxiandemo]

I strongly support including support for Luau, which for context is an open-source Lua 5.1 fork, with many backward-compatible features (and therefore syntax) changes built onto it over the years.

With that being said, I do think it's worth noting that Vector3 (or any other Roblox types/globals) probably shouldn't be included in this. Standalone Luau has no built-in Vector3 type or globals such as game, this seems largely out of scope.

I will say that it seems the larger scope of usage of Luau is under the Roblox platform.

[phoriah]

I strongly support including support for Luau, which for context is an open-source Lua 5.1 fork, with many backward-compatible features (and therefore syntax) changes built onto it over the years. With that being said, I do think it's worth noting that Vector3 (or any other Roblox types/globals) probably shouldn't be included in this. Standalone Luau has no built-in Vector3 type or globals such as game, this seems largely out of scope.

I will say that it seems the larger scope of usage of Luau is under the Roblox platform.

I agree with this, better have them just in case.

[phoriah]

    * Hex value: `#14a5ff`
    * Rationale: Logo color from https://luau-lang.org/ at the time of writing.

Any chance you could update the color to #479df9 ? Not sure where you got yours from

[phoriah]

@RobloxianDemo

[robloxiandemo]

    * Hex value: `#14a5ff`
    * Rationale: Logo color from https://luau-lang.org/ at the time of writing.

Any chance you could update the color to #479df9 ? Not sure where you got yours from

Sure, I'll submit a commit for this now.

[robloxiandemo]

    * Hex value: `#14a5ff`
    * Rationale: Logo color from https://luau-lang.org/ at the time of writing.

Any chance you could update the color to #479df9 ? Not sure where you got yours from

Resolved here, thank you for your contribution!

[lewisakura]

I do hate to add to the color debate, but neither of those colors is correct for the logo. The logo uses #00A2FF:

https://github.com/luau-lang/site/blob/c4e26ca38e3d075a365f548951f05a9be0eda345/logo.svg?short_path=5525394#L2

[robloxiandemo]

I do hate to add to the color debate, but neither of those colors is correct for the logo. The logo uses #00A2FF:

https://github.com/luau-lang/site/blob/c4e26ca38e3d075a365f548951f05a9be0eda345/logo.svg?short_path=5525394#L2

Haha, thank you for pointing that out, I'll write up a commit later today to resolve that. 👍

[robloxiandemo]

I do hate to add to the color debate, but neither of those colors is correct for the logo. The logo uses #00A2FF:

https://github.com/luau-lang/site/blob/c4e26ca38e3d075a365f548951f05a9be0eda345/logo.svg?short_path=5525394#L2

Resolved here, thank you for your contribution!

[robloxiandemo]

Seems we'll have to make some changes with the grammar. I'll try to submit a few PRs later this week to resolve it.

[lildude]

Seems we'll have to make some changes with the grammar. I'll try to submit a few PRs later this week to resolve it.

Really? The current test failures aren't due to the grammar… they're ordering issues.

[robloxiandemo]

Seems we'll have to make some changes with the grammar. I'll try to submit a few PRs later this week to resolve it.

Really? The current test failures aren't due to the grammar… they're ordering issues.

I ran some private tests with the grammar shortly after seeing those failures, and eleven errors had shown up, apparently due to some backtracking issues with the language. I'd have to check out what's causing it, though I'm not entirely sure.

[phoriah]

Pinging the people from the issue to hopefully aid & speed up the process.

@checkraisefold @RBXTyler @MaximumADHD

Sorry in advance!

[lildude]

I ran some private tests with the grammar shortly after seeing those failures, and eleven errors had shown up, apparently due to some backtracking issues with the language. I'd have to check out what's causing it, though I'm not entirely sure.

I see it now. It's because of the new/NewLuau.YAML-tmLanguage file. Linguist isn't fussy about the location of the *.YAML-tmLanguage files hence it is finding it and attempting to convert it. It doesn't look like this file is intended to be used yet so the best thing would be for the grammar maintainer to move this file into it's own branch so we can use the main branch without it causing troubles.

[phoriah]

It's because of the new/NewLuau.YAML-tmLanguage file. Linguist isn't fussy about the location of the *.YAML-tmLanguage files hence it is finding it and attempting to convert it. It doesn't look like this file is intended to be used yet so the best thing would be for the grammar maintainer to move this file into it's own branch so we can use the main branch without it causing troubles.

@JohnnyMorganz

[JohnnyMorganz]

Please open a PR to delete it! I would be happy to accept. Must have accidentally been committed. I don't have suitable access to do it myself right now.

Although when I last tried to add to linguist locally I also experienced issues where the linguist TextMate grammar parser does not support non-fixed lookbehind (whilst VSCode does). Would be awesome to see if it works though! (Removing the non-fixed lookbehind requires non trivial amounts of effort so I did not get round to it)

[moo1210]

This seems more like a parsing issue, which happens regardless of which yaml file exists.

- [ ] repository `vendor/grammars/Luau.tmLanguage` (from https://github.com/moo1210/Luau.tmLanguage.git) (10 errors)
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b_\b`": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([A-Z_][A-Z0-9_]*`...": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([a-zA-Z_]\w*)\b`": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<!:\s*|->\s*)\b([a-zA-Z_]\w*)\`...": lookbehind assertion is not fixed length (at offset 8))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b_\b`": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([A-Z_][A-Z0-9_]*`...": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([a-zA-Z_]\w*)\b`": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b_\b`": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([A-Z_][A-Z0-9_]*`...": lookbehind assertion is not fixed length (at offset 12))
  - Invalid regex in grammar: `source.luau` (in `Luau.YAML-tmLanguage`) contains a malformed regex (regex "`(?<![:>.]\s*)\b([a-zA-Z_]\w*)\b`": lookbehind assertion is not fixed length (at offset 12))

[lildude]

Although when I last tried to add to linguist locally I also experienced issues where the linguist TextMate grammar parser does not support non-fixed lookbehind (whilst VSCode does)

If it helps, this is because the highlighting engine uses PCRE for performance reasons and as such the compiler enforces the PCRE rules.

[JohnnyMorganz]

Yup that's fair, same errors that I see.

The grammar in its current state is unsuitable. Fixing the grammar will require some considerable changes to make it compatible (it may even be simpler to just rewrite it)

[AsynchronousAI]

I believe Luau should be an option for better syntax highlighting supporting types, format strings, etc. and seperating Roblox projects from normal Lua projects in trending or search.

[CompeyDev]

The grammar in its current state is unsuitable. Fixing the grammar will require some considerable changes to make it compatible (it may even be simpler to just rewrite it)

Any updates on this? Would be SUPER cool to have luau support in discovery and syntax highlighting.

[JohnnyMorganz]

Any updates on this?

Nope, unfortunately I don't have the time to work on that grammar right now. Contributions welcome!

[JohnnyMorganz]

I've fixed the grammar so it should now support Linguist. Can you pull latest changes @robloxiandemo?

Also, I expect to still make improvements. I think I already read this somewhere, but just to double check, grammars automatically get updated in the repo (before the next version release)?

[robloxiandemo]

I've fixed the grammar so it should now support Linguist. Can you pull latest changes @robloxiandemo?

Hey, and good stuff! Although, I have been trying to pull them, though it just refuses to update to the following regex-related error:

I'm unsure on how or why it's attempting to use the nonexistent "NewLuau.YAML-tmLanguage" file. Any ideas on how I could get around this, or simply to resolve it?

[JohnnyMorganz]

This file was already deleted. Can you double check the submodule was updated appropriately? What git revision is checked out

[robloxiandemo]

This file was already deleted. Can you double check the submodule was updated appropriately? What git revision is checked out

I checked and it seems to be pulling this as the latest revision, a bit stumped on how to get around this to retrieve the repo's latest commit.

[robloxiandemo]

I found a workaround to the issue and have updated the submodule to the latest commit, I just have to avoid trying to update it through the script, as it seems to fetch the old commit as the head, rather than the latest one.