tm4e icon indicating copy to clipboard operation
tm4e copied to clipboard

Automatically search for grammar on GitHub when none is configured yet

Open mickaelistria opened this issue 7 years ago • 15 comments

GitHub textmate organization is actually a repo of TextMate grammars. When a file is opened with Generic Editor and expects TM4E to participate but tm4e has no grammar, it should open a dialog to user suggesting to search for grammars on GitHub and allow to fetch them. A query like "<string>cpp</string>" user:textmate repo:*.tmbundle extension:plist (in case of .cpp) can for example show interesting results.

mickaelistria avatar May 03 '18 12:05 mickaelistria

I love this great idea! But it means that we must use git protocol to download it?

angelozerr avatar May 04 '18 10:05 angelozerr

GitHub does allow to access raw content of repos through http: https://raw.githubusercontent.com/textmate/c.tmbundle/master/Syntaxes/C.plist

mickaelistria avatar May 04 '18 10:05 mickaelistria

Yes you are right, but how we can map a textmate grammar with this URL? I mean genericeditor opens a file with extends .c, how we can discover the well URL https://raw.githubusercontent.com/textmate/c.tmbundle/master/Syntaxes/C.plist? Perhaps we should have a textmate serach engine which does that?

angelozerr avatar May 04 '18 10:05 angelozerr

An idea is to have a server like https://tmsearch.com/ which does the same thing for npm https://npmsearch.com/ (to find package)

angelozerr avatar May 04 '18 10:05 angelozerr

@ainslec perhaps this textmate search engine could b ean issue for https://eeyo.io/iro/ ? What do you think about that?

angelozerr avatar May 04 '18 10:05 angelozerr

but how we can map a textmate grammar with this URL?

We don't map it, we send the search request to GitHub and tell user "We found this grammar from GitHub, do you want to use it?". No additional registry involved, GitHub is the registry and search engine.

mickaelistria avatar May 04 '18 11:05 mickaelistria

"We found this grammar from GitHub, do you want to use it?"

Sorry I have not understood your idea?

An idea is that we could use teh APi Search of github https://developer.github.com/v3/search/

For instance:

  • we can retrieve list of github repositories with https://api.github.com/search/repositories?q=tmBundle
{
  "total_count": 1673,
  "incomplete_results": false,
  "items": [
    {
      "id": 793799,
      "name": "SCSS.tmbundle",
      "full_name": "MarioRicalde/SCSS.tmbundle",
      "owner": {
        "login": "MarioRicalde",
        "id": 125900,
        "avatar_url": "https://avatars0.githubusercontent.com/u/125900?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/MarioRicalde",
....

  • after we can retrieve the URL like this https://api.github.com/search/code?q=repo:MarioRicalde/SCSS.tmbundle+extension:tmLanguage
{
  "total_count": 1,
  "incomplete_results": false,
  "items": [
    {
      "name": "SCSS.tmLanguage",
      "path": "Syntaxes/SCSS.tmLanguage",
  • we can build the raw URL by using ""full_name": "MarioRicalde/SCSS.tmbundle" and "path": "Syntaxes/SCSS.tmLanguage", https://raw.githubusercontent.com/MarioRicalde/SCSS.tmbundle/master/Syntaxes/SCSS.tmLanguage

angelozerr avatar May 04 '18 22:05 angelozerr

@angelozerr

perhaps this textmate search engine could b ean issue for https://eeyo.io/iro/ ? What do you think about that?

What kind of feature do you want it to be? I understand the concept of a .tmLanguage registry, but what role do you think Iro should play? Should it add an option to publish to a central registry, or provide a registry itself? Just trying to understand your comment.

ainslec avatar May 05 '18 06:05 ainslec

What kind of feature do you want it to be?

Just having a textmate search engine like search textmate grammar .tmLanguage by file extension which links to grammar (not host the grammar, just give a link that we can download it)

angelozerr avatar May 05 '18 06:05 angelozerr

Problem is that Iro is a static web page, nothing more. And it doesn't store anything on the serverside (because there is no serverside).

Seems like the problem is not a tech problem, but a hosting problem. Maybe the Eclipse project can host the search engine?

ainslec avatar May 05 '18 07:05 ainslec

Seems like the problem is not a tech problem, but a hosting problem. Maybe the Eclipse project can host the search engine?

I think we can consume the Github API Search for that even if it's not perfect, but IMHO I think the better clean solution is to have a search engine like we have for npm.

angelozerr avatar May 05 '18 07:05 angelozerr

Creating and hosting a search engine is not an affordable effort for this project (and is not really the topic of this issue initially) and I'm confident that sending good queries to GitHub search service would work well for the vast majority of cases.

mickaelistria avatar May 11 '18 15:05 mickaelistria

A query like "cpp" user:textmate repo:*.tmbundle extension:plist (in case of .cpp) can for example show interesting results.

Could you give me the right github URL which retrieves this grammar please?

angelozerr avatar May 18 '18 08:05 angelozerr

I'm going submarine and doing only bugfix, releng or consolidation of existing features for Photon. I'll come back to this topic in a few weeks as discovery (of Docker based LS, or TM grammars) is likely to become a hot topic for me.

mickaelistria avatar May 18 '18 08:05 mickaelistria

Ok I wanted to start something, but if you have the intetion to do it, I let you implement this feature.

angelozerr avatar May 18 '18 11:05 angelozerr