Proposal: Repo specific settings for eslint-bot

[gyandeeps]

As we have more n more plugins, I have seen some need where some repos inside ESLint org may need to deactivate the behavior of a particular plugin and not make the plugin do something slightly different (this might be a rare case).

Proposal

• Start using settings files for eslint-bot, which is natively supported by Probot architecture.
• Introduce .github/eslint-bot.yml file which can store settings for each repo when needed.
• All the plugin can read these settings using the context.config function. Where a plugin also supply defaults for repos which do not have any settings.
• This feature can be used by plugins on a as needed bases. For example triage plugin needs no settings and it should be active for all repos always.

Current plugin usage

commit-message

• Default: On
• Add .github/eslint-bot.yml file inside eslint.github.io and tsc-meetings repo to disable this plugin.

release-monitor

• Default: Off
• Add .github/eslint-bot.yml file inside eslint repo to enable this plugin.

eslint-bot.yml

• Have object for each plugin out their and inside that object store different settings for that plugin.
• one common setting will be active with true or false value..

[not-an-aardvark]

I'm not opposed to this, but I'm also not convinced that it's necessary based on our current set of plugins. It seems like the only plugin that currently needs any configuration is commit-message (the release-monitor plugin will work fine if it's always enabled, even if it doesn't do anything useful on other repositories). We don't have many repositories and we don't need to change configuration very often, so I'm not sure a configuration system is worth the additional complexity.

If we do add this, I don't think we should put the config file in the .github folder, because it's not intended to be used by GitHub.

[gyandeeps]

If we do add this, I don't think we should put the config file in the .github folder, because it's not intended to be used by GitHub.

Probot is built by the Github team and they store all the config inside .github folder. That where probot is going to read from.

I think implementing this would help us in the future also when we introduce more plugin. I think release-monitor need some settings as it will put status on PR about patch release and which is not necessary for repo which do not have releases.

These technique will help us enchance our plugins to be more robust by behaving based on the repo. For example, needs-info plugin can post different messages based on the repo.

[not-an-aardvark]

Probot is built by the Github team and they store all the config inside .github folder. That where probot is going to read from.

My interpretation of the .github folder is that it stores files intended to be used by the github.com website. I don't think it's relevant that we're using a tool created by people who work at GitHub.

I think implementing this would help us in the future also when we introduce more plugin.

I agree -- I'm just not sure we've reached a point yet where it's worthwhile. In my opinion, the downsides are:

1. We would effectively be committing to a public API for eslint-github-bot, so that projects could create a configuration for it. This seems like it would limit our ability to make changes in the future without also changing a bunch of config files in different repositories.
2. When we add a new plugin for eslint-github-bot, we might need to update several other repositories to configure the desired behavior for the plugin.
3. Whenever we perform an action on a repository, we would need to do an additional fetch to determine whether the config has changed.
4. It would add some complexity to eslint-github-bot (this isn't a big deal, but it's worth considering).

What if we stored the configuration for all projects in the eslint-github-bot repository instead? That seems like it would avoid the first three issues.

[gyandeeps]

My interpretation of the .github folder is that it stores files intended to be used by the github.com website. I don't think it's relevant that we're using a tool created by people who work at GitHub.

I cant comment on this point as this is how Probot works. Any changes first needs to be made into probot. Personally, it doesnt matter as .github to me is a place to keep settings/configs for github related stuff.

We would effectively be committing to a public API for eslint-github-bot, so that projects could create a configuration for it. This seems like it would limit our ability to make changes in the future without also changing a bunch of config files in different repositories.

This is an internal plugin and we have full jurisdiction over all the repos. And 90% of cases repo would use the default behavior.

When we add a new plugin for eslint-github-bot, we might need to update several other repositories to configure the desired behavior for the plugin.

Why, again 90% of the times no changes are needed. Rarely we need changes.

Whenever we perform an action on a repository, we would need to do an additional fetch to determine whether the config has changed.

We already do so many interactions so doing another useful one is not an issue. Also this is not a time critical app. Also that how probot architecture has been designed.

It would add some complexity to eslint-github-bot (this isn't a big deal, but it's worth considering).

I dont think so. Again i think these settings should be used in rare scenarios and not all the time. Even adding another plugin to eslint-bot increases the complexity of eslint-bot..

Overall, just trying to make things better and more robust. I believe its a good infrastructure to leverage to make the bot smart. Using needs info label and posting the current message doesn't make any sense for example.

What if we stored the configuration for all projects in the eslint-github-bot repository instead?

I am not opposed to it but i wanted to stick to what the Probot API provides us.

[platinumazure]

For what it's worth, in my WIP implementation of auto-accepting rule issues based on champion and :+1: count, due to limitations in the GitHub webhooks (i.e., no webhook for reactions), I've had to consider using a scheduler for checking all open issues/PRs and in that case I would want repository-level configuration. (Of course, I would prefer that the limitations of the webhooks be addressed instead, but that's not where we are at right now.)

[nzakas]

Repo-level configuration is a good idea. I'm sure there will be many more plugins that we will need on the main repo that don't make sense for the others.

[gyandeeps]

I did implement this here at work as I use the same bot at work also. It works fine and it has not created any issues. I think we can implement this here and turn off certain plugin for certain repos. In the future leverage the same infrastructure to have plugin options, etc

[not-an-aardvark]

My point of view is the same as it was in https://github.com/eslint/eslint-github-bot/issues/46#issuecomment-353750457. I don't object to adding repo-specific configuration, but I think it would be a lot better for that configuration to be stored in the eslint-github-bot repository and loaded at startup, rather than having to load a config file from GItHub on every request. This would allow us to keep everything in one place without needing to make PRs to many repos when we add a plugin.

[gyandeeps]

@eslint/eslint-team please let me know what you all think so that we can make a decision and close the issue accordingly. I dont want to keep the issue open if we are not going to act on it.