singa icon indicating copy to clipboard operation
singa copied to clipboard

Adding dependabot support

Open moazreyad opened this issue 4 years ago • 5 comments

Dependabot is a native tool in github that automatically checks the dependencies of the project and creates pull requests for outdated or insecure dependencies. See how it works from here.

Currently it will analyze only the dependencies defined in pom.xml in SINGA because other dependencies are hard coded in configuration or shell scripts which are not supported by this tool. In future commits, the dependencies will be moved to standard files such as python requirements.txt so that they become available to dependabot and similar tools.

moazreyad avatar Aug 18 '20 20:08 moazreyad

shall we move all CI functions from travis to github to avoid the duplicated testing?

nudles avatar Aug 30 '20 03:08 nudles

shall we move all CI functions from travis to github to avoid the duplicated testing?

Yes, I will create another PR for the work in progress on replacing travis with github actions.

moazreyad avatar Aug 30 '20 10:08 moazreyad

Both java and python dependencies are enabled now. Dependabot will check python daily and java monthly. It will create a PR for upgrade insecure or outdated dependencies in SINGA like this java PR and this python PR.

notes:

  • Dependabot works only on the master branch. It will be enabled after merging with master.
  • The proposed requirements.txt for python is not complete. Later we need to add all the requirements for all the configurations.

moazreyad avatar Aug 30 '20 17:08 moazreyad

let's consider this PR in the next version. Now we support both conda and pip. One potential issue is that protocol buffer (PB) is not backward compatible. If there is a new version of PB, we may have to update many dependent libs to remove the warnings from the dependent bot.

nudles avatar Sep 27 '20 07:09 nudles

let's consider this PR in the next version.

Ok. Although we can at least enable the dependabot support and badge, even though we don't resolve the problems that dependabot report. Just like we enabled lgtm but we don't have to fix all its errors, and we enabled codecov but we don't have to test with 100% coverage.

One potential issue is that protocol buffer (PB) is not backward compatible. If there is a new version of PB, we may have to update many dependent libs to remove the warnings from the dependent bot.

We don't have to update immediately all the dependencies that dependabot found obsolete or insecure. At least we can have the results of the dependency analysis and the singa team can decide the work priorities. The PB team are working now on a release candidate for version 4 and planning to upgrade singa PB to at least version 3 can be useful.

Now we support both conda and pip.

We don't support machine readable requirements that can be found by Github tools. The instructions to installing singa dependencies with pip is only human readable in the web site documentation, or in the conda scripts. We need to extract these dependencies into something like requirements.txt so that Github (and other tools) can find them.

moazreyad avatar Sep 27 '20 11:09 moazreyad