Automation analysis

[simcha-vos-from-tu-delft]

I'm with the Software Engineering Research Group at the TU Delft, and I am investigating automations. I have looked at your repository and which GitHub workflows and Maven plugins you are using. I'll give you a quick summary of what I found, and the automation tasks I recommend you to focus on next (:

Level of maturity	Basic	Intermediate	Advanced
Collaboration	‎ ✅ Completed this level!
	✔️ Prepare or create documentation artifacts	❌ Generate documentation from source code ❌ Commit validation ❌ Bot commits	✔️ Issues or PRs management ❌ Publish documentation
Code quality	‎ ⚠️ Still working on this level!
	❌ Run tests ❌ Static code style analysis	✔️ Static code quality analysis ❌ Test coverage and validity ❌ Generate test reports ❌ Automatic code formatting	❌ Verify packaging correctness ❌ Vulnerability scans ❌ Sign artifacts ❌ License checks
Development	‎ ⚠️ Still working on this level!
	✔️ Build environment configuration ❌ Package management	❌ Build files configuration	❌ Optimization
Artifacts	‎ ✅ Completed this level!
	✔️ Code compilation ✔️ Dependency management of artifact	✔️ Build tasks, resources and configuration ✔️ Packaging ✔️ Generate source and metadata artifacts ✔️ Release tagging ✔️ Publish artifacts to a registry ❌ Source code version control	❌ Generate release notes ❌ Containerization ❌ Push container to remote

And now? Next steps!

It is not always clear which automation tasks should be prioritized. It is however important to balance your automation efforts, as a uniform level of maturity is most productive. I'm here to help! Below is a list of tasks that you can work on to help level up your maturity across the automation domains:

• Implement Package management; implemented by 36% of GitHub repositories
• Implement Run tests; implemented by 23% of GitHub repositories

What do you think?

Do you think my analysis is correct, do these recommendations help you? Or have I missed something?

For my master's thesis I'm doing research into the use of automations in GitHub repositories. With my findings, I want to see if I can help out developers with what automations they could focus on. Do you want to help me out? Leaving a response is much appreciated!

[azoitl]

@simcha-vos-from-tu-delft thank you for the analysis of our repository and the feedback. We are always interested how we can reduce our maintenance effort and improve our quality.

I'm not sure if I read your report right but if I do then your assessment misses a lot of things. We are running tests on all of our PRs and builds. We also report on them. As part of that some license checks are done.

Some of your pointed out elements are done elsewhere:

• As we are an official Eclipse project Package generation, release management, documentation generation, code signing is done at our Jenkins instance.
• Code formatting is done in our setup for developers directly when they are saving there code.

Stuff that does not make sense for us:

• We are a library to be used by others therefore containerization and pushing containers is not usefull.

[simcha-vos-from-tu-delft]

@simcha-vos-from-tu-delft thank you for the analysis of our repository and the feedback. We are always interested how we can reduce our maintenance effort and improve our quality.

I'm not sure if I read your report right but if I do then your assessment misses a lot of things. We are running tests on all of our PRs and builds. We also report on them. As part of that some license checks are done.

Some of your pointed out elements are done elsewhere:

• As we are an official Eclipse project Package generation, release management, documentation generation, code signing is done at our Jenkins instance.
• Code formatting is done in our setup for developers directly when they are saving there code.

Stuff that does not make sense for us:

• We are a library to be used by others therefore containerization and pushing containers is not usefull.

Hello, I've further looked into running tests. I think the issue is that the mvn verify command only runs tests if they have actually been implemented, so there is not really a quick way to detect this (although from the context of the workflow it is quite clear that you are running tests). That is indeed a shortcoming here

I have heard also from other that they are handling some automations through their own instances instead of through GitHub workflows. Which I think makes a lot of sense for larger projects such as this one! In those cases, it is not possible 100% reliably pick up all of the automations, which is a shame in this case.

You're right about containerization; it is not always necessary. I think I will make some changes based on that!

Thanks a lot for your helpful and constructive comment!

[ptziegler]

Maybe just as a remark from my side: I think it's dangerous to look at GEF as a "pure" Maven project because that will give you only an incomplete view over the entire build process.

As an example:

Generate documentation from source code Publish documentation

The documentation is generated by e.g. this Ant script and deployed together with the remaining artifacts. It can then be accessed directly from within the IDE.

Sign artifacts

As @azoitl already mentioned, a lot of the heavily lifting, including signing, is done via the Eclipse infrastructure. So the configuration is primarily done by the Jenkinsfile or directly on the build node itself.

[simcha-vos-from-tu-delft]

Maybe just as a remark from my side: I think it's dangerous to look at GEF as a "pure" Maven project because that will give you only an incomplete view over the entire build process.

As an example:

Generate documentation from source code Publish documentation

The documentation is generated by e.g. this Ant script and deployed together with the remaining artifacts. It can then be accessed directly from within the IDE.

Sign artifacts

As @azoitl already mentioned, a lot of the heavily lifting, including signing, is done via the Eclipse infrastructure. So the configuration is primarily done by the Jenkinsfile or directly on the build node itself.

Thanks for your response! I am indeed also going to look at how often scripts and other external infrastructure are used for automation, since it is not so trivial for me to also investigate that!

I have a final question to help me better understand your interest in tools like this. Given that this tool would be extended to cover more than just maven and GH workflows, would you be interested in using such a tool, as a way to easily report on your current state of automations, and to get recommended next automations to work on? For example, if it could be available as a badge in your readme? Thank you!

[ptziegler]

I have a final question to help me better understand your interest in tools like this. Given that this tool would be extended to cover more than just maven and GH workflows, would you be interested in using such a tool, as a way to easily report on your current state of automations, and to get recommended next automations to work on? For example, if it could be available as a badge in your readme?

To be completely honest, I think a goal-driven automation would be much more efficient than simply automating for the sake of automation.

For example: Automating the required version increases and dependency management start of each release cycle .

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.