avocado
avocado copied to clipboard
avocado-framework
Spliting out optional_plugins from Avocado core repo
This issue is to track the progress of splitting optional_plugins from the Avocado main repo.
My only suggestion is to remove the word "optional" since plugins already give us the idea of plug&play. Maybe we could have an "avocado-plugins-extra" repo that will host all plugins that are extra but developed by us.
Any thoughts?
The main problem we have to solve here is the one of testing across multiple repositories. With Avocado-VT, for instance, it tests itself with different Avocado versions, including the latest version from the master branch. While not ideal (a mistake committed to Avocado can still break Avocado-VT), this "works" because Avocado-VT is very active, so a regression would be caught fairly quickly.
I believe the ideal situation, if it was not clear from the previous sentence, would be one in which a CI job for Avocado-VT would be triggered alongside with the change proposal on Avocado, but its results would serve as advice, and not as a gating mechanism.
But, we lack the ideal situation on GitHub. Also, Avocado's optional_plugins are not as active, so a breakage in this model can go a long time without being caught. In my opinion the compromise for a first milestone, would be to include the testing of the eventually split plugins as part of the Avocado's CI jobs. This can include manually obtaining the latest version of the plugins from their repos, and testing them on every Avocado PR.
See https://github.com/avocado-framework/avocado/pull/4608 for an example.
But, we lack the ideal situation on GitHub.
Could you please elaborate here?
See #4608 for an example.
IIUC, this example is only cloning the external repos and running the tests. Why we can' t do that with GH? Not sure if I understood.
The only thing that I would recommend changing on that PR is to move the functional tests to the plugin itself inside tests/ folder of individual plugin. We could still trigger plugins' tests from the main Avocado git repo, IMO.
But, we lack the ideal situation on GitHub.
Could you please elaborate here?
We can not trigger project "X"'s GH actions from a PR sent to project "Y", and still have its result seen on project "Y" for advisory purporses.
See #4608 for an example.
IIUC, this example is only cloning the external repos and running the tests. Why we can' t do that with GH? Not sure if I understood.
We probably can do that with GH, just like I did with Cirrus CI. But that's not the ideal situation I described.
The only thing that I would recommend changing on that PR is to move the functional tests to the plugin itself inside tests/ folder of individual plugin. We could still trigger plugins' tests from the main Avocado git repo, IMO.
Agreed.
We might be able to do this. I was reading last week something about this. I will try to investigate.
I started a discussion about this at https://github.com/avocado-framework/avocado/discussions/4756 The last comment tries to summarize our discussion from today https://github.com/avocado-framework/avocado/discussions/4756#discussioncomment-1023848
This may become easier once we have better module testing. Expecting such testing features to be part of the autils effort. Until then, I'm closing this issue as this is not a priority.
@beraldoleal feel free to reopen if you think differently.
Thanks!