neos
TASK: Add integration build on a neos distribution
Adds an additional build matrix entry that will try to run tests within a neos-development-distribution to make sure things still work.
Follow-up to #2200
There were 2 failures:
Neos\Flow\Tests\Functional\Mvc\RoutingTest::routesWithoutRequestedHttpMethodConfiguredResultInA404 with data set #0 ('GET', 404)Failed asserting that 500 matches expected 404.
Neos\Flow\Tests\Functional\Mvc\RoutingTest::routesWithoutRequestedHttpMethodConfiguredResultInA404 with data set #1 ('PUT', 404)Failed asserting that 500 matches expected 404.
😬
I believe we've touched this subject before on Slack? About installing a Neos CMS distribution and run tests, and I'm still oponent of it - it will take longer for Flow to run tests and turn Flow into a subproject of the CMS platform build on top of it.
It's should be the responsibility of Neos CMS to run tests, when a branch changes (being master, a concrete version etc)
I believe we've touched this subject before on Slack? About installing a Neos CMS distribution and run tests, and I'm still oponent of it - it will take longer for Flow to run tests and turn Flow into a subproject of the CMS platform build on top of it.
Well, it's a single test run like if we did test for one additional PHP version and it runs in parallel to the others, so not much longer (it was a bigger issue with travis and the limited runners that were already stacking up and Flow builds were blocking Neos builds etc.). Regarding the "turn Flow into a subproject of CMS" I don't really agree. I see it as a responsibility of the Framework to not break it's major usage platform and this is only a small measure to get earlier feedback on such an accidential breakiness. It is supposed to make releases more reliable. Now I could agree on making the Neos build step not mandatory (because the failure could be a necessary change in Neos and that shouldn't block from merging something into Flow) and/or running it only for merges.
It's should be the responsibility of Neos CMS to run tests, when a branch changes (being master, a concrete version etc)
That would be nice, but I think it's not realistic to rerun all Neos CMS tests whenever the Flow branch changes, as that can be a lot more often than necessary (would it run on every single commit to the base branch or only for merges? What about getting feedback inside the feature branch of a PR already?). Also, the Neos CMS test suite is longer, so it would only make the feedback on Neos slower (test runs are not infinitely parallel) and last but not least, when you merge a PR on Flow, you would have to remember to consult the Neos actions to see if it broke something. The feedback should IMO be visible where the change happens.
- Downgrading doctrine/persistence (2.1.0 => 1.3.8): Extracting archive
- Downgrading doctrine/inflector (2.0.3 => 1.4.3): Extracting archive
- Downgrading doctrine/common (3.1.1 => 2.13.3): Extracting archive
- Downgrading doctrine/orm (2.8.2 => 2.7.5): Extracting archive
Now I'm interested in why the neos-dev-collection setup downgrades those dependencies...
when you merge a PR on Flow, you would have to remember to consult the Neos actions to see if it broke something. The feedback should IMO be visible where the change happens.
I will have to be the devils advocate on this one 😈 (with nothing but love and care involved!)
What if the author don't work with Neos CMS but only the Flow Framwork? How do we go about a change in Flow, breaking something in the CMS in regards of the responsibility for creating a patch for the CMS platform and will it be blocking the original PR?
when you merge a PR on Flow, you would have to remember to consult the Neos actions to see if it broke something. The feedback should IMO be visible where the change happens.
I will have to be the devils advocate on this one 😈 (with nothing but love and care involved!)
Always welcome! :)
What if the author don't work with Neos CMS but only the Flow Framwork? How do we go about a change in Flow, breaking something in the CMS in regards of the responsibility for creating a patch for the CMS platform and will it be blocking the original PR?
Totally valid and I agree, sometimes a failure in this step should not block the PR - that's what I tried to address with
I could agree on making the Neos build step not mandatory (because the failure could be a necessary change in Neos and that shouldn't block from merging something into Flow) and/or running it only for merges.
Not 💯 sure yet which is preferable, but I lend towards this build step being optional (i.e. failure not blocking a merge). Otherwise after merge the build pipeline could still be "red", but not visibly in the PRs (though a "red" visibility of something that can not be taken care of by the PR contributor is also bad - see the psalm step). Curious what the others think how we should handle this.
@neos/upmergers wdyt? Should we have this in our builds? If so, should it be visible on each PR or only in merge commit builds (i.e. the Actions tab)
@kitsunet didnt we just talk about this? That flows tests doesnt work when neos is present? So i guess this is an important test.
I would suggest that we run Neos tests after a merge to the main branch(es), possibly with some slack notification if they fail. This way we won't slow down / intercept Flow PRs but will be notified when a change leads to a broken Neos distribution
