Regression Testing of REST API

[IanMayo]

Is your feature request related to a problem? Please describe.

At this early point in its life Soul is going through lots of changes. It would be quite easy for the addition of a new feature, or implementation of a bug-fix to break some existing part of the API.

Soul developers could work with more confidence if there was an automated test-suite that verified the full API worked.

Describe the solution you'd like

Introduction of a GitHub action that runs a test-site across the full Soul REST API. This would run in each Pull Request, and give the maintainer(s) confidence that a new feature (or bug-fix) doesn't break any existing functionality.

Since Soul is a js project, a library such as Frisby may be an obvious choice.

I guess we'd build up a chinook.db instance that could be used for the unit testing. This would grow as required. So, when we were developing/reviewing the export views feature, we would have added a SQL VIEW to the database, to check it supported views.

Any other candidates for a library to run unit tests on the Soul REST API:?

Describe alternatives you've considered

• Manual testing - that's the only current option.

Additional context Add any other context or screenshots about the feature request here.

[IanMayo]

Ok, I've found this existing test content: https://github.com/thevahidal/soul/blob/main/core/src/controllers/rows.test.js

This may have fallen behind the growth of new features, but we should be able to update/extend this test suite, and include it as a Pull Request action.

[thevahidal]

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind. But to run tests in the pipeline is a must-have and I'll try to do it asap.

[thevahidal]

We should have an strict acceptance criteria to enforce tests on newly added features, and also should write tests for the parts that lack any.

[IanMayo]

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind. But to run tests in the pipeline is a must-have and I'll try to do it asap.

Here is how to do it in a GitHub action: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

You'd then go into repo settings / branches, create a new config for develop, and require that status checks are complete.

[thevahidal]

Thanks for the link, I'll make it a priority.