Let's reduce the number of build targets

[rbarrois]

The problem

We currently have 17 targets in our test matrix.

Since most of the time is spend on setting up the host and installing dependencies, I'm wondering if we could cut down a bit.

Proposed solution

We could look at:

• Running a single build per Python version
• Adding some kind of --keep-going to our testing runs, so that a maximum number of errors can be caught per run (thus making it easier to group jobs)