Decide whether the exercises in config.json should follow any specific order

[petertseng]

In Exercism v1, we had the rule to organise all exercises in config.json by difficulty. In Exercism v2, this does not make much sense anymore since it doesn't reflect the order that students will get the exercises. So, what do we want to do?

First, decide whether core exercises should be sorted relative to side exercises:

• All core exercises go before any side exercise
• Core exercises interspersed with side exercises, according to whatever sorting rule is decided below:

Then, decide what is the sorting rule for side exercises:

• Continue to sort by difficulty even though it doesn't make sense
• Sort by (unlocking core exercise, difficulty)
• Explicitly no order; Wild West, anything goes.
• Some other idea goes here

If we decide on an ordering other than the Wild West option, consider making a Travis CI check that ensures that the order has been followed.

[sshine]

All core exercises go before any side exercise

I think we should continue with this practice.

Core exercises are already constrained to the order they appear on the website. Interspersing with non-core exercises makes that constraint less clear and more prone to errors.

Continue to sort by difficulty even though it doesn't make sense

I agree, it doesn't make any sense.

Sort by (unlocking core exercise, difficulty)

The drawback here is that when core exercises are replaced, the unlocking exercise may change, requiring a reorder of ~~unaffected~~ exercises only affected by the "unlocked_by": ... attribute. This creates a lot of traffic in config.json.

How about alphabetical by exercise name?

Then exercises are only moved for the purpose of changing them to core exercises, and there's only one sorting key.

Update: And it is the order produced by ls, so presumably it is the order in which some automated processes handle them. (I'm not sure that counts for anything, but I appreciate the symmetry even though it isn't guaranteed or universal.)

making a Travis CI check that ensures that the order has been followed.

This is a good idea. I'll look into this. Should it be a shell script using jq so other tracks may adopt it if they like?

[petertseng]

Sort by (unlocking core exercise, difficulty)

The drawback here is that when core exercises are replaced, the unlocking exercise may change, requiring a reorder of exercises only affected by the "unlocked_by": ... attribute. This creates a lot of traffic in config.json.

True. I like to avoid the unnecessary traffic.

How about alphabetical by exercise name?

Indeed, that does remain pretty stable.

Note that this will make it harder to understand the track unlock structure by simply reading the JSON file without tool assistance. It may become necessary to use tools such as https://github.com/exercism/configlet/blob/master/cmd/tree.go or https://github.com/petertseng/exercism-problem-specifications/blob/list-tracks/tree.rb to achieve that.

I've decided I'm okay with this because reading the JSON file unassisted is not something I would have done to figure out the unlock structure anyway; there's too much extra information in there.

So, as long as anyone else interested can agree that the disadvantage is acceptable, alphabetical is good.

[sshine]

@kytrinyx: Do you have an opinion on whether to make decisions like these standardized across tracks?

[kytrinyx]

I don't have a strong opinion about this one. I've not had to maintain the ordering at all, so I don't know what's (more) painful. @ErikSchierboom @F3PiX -- do you have any thoughts on this?

[emcoding]

I rearranged the Ruby config.json to list the core exercises first, and that made subsequent changes way easier to do, so ➕ 1 for cores first.

For the side exercises, I'd prefer an alphabetical order. But, I'd hate to need to fix that order all at once (for ~ 100 side exercises in Ruby), just because a tool enforced it. I have no problem at all working with the current disorder.

One other thing to consider: put the deprecated exercises out of the way, at the end of the file.

[ErikSchierboom]

My vote goes to:

• Core exercises first, in the order in which they appear
• Side exercises second, in alphabetical order
• Deprecated exercises third, perhaps in alphabetical order, but this doesn't matter much

[kytrinyx]

@F3PiX Would you be OK with a tool that enforces it if we script the PR to fix all the tracks before the requirement goes into effect?

[emcoding]

That would be okay - even better - for Ruby 👍

As I learned a bit about other tracks in the meantime:

• There are tracks that don't have exercises marked as core in config.json. The website picks 10. If that's the first 10, it would be weird if that's all exercises starting with 'A'. 😂
• There are tracks that run extra tests with regard to config.json. (I know Rust does.)

Maybe it's better to let tracks opt-in for the tool? Or parts of the tool? Until all the tracks are more consistent.

[kytrinyx]

Maybe it's better to let tracks opt-in for the tool?

@F3PiX That's a good call. We can do this by passing a flag to opt in and change the Travis CI config at the same time as we update the config.json. (We could also have a flag to opt out but then we'd have to spuriously change the tracks that aren't changing, which seems less than ideal.)

[petertseng]

Personally I see no need to enforce any sort of requirement on any tracks. ~~Of course, I can't dictate what others choose to spend their time on, but I would estimate that others should also place very little value and priority on enforcing this sort of thing.~~ Edit: The sentence was too presumptuous and not necessary for the message I want to convey. It will be struck out for I no longer want it as part of this message.

This track is taking on a particular ordering by the free choice of its maintainers, based on weighing the merits of making this choice. If other tracks wish to make the same choice because they agree with the merits, so be it. On the other hand, if their values are different such that the merits here become demerits there, I cannot condone forcing our choices upon them.

I do support making tools usable by as many tracks as possible, so that tracks that wish to make a certain choice have easy ways to do so.

That means that my answer to the following previously-asked question

Should it be a shell script using jq so other tracks may adopt it if they like?

would be "yes, or using any other method that is similarly track-agnostic"

[kytrinyx]

I would recommend making it part of configlet, since all the tracks use that anyway. But we don't have to make it part of the CI build, just a tool that is available and can be used if maintainers want to.

[sshine]

It appears that #936 (which has just been merged) creates an order:

Re-order the practice exercises using the following ordering:

1. Core exercises, retaining their existing ordering
2. Non-core exercises, ordered by the order of the core exercise that unlocks them, then by difficulty and then alphabetically.
3. Bonus exercises, ordered by difficulty then alphabetically.

This creates one resolution of the initial question:

All core exercises go before any side exercise Continue to sort by difficulty even though it doesn't make sense Sort by (unlocking core exercise, difficulty)

Except difficulty is moved down in sorting priority.

consider making a Travis CI check that ensures that the order has been followed.

I do not know if this order is preserved by configlet v3, or if additional scripting is required.

I am closing this issue, since the original intent was resolved.

Let us consider more CI checks in the process of unbreaking CI (#952).

[sshine]

I'm re-opening this issue with the intent of carrying this information into the general documentation.