Create inventory of concepts lacking exercises

[ErikSchierboom]

Exercism v3 Practice Exercises (of which most are implementations of exercises in this repo) have a practices field. This field list the concepts that can be practiced in that exercise. What several maintainers found was that for concepts, there are very few exercises that can be used to practice that concept. One example concept that comes to mind is dates, for which we have (out of the top of my head) just two exercises: meetup and gigasecond. Contrast that with the number of string exercises (anagram, isogram, pangram, bob, the song-related exercises, and so on).

The idea of this issue is to somehow compile a list of those concepts we feel could benefit from having more exercises. We can do this several ways:

1. Create a list with all exercises and their concepts.
2. Create a list with the concepts with few (or no) exercises

The first option is a lot more work, but would have the benefit that it could be used as a reference for track maintainers when choosing an exercise to implement. The second option is easier to do.

I'd be curious in hearing your opinion.

[jiegillet]

Elixir has most of the exercises implemented (minus a few foregone and some exercises without canonical data) and they are all tagged with prerequisites, concepts and practices, so looking at config.json might be a good start for creating a full list.

[junedev]

I don't know all the practice exercises but the first thing that comes to mind is concurrency. Learning about the currency features of a language is often requested by students, so in Go and JavaScript we are in the process of adding the respective concepts. I know of parallel-letter-frequency and bank-account but since languages can have multiple aspects/features related to concurrency (e.g. for doing something in parallel, waiting for all the things to finish, thread safe data structures, communication between threads) that doesn't seem like a lot to practice all of those.

Besides that, here some other candidates from looking at the Go/JS concept lists (we don't have prerequisites/practices yet):

• date/time/duration (as Erik mentioned)
• dealing with non-ascii characters (chars, codepoints, ...); there were discussions about extending some exercises in that direction, not sure what the status is there
• inheritance / OOP beyond 1 class with a couple of methods / "abstraction" more generally speaking; my feeling here is that exercises rarely describe a situation that warrants multiple entities that have some relation etc., C# has a only 1 practice exercise listed there
• JSON might be relevant for enough languages, not sure
• Advanced error/exception handling, anything beyond just returning some error, e.g. checking for error types
• very long integers (bigint)

Please correct me if I mentioned something that has enough exercises already. As I said, I don't have all the exercises in my head.

As for the lists, I think both versions would be very helpful. @ErikSchierboom Is it really that much more work to create list 1? There are only a couple of tracks that have prerquistes/practices filled out and for those the config.json could be parsed to create a map of exercise slug to concepts (the result does not need to show which track has which concept).

[jiegillet]

One could also scrape ALL the config.json files and gather all the (unique) concepts/prerequistes/practices attached to each exercise, and clean that up.

[ErikSchierboom]

One could also scrape ALL the config.json files and gather all the (unique) concepts/prerequistes/practices attached to each exercise, and clean that up.

This is a great idea!

[junedev]

@jiegillet Yes, exactly.

[SaschaMann]

I'm unsure if this is about collection concepts that lack exercises already or not. Feel free to mark my comment as off-topic if it isn't.

Linear algebra related concepts are desperately lacking exercises, e.g. exercises that deal with matrices/multi-dimensional arrays or vectors. There are obviously exercises that require arrays of some sort but they rarely make use of their properties beyond using them as a collection of values. Some, like transpose, are really just string exercises in disguise.

dealing with non-ascii characters (chars, codepoints, ...); there were discussions about extending some exercises in that direction, not sure what the status is there

I think just about any string exercise could benefit from scenarios that include non-ascii characters. Assuming that any string is ascii is a bad but very common practice. Something like pangram would be an obvious exception because it relies on a specific definition of an alphabet.

[ErikSchierboom]

e.g. exercises that deal with matrices/multi-dimensional arrays or vectors.

I can imagine how this would be really useful, especially for tracks which language is often used in machine learning.

[junedev]

@SaschaMann I also thought about these math topics, not only matrices but also things like being able to use a trigonometric function, understanding in what format the argument needs to be supplied etc. I wasn't really sure about this but my gut feeling is the only thing from "math" packages we make use of is abs and pow most of the time.

For example, there is probably some nice exercise that could be built around rotation matrices which would combine trigonometric functions and matrix multiplication (e.g. similar to day 19 in this years advent of code, just easier).

Of course we should ensure good resources for the respective topic are linked at it is clear that the exercise involves some math etc. like we discussed on Slack recently.

Re the string exercises, I am not sure all that could be extended should be extended to non-ascii characters. I think there is also value in keeping some very simple so they can unlock early in the track, e.g. before the details of the character encoding of the respective language etc were covered. Of course maintainers can remove some tests again but that's extra work so I would aim for a good mix here.

[SaschaMann]

I also thought about these math topics, not only matrices but also things like being able to use a trigonometric function, understanding in what format the argument needs to be supplied etc. I wasn't really sure about this but my gut feeling is the only thing from "math" packages we make use of is abs and pow most of the time.

Do you think those would qualify as a concept? To me they largely seem like any other function while vectors and matrices often have special syntax or behaviour. (I'm not opposed to adding more exercises with those obviously, just curious)

Re the string exercises, I am not sure all that could be extended should be extended to non-ascii characters. I think there is also value in keeping some very simple so they can unlock early in the track, e.g. before the details of the character encoding of the respective language etc were covered. Of course maintainers can remove some tests again but that's extra work so I would aim for a good mix here.

For me it's similar to return vs print. If you let students assume things are ascii early on, they get used to it and will develop a bad habit to always do so. It depends on the (age of the) language but I think in general having those cases in prob-specs is worthwhile. Maintainers won't have to remove them as every test case is optional so the default is that they don't implement it anyway.

[ErikSchierboom]

Maintainers won't have to remove them as every test case is optional so the default is that they don't implement it anyway.

That is not necessarily true, as test generators might by default implement all test cases :) I'm not opposed to having additional, non-ascii test cases as long as we have a separate scenario with which tracks can easily exclude them if they don't want to deal with non-ascii test cases.

[SaschaMann]

That is not necessarily true, as test generators might by default implement all test cases :)

That's their choice to not follow the spec and cause issues for themselves then. There's a scenario for it anyway.

[junedev]

Do you think those would qualify as a concept?

@SaschaMann I would phrase it this way: I think both topics we talked about count as "it would be good to have practice exercises for them". The specific concept those exercises would practice depends on the language.

• a matrix related exercise might belong to the concept matrices in a language that has those as a built-in/std-lib type and to the concept arrays in a language that does not
• math functions beyond abs and pow can be the concept "math" package, another language might have different packages for different function categories, the reason I think it makes sense to practice those is that some languages have quirks in there that are good to know about (JS returns wrong results for some functions/arguments) and languages might or might not support some concept of units when providing the arguments (e.g. deg vs rad)

[ErikSchierboom]

I've gone and checked all practice exercises across all tracks to count the number of times they are practiced and used as a prerequisites:

Concept	Tracks	# practiced	# prerequisite
advanced-enumeration	ruby	8	0
agent	elixir	1	2
allocators	zig	3	3
anonymous-functions	elixir, python	4	3
arithmetic	common-lisp	9	11
array-comprehension	haxe	6	0
arrays	common-lisp, csharp, fsharp, haxe, java, ruby, vbnet, zig	37	67
ast	elixir	0	1
atoms	elixir	9	28
basics	elixir, haxe, java, python, red, ruby, z3	8	128
big-integers	csharp, fsharp	2	2
binaries	elixir	1	7
bit-manipulation	csharp, elixir, fsharp, haxe, ruby	12	13
bitflags	python	2	0
bitstrings	elixir	1	1
bitwise-operations	zig	5	5
bitwise-operators	python	2	0
blocks	red, ruby	5	4
booleans	clojure, csharp, elixir, fsharp, haxe, ruby, rust, vbnet	10	56
bools	python	8	69
builtin-functions	zig	8	8
bytes	python	1	0
case	elixir	2	52
casting	csharp, fsharp, ruby	7	7
characters	common-lisp	5	5
charlists	elixir	7	15
chars	csharp, fsharp, java, vbnet	12	38
class-composition	python	6	0
class-customization	python	6	0
class-inheritance	python	6	0
classes	csharp, fsharp, python, ruby, wren	32	48
collection-processing	fsharp	10	3
collections	factor, python	8	1
combinators	factor	3	1
comparing	csharp, fsharp	2	2
comparisons	python, z3	19	0
concurrency	csharp, fsharp	2	0
cond	elixir	5	49
conditionals	clojure, common-lisp, factor, fortran, python, red, ruby, z3, zig	64	155
conditionals-if	java	1	3
constructors	csharp, fsharp, java, ruby, vbnet	23	34
context-manager-customization	python	1	0
control-flow	zig	10	10
dataclasses-and-namedtuples	python	1	0
date-time	common-lisp	1	1
dates	ruby	1	1
dates-and-time	elixir	2	2
datetimes	csharp, fsharp	4	4
decorators	python	3	0
default-arguments	elixir	2	10
descriptors	python	2	0
dict-methods	python	8	0
dictionaries	csharp	9	7
dicts	python	8	41
discriminated-unions	fsharp	9	12
dsl	red	1	0
enum	elixir	9	88
enumerable	ruby	6	14
enumerables	csharp, vbnet	7	18
enumeration	ruby, wren	8	7
enums	csharp, haxe, java, python, rust, vbnet, zig	20	16
equality	csharp, fortran, fsharp, ruby	11	11
erlang-libraries	elixir	2	6
error-sets	zig	10	10
errors	elixir, red	4	10
evaluation	red	3	0
events	csharp, fsharp	2	2
exceptions	csharp, elixir, fsharp, haxe, ruby, vbnet, wren	25	69
extension-methods	csharp, fsharp, vbnet	8	8
fields	wren	3	0
file	elixir	1	1
filtering	common-lisp	2	2
final	haxe	3	0
flag-discriminated-unions	fsharp	2	1
flag-enums	csharp	2	1
floating-point-numbers	clojure, common-lisp, csharp, elixir, fsharp, haxe, ruby	23	25
fold	rust	2	0
for-loops	csharp, fsharp, haxe, java, vbnet	22	52
foreach	csharp, vbnet	2	2
foreach-loops	csharp	5	7
format-basics	common-lisp	3	3
function-arguments	python	6	0
functional-tools	python	2	0
functions	common-lisp, python, red, zig	25	20
functools	python	2	0
generator-expressions	python	6	0
generators	python	3	0
generic-constraints	csharp, fsharp	2	2
generic-functions	fsharp	1	1
generic-methods	csharp, fsharp, ruby	12	19
generic-types	csharp, fsharp, vbnet	9	14
generics	zig	1	1
getters	wren	1	0
globalization	csharp, fsharp	2	2
guards	elixir	6	52
hash-tables	common-lisp	1	1
hashes	ruby	5	3
higher-order-functions	csharp, fsharp, python, ruby	9	9
if	elixir	1	48
if-statements	csharp, haxe, vbnet	17	29
if-then-else-expressions	fsharp	0	21
immutability	csharp, fsharp, ruby	13	10
importing	zig	4	4
inheritance	csharp, fsharp	2	4
instance-variables	ruby	5	6
integers	common-lisp, csharp, elixir, ruby, vbnet	17	40
integral-numbers	csharp, fsharp, haxe, ruby	18	18
interfaces	csharp, fsharp	2	2
intermediates	z3	0	1
iteration	python	8	0
iterators	python	2	0
itertools	python	7	0
keyword-lists	elixir	1	1
lambda	haxe	8	0
lambdas	csharp, ruby, vbnet	3	3
lazy-evaluation	csharp, fsharp, ruby	16	3
linq	csharp, vbnet	11	0
list-comprehensions	elixir, python	14	8
list-methods	python	9	71
lists	clojure, common-lisp, csharp, elixir, fsharp, java, python, vbnet, wren	30	184
logical-operators	factor	0	1
loops	python, red, ruby, rust, wren, z3	36	122
mapping	common-lisp	3	3
maps	elixir, fsharp, haxe, red, wren	16	47
match	wren	1	0
math	factor, wren	4	0
math-operators	csharp, fsharp, ruby	23	23
maths	red	4	2
members	fsharp	8	9
methods	zig	4	4
module-attributes-as-constants	elixir	4	4
multi-dimensional-arrays	csharp, fsharp	4	4
multiple-clause-functions	elixir	9	66
mutability	rust	3	0
nil	elixir, ruby	3	6
none	python, red	8	0
nullability	csharp, fsharp	4	4
numbers	clojure, csharp, fsharp, haxe, java, python, ruby, vbnet, wren, z3	69	300
observables	csharp, fsharp	0	2
operator-overloading	csharp, fsharp, python, wren	11	2
optional-arguments	haxe	1	1
optional-parameters	common-lisp, csharp, fsharp, vbnet	5	5
optional-values	wren	2	0
optionals	zig	2	2
options	fsharp	0	30
ordering	csharp, fsharp, ruby, vbnet	10	10
other-comprehensions	python	8	0
parallellism	csharp, fsharp	2	2
parameters	csharp, fsharp, ruby	3	3
pattern-matching	elixir, haxe	11	64
pids	elixir	1	4
pipe-operator	elixir	5	7
pointers	zig	2	2
processes	elixir	4	4
properties	csharp	9	11
protocols	elixir	1	3
queues	csharp, fsharp, ruby	3	3
quotations	factor	7	4
raising-and-handling-errors	python	8	0
randomness	csharp, elixir, fsharp, ruby	13	13
ranges	elixir, fsharp	13	32
records	fsharp	0	19
recursion	csharp, elixir, fsharp, haxe, ruby, wren	30	52
regular-expressions	csharp, elixir, fsharp, haxe, python, ruby	39	25
resource-cleanup	csharp, fsharp	2	2
result	fsharp	3	3
rich-comparisons	python	9	0
script	red	1	0
sequences	python, wren	9	0
series	red	3	0
sets	csharp, fsharp, python, ruby	18	19
slices	zig	8	7
stack-effect	factor	6	8
stacks	csharp, fsharp, ruby, vbnet	4	4
streams	csharp, elixir, fsharp	2	5
string-builder	csharp, fsharp	4	0
string-formatting	csharp, fsharp, python, ruby	29	22
string-interpolation	haxe, vbnet	4	4
string-methods	python	8	65
string-methods-splitting	python	6	0
strings	clojure, common-lisp, csharp, elixir, factor, fsharp, haxe, java, python, red, ruby, rust, vbnet, wren, z3	130	393
structs	csharp, elixir, fsharp, ruby, rust, zig	26	26
switch-expressions	csharp, fsharp	3	0
switch-statement	java	2	0
switch-statements	csharp, fsharp	10	1
symbols	ruby	3	6
tail-call-recursion	elixir	4	6
tasks	elixir	1	1
text-formatting	wren	2	0
time	ruby	2	2
tuples	csharp, elixir, fsharp, python	19	73
type-coercion	zig	7	6
typedefs	haxe	4	0
unpacking-and-multiple-assignment	python	3	0
user-defined-errors	python	3	0
vectors	clojure	0	15
while-loops	csharp, fsharp	12	14
with	elixir	0	2
with-statement	python	2	0
word-definition	factor	5	8
yield	csharp, fsharp, vbnet	10	0

[junedev]

@ErikSchierboom JavaScript did not assign any "practices" keys yet so it should not be included in the list. Not sure whether this also applies to some other track as well.

[ErikSchierboom]

@junedev I also included the prerequisites key, which some tracks have included but don't yet have any practices.

[junedev]

@ErikSchierboom We set prerequisites for like 20% of the exercises in JavaScript. I thought this might result in a wrong picture above to include tracks like that. E.g. it looks like we would need more exercises for "template-strings" or "type-conversion" from the table above but there are plenty of those, they have just not been labelled yet.

You can set up the table however you see fit, I just wanted to be sure you are aware that the track is not that far along with the prerequisites/practices assignment issue.

[ErikSchierboom]

@junedev Right. I could exclude JavaScript if you think that makes sense?

[junedev]

Yes, I think that would give a better picture here.

[ErikSchierboom]

I've updated the list

[junedev]

@ErikSchierboom It seems currently the numbers do not reflect distinct exercises, is that intentional? If 3 tracks have some exercise implemented and labelled with the same concept, the table would show "practiced 3" even though we only have 1 exercise for the concept. I noticed this because I wanted to check the 2 exercises for "big-integers" that are mentioned and saw it is the same one.

[ErikSchierboom]

🤔 It was intentional, but it might not be a great idea. The idea was to get an indication how wide-spread a concept was used. I'll work on a distinct count tomorrow.

[ErikSchierboom]

Wow, that's quite the comment :) I totally agree that the above list is only marginally helpful, but that was what I was going for: to just have some idea of what is being practiced. Looking at that list and using my own experience from the tracks I maintain, I'd like to see more exercises about:

• Dates
• Timezones
• Times
• Durations
• Stacks
• Results/errors
• Parallellism
• Concurrency
• Type conversions
• Exception/error-handling
• Numeric overflow
• Inheritance
• Classes
• Constructors

[BethanyG]

Yup. The comment was too much. I've removed it. 😄

[ErikSchierboom]

FYI this is the list of new exercise idea issues