Separate tests from examples

[szhorvat]

Right now the examples/simple directory contains both tests and examples linked from the documentation. Many of these "examples" are not actually useful as sample code. These should be identified and moves to examples/tests.

Additional;y, files in examples/tests should be moved to tests/unit.

[ntamas]

examples/tests is no more, all the relevant files are now in tests/unit

[ntamas]

I went through the files in examples/simple and moved those that were not explicitly referenced from the docs to tests/unit. I don't know when I will have time to continue this so I'll just put it back to the common stack.

[szhorvat]

Related: #1585.

[ntamas]

Phew, I started thinking about this, but I had to realize that this is going to be pretty subjective. I propose sorting the .c files in examples/simple by size in descending order, and then to start processing the list from the top. The reasoning is that files that are large are more likely to be tests than real examples. Ideally, we should limit ourselves to including small examples in the documentation only, preferably not more than, say, 15-20 lines of "real" code (not counting comments, empty lines and such).

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[Jeroenvh99]

I would like to work on this, does this task still need to be worked on?

[ntamas]

Thanks for your interest in this issue! Yes, this is still something that needs to be done; @GroteGnoom was working on tidying up the examples in PR #2175, so please take a look at that PR as well, but other than that you are free to work on it. If you do, please open a draft PR so we can follow the progress and others can see that you are already working on it.

The ultimate goal is to ensure that tests/unit contains unit tests only (i.e. code that is meant to be executed to test that the library is working correctly) and examples/simple contains illustrative examples only (i.e. code that is presented in a way that it can be embedded in the user manual). Right now it's a mixed bag; historically, the unit tests were also in examples/simple and many of the examples that we still have there are more like unit tests than illustrative examples.

[Jeroenvh99]

Hi @ntamas, I'm going to work on this issue, could you assign me to this issue then?

[szhorvat]

Don't forget that you need to work together and keep in sync with @GroteGnoom who has already done part of the job in #2175. It might make sense to merge #2175 first, or otherwise be careful to avoid conflict.

[Jeroenvh99]

I think it's easiest then that I open a PR with @GroteGnoom's repo, to ensure we stay in sync. @GroteGnoom do you think that would be the best way to go about this, if not what would you suggest?

[GroteGnoom]

This is a list of examples that are more than 100 lines after #2175:

(smaller_examples)> find . -type f -name "*.c" | xargs wc -l | sort | awk '$1 > 100'
     104 ./igraph_fisher_yates_shuffle.c
     106 ./flow.c
     107 ./igraph_get_eids.c
     109 ./igraph_barabasi_game2.c
     109 ./igraph_complementer.c
     110 ./igraph_community_optimal_modularity.c
     112 ./igraph_community_multilevel.c
     112 ./igraph_mincut.c
     114 ./igraph_get_shortest_paths.c
     117 ./dqueue.c
     117 ./igraph_compose.c
     124 ./igraph_intersection.c
     125 ./igraph_feedback_arc_set_ip.c
     126 ./igraph_subisomorphic_lad.c
     131 ./igraph_get_laplacian_sparse.c
     133 ./cattributes.c
     138 ./igraph_difference.c
     145 ./igraph_cliques.c
     145 ./igraph_union.c
     151 ./igraph_lapack_dgesv.c
     152 ./igraph_degree.c
     155 ./igraph_maximal_cliques.c
     157 ./igraph_all_st_mincuts.c
     172 ./igraph_deterministic_optimal_imitation.c
     172 ./igraph_minimum_size_separators.c
     173 ./igraph_sparsemat.c
     179 ./igraph_random_sample.c
     181 ./igraph_bipartite_projection.c
     182 ./igraph_stochastic_imitation.c
     189 ./igraph_community_fastgreedy.c
     194 ./igraph_community_edge_betweenness.c
     209 ./igraph_sparsemat8.c
     213 ./igraph_roulette_wheel_imitation.c
     311 ./igraph_sparsemat4.c
     312 ./igraph_sparsemat3.c

So those still need to be done. I don't remember why I skipped the sparsemat examples, but you should probably not start with those 🙂

100 lines is of course a pretty random number. The cutoff should probably be lower.

[Jeroenvh99]

@GroteGnoom I'll make a PR to your repo then and try to sort out the remaining files

[GroteGnoom]

No, please work on master, you should be able to work independently from #2175. #2175 is already way too big, and I shouldn't have done that. If you work on a PR on master it's all clearer. Just try one file, and finish the whole PR before going to the next one.

[Jeroenvh99]

Ok, I can do that

[GroteGnoom]

We've merged #2175.

@Jeroenvh99 have you started working on this? Do you need any help, for example in picking a file to work on?

[Jeroenvh99]

@GroteGnoom I was a bit busy lately with moving to my new room, so I haven't really had a lot of time to look at it, but I have more time this week, so if you could suggest a file to begin with I'll look at it tomorrow. Thanks very much for your help!

[szhorvat]

You can grep the example sources for the word "test", which will immediately show a number of examples which are really just tests. These should be moved to tests. Ideally, a true educational example would be created in their place.

Using ripgrap, which I previously recommended:

$ rg -w --iglob '*.c' test
simple/igraph_reciprocity.c
42:    /* Small test graph */

simple/igraph_cocitation.c
27:    /* Create a small test graph. */

simple/graphml.c
28:    const char *infilename  = "test.graphml";
42:    FILE *infile = fopen("test.graphml", "r");
50:     * We temporarily disable the default error handler so we can test for this condition. */

simple/igraph_community_multilevel.c
64:    /* Unweighted test graph from the paper of Blondel et al */

simple/igraph_bipartite_projection.c
93:    /* More sophisticated test                             */
130:    /* Multiplicity test                                   */

simple/igraph_random_sample.c
28:/* test parameters */
36:/* Get a few random samples and test their properties. */

simple/igraph_lcf.c
77:    // Regression test for bug #996
80:        printf("Failure: regression test for #996\n");

simple/igraph_deterministic_optimal_imitation.c
25:/* test parameters structure */
65:        printf("Isolated vertex test failed.\n");
70:            printf("Isolated vertex test failed.\n");

simple/igraph_community_fastgreedy.c
152:    /* Regression test -- graph with two vertices and two edges */
158:    /* Regression test -- asking for optimal membership vector but not
168:    /* Regression test -- asking for optimal membership vector but not
177:    /* Regression test -- asking for optimal membership vector but not

simple/igraph_roulette_wheel_imitation.c
27:/* test parameters structure */
52:    strategy_test_t *test;
71:    /* test parameters */
91:        test = all_checks[i];
93:        ret = igraph_roulette_wheel_imitation(test->graph, test->vertex,
94:                                              test->islocal, test->quantities,
95:                                              &stratcopy, test->mode);
96:        if (ret != test->retval) {
104:        known = test->known_strats;
106:            if (VECTOR(*known)[k] == VECTOR(stratcopy)[test->vertex]) {
112:            printf("Roulette wheel imitation failed for vertex %" IGRAPH_PRId ".\n", test->vertex);

simple/igraph_barabasi_game.c
76:    /* outpref, we cannot really test this quantitatively,

simple/igraph_stochastic_imitation.c
25:/* test parameters structure */
66:        printf("Isolated vertex test failed.\n");
71:            printf("Isolated vertex test failed.\n");
98:    strategy_test_t *test;
130:        test = all_checks[i];
132:        ret = igraph_stochastic_imitation(test->graph, test->vertex, test->algo,
133:                                          test->quantities, &stratcopy,
134:                                          test->mode);
136:            printf("Stochastic imitation failed for vertex %" IGRAPH_PRId ".\n", test->vertex);
142:        knownstrats = test->known_strats;
144:            if (VECTOR(*knownstrats)[k] == VECTOR(stratcopy)[test->vertex]) {
150:            printf("Stochastic imitation failed for vertex %" IGRAPH_PRId ".\n", test->vertex);

[Jeroenvh99]

@GroteGnoom @szhorvat I've opened a PR #2210 could you have a look at it and tell me if I'm on the right track? Thank you!

[ntamas]

Responded in #2210