Idea: custom REPL mode to launch tests

[giordano]

I think a custom REPL mode (with ReplMaker.jl?) could simplify the workflow for running the tests.

[rfourquet]

That might be a good idea! Do you have specific ideas on what this might look like?

[giordano]

Admittedly I haven't use the package, yet, so I don't have first-hand experience with the workflow, but by reading the README I was thinking to

• a command to load the tests for a given package (looking for it in the active environment)
• and another one to run the tests, maybe filtered

Does it make sense?

[rfourquet]

Sounds useful yes! Using TerminalMenus might also be an option for something like that (to select the packages to load tests from, and to select the tests to run from a given (set of) package(s).

[giordano]

The other day I started playing with CondaPkg.jl and noticed that it hooks into internals of Pkg's own REPL mode to extend it, instead of creating yet another REPL mode. The code is at https://github.com/cjdoris/CondaPkg.jl/blob/cb1605b7d85460e013b29c4552358a155987f6a6/src/PkgREPL.jl.

I started experimenting with something similar for ReTest (this is a very proof of concept):

diff --git a/Project.toml b/Project.toml
index faabfde..f17ca82 100644
--- a/Project.toml
+++ b/Project.toml
@@ -6,6 +6,8 @@ version = "0.3.2"
 [deps]
 Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
 InlineTest = "bd334432-b1e7-49c7-a2dc-dd9149e4ebd6"
+Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
+Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
 Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 Sockets = "6462fe0b-24de-5631-8697-dd941f90decc"
diff --git a/src/PkgREPL.jl b/src/PkgREPL.jl
new file mode 100644
index 0000000..21ea84b
--- /dev/null
+++ b/src/PkgREPL.jl
@@ -0,0 +1,45 @@
+# Inspired by https://github.com/cjdoris/CondaPkg.jl/blob/cb1605b7d85460e013b29c4552358a155987f6a6/src/PkgREPL.jl
+
+module PkgREPL
+
+import ..ReTest
+import Pkg
+import Markdown
+
+## run
+
+function run(args)
+    ReTest.retest(args...)
+end
+
+const run_help = Markdown.parse("""
+```
+retest run module [filter]
+```
+
+Run the tests for the given module, optionally applying filtering them.
+""")
+
+const run_retest = Pkg.REPLMode.CommandSpec(
+    name = "run",
+    api = run,
+    should_splat = false,
+    help = run_help,
+    arg_count = 1 => 2,
+    description = "run the tests",
+)
+
+### all specs
+
+const SPECS = Dict(
+    "run" => run_retest,
+)
+
+function __init__()
+    # add the commands to the repl
+    Pkg.REPLMode.SPECS["retest"] = SPECS
+    # update the help with the new commands
+    copy!(Pkg.REPLMode.help.content, Pkg.REPLMode.gen_help().content)
+end
+
+end # module PkgREPL
diff --git a/src/ReTest.jl b/src/ReTest.jl
index 386bcb8..0c88580 100644
--- a/src/ReTest.jl
+++ b/src/ReTest.jl
@@ -1749,4 +1749,6 @@ function runtests(tests::String="")
     Pkg.test("ReTest", test_args=Vector{String}(split(tests)))
 end
 
+include("PkgREPL.jl")
+
 end # module ReTest

but I believe the tricky part is to get process_args evaluate the module in the right scope, which I have no idea how to do. At the moment I get:

julia> # MyPackage.jl file

       module MyPackage
       using InlineTest

       greet() = "Hello World!"

       @testset "greet" begin
           @test greet() == "Hello World!"
       end

       end # module
Main.MyPackage

julia> using .MyPackage, ReTest

(tmp) pkg> retest run MyPackage
No matching tests for module Main.MyPackage.

which is not very useful.

[giordano]

The other annoying thing is that the arguments of

(tmp) pkg> retest run MyPackage r"greet"

are parsed as ["run", "MyPackage", "r", "greet"], so I don't think it's possible to pass in regexes, which is also not very useful.