support for recent rebar3 version

[tothlac]

I was using a very old rebar3 version (3.13.0). During the recent days I wanted to try out 3.18.0.

On this repository when I run rebar3 eunit katt_blueprint.erl is generated, but it can't generate katt_blueprint.beam. The reason is simple as I see. Earlier you had this commit which changed pre_hooks to post_hooks in rebar.config:

https://github.com/for-GET/katt/commit/2bc3255dce5eff42f726d1ea577037c5cc69a13f

The problem is that with the current version of rebar3 priv/compile-parser is executed after katt is compiled, so it generates katt_blueprint.erl but compile won't run twice, so rebar3 won't generate katt_blueprint.beam into the ebin directory.

I made some modifications on katt for this repository, and now it is called twice (from pre_hooks and from post_hooks).

I see the problem why it can't work with pre_hooks is that neotoma is compiled after the pre_hooks of katt is called so neotoma:file("priv/katt_blueprint.peg", [{output, "src/"}]) will fail since neotoma.beam had not been generated.

===> Fetching unicode_util_compat v0.7.0
Compile katt_blueprint
Could not compile katt_bluepring.peg. Reason: undef
===> Analyzing applications...
...
===> Compiling neotoma
===> Compiling jsx
...
===> Compiling katt
Compile katt_blueprint
===> Analyzing applications

The first

Compile katt_blueprint
Could not compile katt_bluepring.peg. Reason: undef

is printed when compile-parser is called from the pre_hooks and as you can see neotoma:file/2 is undef, the second Compile katt_blueprint is successful but compile has been already called, so katt_blueprint.beam won't be generated.

I've pushed the code here: https://github.com/tothlac/try_katt

[tothlac]

The problem is definitely with the current compile order in rebar3. Now the pre_hooks are called before the dependencies of the project are compiled which is wrong.

Fred Herbert mentioned me this neotoma plugin:

https://github.com/tsloughter/rebar3_neotoma_plugin

I've tried and it actually works, the only problem is that it does not support an src_dir parameter. This is a problem because katt_blueprint.peg is stored in the priv folder, and the plugin wants to compile .peg files stored only in the src directory.

Either katt_blueprint.peg should be moved into ./src or the plugin should be slightly modified. Can you see any other options?

[tothlac]

So I have just tried it on our projects with a simple modification in katt's code. I've moved prv/katt_blueprint.peg => src/katt_blueprint.peg and added

{plugins, [
           rebar3_neotoma_plugin
          ]}.

{provider_hooks, [
                 {pre, [
                        {compile, {neotoma, compile}}
                       ]}
                 ]}.

to rebar.config and commented out

{post_hooks, [{compile, "./priv/compile-parser"}]}.

Actually now there is no need for priv/compile-parser anymore. Should I create a pull request?

[brucify]

@tothlac I tried your https://github.com/tothlac/try_katt with a branch of katt using the rebar3_neotoma_plugin plugin:

               {katt,
                   {git,
                       "[email protected]:brucify/katt.git",
                       {branch,"use-rebar3-neotoma-plugin"}}}

With katt rebar.config:

{plugins, [rebar3_neotoma_plugin]}.
{provider_hooks, [{pre, [{compile, {neotoma, compile}}]}]}.

It seems that katt_blueprint.erl is still not generated:

✗ rebar3 eunit
...
===> Performing EUnit tests...
F
Failures:

  1) try_katt_test:katt_test/0: module 'try_katt_test'
     Failure/Error: {error,undef,
                        [{katt_blueprint,module_info,[],[]},
                         {try_katt_test,katt_test,0,
                             [{file,
                                  "/Users/bruce/git/try_katt/test/try_katt_test.erl"},
                              {line,6}]},
                         {try_katt_test,katt_test,0,[]}]}
     Output:

Finished in 0.063 seconds
1 tests, 1 failures

✗ ls -l _build/test/lib/katt/src
total 240
-rw-r--r--  1 bruce  staff   1140 Feb 16 17:47 katt.app.src
-rw-r--r--  1 bruce  staff  16572 Feb 16 17:47 katt.erl
-rw-r--r--  1 bruce  staff   4006 Feb 16 17:47 katt.hrl
-rw-r--r--  1 bruce  staff   2174 Feb 16 17:47 katt_blueprint_parse.erl
-rw-r--r--  1 bruce  staff  12642 Feb 16 17:47 katt_callbacks.erl
-rw-r--r--  1 bruce  staff   7407 Feb 16 17:47 katt_callbacks_json.erl
-rw-r--r--  1 bruce  staff   6989 Feb 16 17:47 katt_callbacks_wfu.erl
-rw-r--r--  1 bruce  staff   6140 Feb 16 17:47 katt_cli.erl
-rw-r--r--  1 bruce  staff   6766 Feb 16 17:47 katt_har_cli.erl
-rw-r--r--  1 bruce  staff   2341 Feb 16 17:47 katt_request.erl
-rw-r--r--  1 bruce  staff   2126 Feb 16 17:47 katt_response.erl
-rw-r--r--  1 bruce  staff  20030 Feb 16 17:47 katt_util.erl
-rw-r--r--  1 bruce  staff  11000 Feb 16 17:47 katt_validate_type.erl
✗ rebar3 version
rebar 3.18.0 on Erlang/OTP 22 Erts 10.6.1

However if I go into _build/test/lib/katt, change the rebar.config and compile, it correctly generates the katt_blueprint.erl file.

Did you get it to work?

Edit: It seems I forgot to move priv/katt_blueprint.peg to src/katt_blueprint.peg. Everything works after I did that.

===> Performing EUnit tests...
.
Finished in 0.105 seconds
1 tests, 0 failures

✗ ls -l _build/test/lib/katt/src
total 320
...
-rw-r--r--  1 bruce  staff  26348 Feb 16 18:39 katt_blueprint.erl
-rw-r--r--  1 bruce  staff   8525 Feb 16 18:39 katt_blueprint.peg
...
✗ ls -l _build/test/lib/katt/ebin
total 328
...
-rw-r--r--  1 bruce  staff  46032 Feb 16 18:39 katt_blueprint.beam
...

This is probably because rebar3_neotoma_plugin only looks for and outputs .peg files in the src directory:

         SourceDir = filename:join(rebar_app_info:dir(AppInfo), "src"),
         CompileFun = fun(Source, Target, _Config) ->
                              OutDir = filename:dirname(Target),
                              neotoma:file(Source, [{output, OutDir}])
                      end,

         rebar_base_compiler:run(Opts, [], SourceDir, ".peg", SourceDir, ".erl", CompileFun, [{check_last_mod, true}])

[andreineculau]

bump Sorry for the silence. Before I jump into this, I wanted to check if you have reached a solution @brucify @tothlac . Thanks for reporting!

[brucify]

Sorry for a late response. I have a commit here that does what I mentioned above:

• uses rebar3 plugin rebar3_neotoma_plugin
• removes priv/compile-parser
• moves priv/katt_blueprint.peg to src/katt_blueprint.peg

I can create a PR for this.