ForkManager now appears to be required when attempting to build and install cloc

[Earnestly]

• linux 4.15.4
• perl 5.26.1

I'm no longer able to build cloc since https://github.com/AlDanial/cloc/commit/fcc846eb91edc4583343152064d563259965e6dd

Unfortunately ForkManager is not available easily for me but perhaps this outcome wasn't intentional.

cd Unix
make DESTDIR="$PWD"/t install

# perl-test - Check syntax
perl -cw cloc
Can't locate Parallel/ForkManager.pm in @INC (you may need to install the Parallel::ForkManager module) (@INC contains: /home/earnest/local/lib/perl5 /usr/lib/perl5/5.26/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/5.26/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/5.26/core_perl /usr/share/perl5/core_perl) at cloc line 54.
BEGIN failed--compilation aborted at cloc line 54.
make: *** [Makefile;89: test-perl] Error 2

Is there a way to work around the requirement or is ForkManager now mandatory?

Thanks

[AlDanial]

The contents of the Unix directory are meant only for people who create packages of cloc for inclusion with package managers like yum, apt, npm, et cetera. Those package maintainers would use their package's bundling system to define dependencies such as Parallel::ForkManager which should be provided by other packages on their system. Installing cloc through one of these package managers should also pull down these dependencies.

Regular users--and I include myself in this category--should just use the cloc source file in the top level directory, in other words, cloc-1.76.pl in the current release (https://github.com/AlDanial/cloc/releases). It works everywhere there's a vanilla Perl installation, regardless of OS. Specifically, it doesn't use Parallel::ForkManager--but consequently does not support parallel processing.

[Earnestly]

I am creating a package. Parallel::ForkManager is not easily available as it's not part of the perl distribution and it's not commonly packaged either.

This is an issue because cloc used to build but now it doesn't. So a regression for all intents and purposes. I'm asking for clarification if the requirement on Parallel::ForkManager is now mandatory as it wasn't before now.

If it is the case that Parallel::ForkManager is required from now on, I can make plans to package it as well.

[AlDanial]

First off, thank you for being a package creator for cloc. (I'm curious which package manager you're working with?) Your efforts make it much easier for others to acquire and use cloc.

Parallel::ForkManager shouldn't be a hard requirement as it just brings the bonus of optional higher performance to cloc. It shouldn't be a roadblock that prevents its use. At this point it is a judgement call between the extra effort of you making an additional package for Parallel::ForkManager (and possibly more since I'm not sure if that itself needs additional dependencies) and unlocking the power of --processes, or commenting out the relevant code lines (I'll post instructions for that when I'm back at my development machine) but then preventing any of your package users from the new capability.

My preference is that you make a package for Parallel::ForkManager too, but that's easy for me to say--more work for you, less for me.

I'll post instructions for by-passing Parallel::ForkManager in a bit.

[AlDanial]

To remove the dependency on Parallel::ForkManager in Unix/cloc, change lines 53 and 54 as follows: BEFORE:

   53 my $HAVE_Parallel_ForkManager = 1;
   54 use Parallel::ForkManager;

AFTER:

   53 my $HAVE_Parallel_ForkManager = 0;
   54 # use Parallel::ForkManager;

[Earnestly]

I package for pacman mostly. A lot of my packages are software currently built from latest commit such as glibc and cloc. This is mostly done in an effort to find potential issues, especially integration issues as that is something very difficult to account for even when attempted synthetically. This bug report is essentially an example of those efforts.

Now I don't mind packaging forkmanager as it seems fairly straight forward and it would better provide a more complete example of the software, cloc in this case. The workaround suggested seems reasonable to me as well.

Thanks for addressing the issues o/

[AlDanial]

If there's more to be done on my end, please re-open.

[Earnestly]

While cloc appears to try dynamically including parallel such that it's not a hard requirement, the test-perl target in the Makefile runs perl -cw $(PL_SCRIPT) which fails.

Unfortunately the install targets have prerequists, one of them being test-perl. Ideally the install targets should not depend on anything other than "sub-install" targets such as install: install-man. The test targets should be standalone.

[Earnestly]

PS: The tests also fail because it looks for cloc_submodule_test in the same directory instead of where it's registered (tests/inputs/issues/131/cloc_submodule_test).

[AlDanial]

Parallel::ForkManager is a hard requirement for the cloc found in the Unix/ subdirectory, but not for the cloc in the parent directory. The perl -cw test should pass for the latter. The Perl installation referenced below doesn't have Parallel::ForkManager:

> /usr/local/perl/5.28.0/bin/perl -cw cloc Unix/cloc
cloc syntax OK

> /usr/local/perl/5.28.0/bin/perl -cw Unix/cloc                           
Can't locate Parallel/ForkManager.pm in @INC (you may need to install the Parallel::ForkManager module) (@INC contains: /usr/local/perl/5.28.0/lib/site_perl/5.28.0/x86_64-linux /usr/local/perl/5.28.0/lib/site_perl/5.28.0 /usr/local/perl/5.28.0/lib/5.28.0/x86_64-linux /usr/local/perl/5.28.0/lib/5.28.0) at Unix/cloc line 871.
BEGIN failed--compilation aborted at Unix/cloc line 871.

I'm open to suggested edits on Makefile (better yet, a PR).

Regarding the submodule tests, do both T1 and T2 tests fail for you?

[Earnestly]

There is no Makefile in the root directory which is why I was using the Unix/Makefile part of the pre-package checking phase which itself carries test targets.

Currently the following works (and doesn't work):

cd Unix

% ./t/00_C.t
# ok 1 - Agda created output
# ok 2 - Agda results match
# ...
# ok 326 - Xtend results match
# 1..326

% ./t/01_opts.t
# ...
# not ok 22 - git submodule handling (github issue \#131 T1) results match
# #   Failed test 'git submodule handling (github issue #131 T1) results match'
# #   at ./t/01_opts.t line 347.
# #     Structures begin differing at:
# #          $got->{Markdown} = HASH(0x55cbad921560)
# #     $expected->{Markdown} = Does not exist
# 1..92
# # Looks like you failed 1 test of 92.

% ./t/02_git.t
# -------------------------------------------------------------------------------
# Directory 'cloc_submodule_test' is not found; git tests skipped.
# To enable the tests, create the directory with
#     git clone https://github.com/AlDanial/cloc_submodule_test.git
# not ok 1 - git tests
# #   Failed test 'git tests'
# #   at ./t/02_git.t line 76.
# -------------------------------------------------------------------------------
# 1..1
# # Looks like you failed 1 test of 1.

If I clone that repo as suggested, into the Unix directory, it gets further:

...
not ok 10 - diff f15bf042b f647093e8b results match
#   Failed test 'diff f15bf042b f647093e8b results match'
#   at ./t/02_git.t line 100.
#     Structures begin differing at:
#          $got->{same}{blank} = '0'
#     $expected->{same}{blank} = '53'
...
# Looks like you failed 1 test of 16.

However this is not where this submodule is registered. In fact the submodule appears to be a total red herring as attempting to use it as described in .gitmodules is completely broken, including the registered commit.

The make test-pod target works without issue. As a result I'm currently only running:

prepare() {
    cd cloc
    # Attempt to make more progress on ./t/02_git.t; ignore the .gitmodules as
    # it's a red herring.
    mv -f "$srcdir"/cloc_submodule_test Unix/cloc_submodule_test

    # Needed to install without `test-perl` breaking on ForkManager::Parallel
    # not being present.
    sed -i 's/^install-bin: test-perl$/install-bin:/' Unix/Makefile
}

check() {
    cd cloc/Unix
    make test-pod
    ./t/00_C.t
    # ./t/01_opts.t
    # ./t/02_git.t
}

[AlDanial]

Fair enough; I have just a cursory knowledge of git submodules so the tests were perhaps beyond the limit of my understanding. I'm puzzled by the comment about .gitmodules since that file is empty.

Current circumstances limit the time I can spend on cloc. While I'm not going to abandon this project by any means, fixing up the test suite will be delayed.

[Earnestly]

Empty?

% git clone https://github.com/AlDanial/cloc.git
Cloning into 'cloc'...
remote: Enumerating objects: 86, done.
remote: Counting objects: 100% (86/86), done.
remote: Compressing objects: 100% (62/62), done.
remote: Total 3214 (delta 45), reused 57 (delta 23), pack-reused 3128
Receiving objects: 100% (3214/3214), 2.68 MiB | 3.51 MiB/s, done.
Resolving deltas: 100% (1787/1787), done.

% cat cloc/.gitmodules
[submodule "tests/inputs/cloc_submodule_test"]
        path = tests/inputs/issues/131/cloc_submodule_test
        url = https://github.com/AlDanial/cloc_submodule_test.git

Also I don't mind, the main issue is that install-bin shouldn't depend on test-perl (or really depend on anything outside the scope of "installing")

[LeamHall]

Installed via cpan App::cloc, and I can't run because of missing Parallel/ForkManager.pm. Esited the cloc code to comment out the "use" and it works.