Organize the files in /john/src

[claudioandre-br]

Well, thinking about this matter, I realized that a simple patch can solve the problem:

• I moved *nt* formats and tested the patch;
• it seems appropriate to me;
• I don't think it introduces new bugs;
• It would be very useful to organize the source tree;
• we can create a folder 'includes' and move non-john_proper header files to it;
• we can create modules like (e.g.): microsoft, sha1, sha2, coins, ...

Weak points:

• Makefile.legacy;
• is it portable as it should be?

What do you guys think? I need to sleep on this but I can't see any bad side effects.

diff --git a/src/configure.ac b/src/configure.ac
index 88d9e902b..4b05e2a6c 100644
--- a/src/configure.ac
+++ b/src/configure.ac
@@ -87,6 +87,9 @@ AC_ARG_ENABLE([fuzz], [AC_HELP_STRING([--enable-fuzz], [Fuzzing prepare(), valid
 AC_ARG_ENABLE([libfuzzer], [AC_HELP_STRING([--enable-libfuzzer], [Fuzzing using libFuzzer])], [libfuzzer=$enableval], [libfuzzer=no])
 
 dnl ===================== Actual tests start here ========================
+dnl Add the modules (folders inside src) and add the headers search path (includes) folder
+CPPFLAGS="-I$(pwd) $CPPFLAGS"
+MODULES=(" " "microsoft/")
 
 dnl Assert that we don't have a stray GNUmakefile left in the tree :-(
 if test -f GNU?akefile; then
@@ -1092,7 +1095,7 @@ else
   AC_MSG_NOTICE([creating *_plug.c rules])
 fi
 if test "`echo *_plug.c`" != "*_plug.c"; then
-   AC_SUBST([PLUGFORMATS_OBJS],[`echo *_plug.c | sed 's/opencl[[A-Za-z0-9_\-]]*\.c //g;s/\.c/.o/g'`])
+   AC_SUBST([PLUGFORMATS_OBJS],[`for i in "${MODULES[[@]]}"; do echo -n $i*_plug.c " "; done | sed 's/opencl[[A-Za-z0-9_\-]]*\.c //g;s/\.c/.o/g'`])
    AC_SUBST([OPENCL_PLUGFORMATS_OBJS],[`echo opencl*_plug.c | sed 's/\.c/.o/g'`])
    if test "x$plug_deps" = xyes -a "x$PERL" != x ; then
       AC_MSG_NOTICE([creating Makefile dependencies])
@@ -1137,7 +1140,9 @@ if test "x$simd" = xno ; then
 fi
 
 if test "`echo *_fmt_plug.c`" != "*_fmt_plug.c"; then
-   $CPP -P -DAC_BUILT $CFLAGS_EX2 $CPPFLAGS $CPU_BEST_FLAGS $CFLAGS $HAVE_OPENCL $CFLAGS_EXTRA $OPENMP_CFLAGS $OPENSSL_CFLAGS -DFMT_EXTERNS_H *_fmt_plug.c | LC_ALL=C $GREP "extern struct fmt_main" | LC_ALL=C $SORT -f > fmt_externs.h
+   for i in "${MODULES[[@]]}"; do
+      $CPP -P -DAC_BUILT $CFLAGS_EX2 $CPPFLAGS $CPU_BEST_FLAGS $CFLAGS $HAVE_OPENCL $CFLAGS_EXTRA $OPENMP_CFLAGS $OPENSSL_CFLAGS -DFMT_EXTERNS_H $i*_fmt_plug.c | LC_ALL=C $GREP "extern struct fmt_main" | LC_ALL=C $SORT -f >> fmt_externs.h
+   done
 fi
 dnl if the file was not created, then bail.
 if test -f fmt_externs.h -a ! -s fmt_externs.h; then
@@ -1151,7 +1156,9 @@ fi
 AC_MSG_NOTICE([creating ./fmt_registers.h])
 rm -f fmt_registers.h
 if test "`echo *_fmt_plug.c`" != "*_fmt_plug.c"; then
-   $CPP -P -DAC_BUILT $CFLAGS_EX2 $CPPFLAGS $CPU_BEST_FLAGS $CFLAGS $CFLAGS_EXTRA $OPENMP_CFLAGS $OPENSSL_CFLAGS -DFMT_REGISTERS_H *_fmt_plug.c | LC_ALL=C $GREP "john_register_one" | LC_ALL=C $SORT -f > fmt_registers.h
+   for i in "${MODULES[[@]]}"; do
+      $CPP -P -DAC_BUILT $CFLAGS_EX2 $CPPFLAGS $CPU_BEST_FLAGS $CFLAGS $CFLAGS_EXTRA $OPENMP_CFLAGS $OPENSSL_CFLAGS -DFMT_REGISTERS_H $i*_fmt_plug.c | LC_ALL=C $GREP "john_register_one" | LC_ALL=C $SORT -f >> fmt_registers.h
+   done
 fi
 dnl if the file was not created, then bail.
 if test -f fmt_registers.h -a ! -s fmt_registers.h; then
-- 
2.39.2

[magnumripper]

Sounds good to me. I think the legacy Makefile can be fixed accordingly without much trouble.

So what directory structure do we want? I'm not sure we'd want to separate OpenCL formats from CPU ones. Or maybe we do. Or maybe we'd just create a plugins/ directory and move all plugins to it... that wouldn't do much except unclobber the first level of src (giving an impression of cleanliness, lol) , leaving just the john proper files plus some other stuff like jumbo modes and "jumbo core" files.

[claudioandre-br]

I like the idea of giving some importance to a few things. Good quality formats [1], cpu x gpu:

Simple:

src
│
└─── plugins (all formats maintained properly and, probably, importants. E.g., NT)
└─── opencl (because we can)
└─── others (using solar's words: "quality requirements are low")
└─── ...

Some other ideas are: microsoft, coins, office. Github stats show bitcoins, office, ssh as very important to people, but not exactly 'src' stuff.

Since the patch is harmless without "modules", I'll test if it produces the same binary and propose a PR.

[1] plugins folder formats should be eligible for static code analysis, clean compilation, and so on.