[Win32] testprogs/TESTrun

[gvanem]

Trying testprogs/TESTrun on Windows, like this (in a home-grown GNU-makefile):

#
# this is from Git, i.e. MSys2
#
PERL := /bin/perl

run_filtertest: bin/filtertest.exe $(OBJ_DIR)/config.h
	export FILTERTEST_BIN=$(realpath bin/filtertest.exe) \
	export TESTRUN_JOBS=20   \
	export CONFIG_H=$(realpath $(OBJ_DIR)/config.h) &&   \
	$(PERL) testprogs/TESTrun
	@echo

Does this make sense?

The output of make run_filtertests is:

export FILTERTEST_BIN=F:/MinGW32/src/inet/libpcap/bin/filtertest.exe \
export TESTRUN_JOBS=20                               \
export CONFIG_H=F:/MinGW32/src/inet/libpcap/objects/config.h &&   \
/bin/perl testprogs/TESTrun
INFO: Using a test timeout of 1.
INFO: This Perl supports threads, using 20 tester thread(s).
TTTTTTTTTTTTTTTTTSSSSSSSSSSSSSSSSSSSSTTTTTTTTTTTTTTTTTTTTTTTT   61 / 1473 (  4%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTSSTTSSSSSTTTSSSTTTTTTTTTTTTTSTTT  122 / 1473 (  8%)
TTTTTTTTTTTTTTTTSSTTSSTTTTTTSSSSSSSSSSSSSSSSSSSSTTTTTTTTTTTTT  183 / 1473 ( 12%)
TTTTTTTTTTTTTTTTTTSTTSSTTSTSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS  244 / 1473 ( 16%)
SSSSSSSSTSSSSTTTTSSSSSSSSSSSSSSSSSSTTSSSSSSTTTTSSSSSSTTSSSSSS  305 / 1473 ( 20%)
SSSSSSSSSSSSSSSSSSSSSSSTTTTTTSSSSSSSSSTTTTTTTTTTTTSSSSSSTTTTS  366 / 1473 ( 24%)
SSSSSTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  427 / 1473 ( 28%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  488 / 1473 ( 33%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTSTTT  549 / 1473 ( 37%)
TTTTTTTTTTTTTTTSSSSSSSSSSSSSSSSSSSSTTTTTTTTTTTTTTTTTTTTTTTTTT  610 / 1473 ( 41%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  671 / 1473 ( 45%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  732 / 1473 ( 49%)
TTTTTTTTTTSSSTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  793 / 1473 ( 53%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  854 / 1473 ( 57%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  915 / 1473 ( 62%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT  976 / 1473 ( 66%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1037 / 1473 ( 70%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1098 / 1473 ( 74%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1159 / 1473 ( 78%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1220 / 1473 ( 82%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1281 / 1473 ( 86%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1342 / 1473 ( 91%)
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 1403 / 1473 ( 95%)
TTSTSSSSSSSTSTSTSTSSTTTTTTTTTTTTTTTTTTTTTTTTTTTSTTTTTSTTT.TTT 1464 / 1473 ( 99%)
TTTTT.TTT                                                     1473 / 1473 (100%)

Skipped tests:
    accept_arp_gateway_NAME_en10mb_opt      : needs /etc/ethers  -- Windows do have this too.
 ...
   accept_ether_proto_ip6_unopt            : INET6!=1   -- but I do have '#define INET6'
 ...
Failed tests:
    accept_ah_opt                           : filtertest timeout
    accept_ah_opt (alias 0)                 : filtertest timeout
 ...
------------------------------------------------
 224 tests skipped
1247 tests failed
   2 tests passed

What does all these letters mean? AFAICS, TESTrun accepts no -h or --help. Would be nice.

But more importantly, why only 2 tests passed, but sometimes 5 (and which are those?).

No .log-files left over AFAICS to give me a clue.

[gvanem]

Update: guessing a T means timeout, then adding export FILTERTEST_TIMEOUT=2, gave a better result:

 -- lots of diffs skipped ---
 224 tests skipped
1097 tests failed
 152 tests passed

But far too many failed.

[infrastation]

Thank you for trying the script, I do not use Windows so never tested it in this context. I am working on a couple improvements, let me see if adding a proper help screen could be done in the same go.

[infrastation]

INET6 has to be defined in config.h, not on the command line.

needs /etc/ethers means the file must be present at the correct location and the script has to be modified to use that correct location, and the file must have certain lines (easy to see near the top of the file). Does Windows provide ether_hostton()? If yes, where does it read the entries from?

[gvanem]

INET6 has to be defined in config.h, not on the command line.

I have it as #define INET6 1. Does not help.

[gvanem]

Does Windows provide ether_hostton()?

No, but there is pcap_ether_hostton() that Windows should use.

If yes, where does it read the entries from?

It should be PCAP_ETHERS_FILE, but that fails on Windows. Something like #define PCAP_ETHERS_FILE pcap_etc_subpath("ethers") that would return %SystemRoot%/System32/drivers/etc/ethers.

Similar for /etc/networks and /etc/services.

[infrastation]

In the current master branch the script now implements --help and a regexp that should match the #define above and a digest of the the characters used in the ASCII map. The simplest way to add an ethers file into the loop would be to modify PCAP_ETHERS_FILE locally, but if you have a better solution, please prepare a pull request.

[gvanem]

a regexp that should match the #define above

My config.h gets generated with 2 leading spaces. So why not:

  my $re_define_uint = qr/^[[:blank:]]+#define[[:blank:]]+([0-9_A-Z]+)[[:blank:]]+([0-9]+)$/;
  my $re_define_str = qr/^[[:blank:]]+#define[[:blank:]]+([0-9_A-Z]+)[[:blank:]]+"(.+)"$/;

And is there a log-file with a summary generated? And TESTrun seems to require a Cygwin64 perl.exe here. Strawberry perl.exe will not work.

[gvanem]

Seems all the TESTrun script failures were caused by me having -DBDEBUG. Removing that, and adding export FILTERTEST_TIMEOUT=0 (what is it for?) I got much further:

 224 tests skipped
   2 tests failed
1248 tests passed

which AFAICS does these:

bin\filtertest.exe RAW ip6 host fe80:0:0:0:0:0:0:0
bin\filtertest.exe RAW ip6 net fe80:0:0:0:0:0:0:0/64

Why does these fail with INET6?

It would be nice to have a --keep (and --clean?) options to see what this script expects.

[guyharris]

Seems all the TESTrun script failures were caused by me having -DBDEBUG.

Yes, that produces a lot of filter compiler debugging output, which will screw up comparisons of compiler output.

Why does these fail with INET6?

It would be nice to have a --keep (and --clean?) options to see what this script expects.

Does the output for filter-compiler failures show a diff between what was expected and what was produced?

If not, it should be made to do so, just as the tcpdump dissection tests do.

[gvanem]

Does the output for filter-compiler failures show a diff between what was expected and what was produced?

Not sure. I get this:

    reject_ip6_host_toolong                 : INET6!=1 
    reject_ip6_net_masklen                  : INET6!=1
    reject_ip6_net_prefix                   : INET6!=1
    reject_outbound_not_supported_linux     : is not linux
    reject_src_gateway                      : needs /etc/ethers

Failed tests:
    accept_ether_proto_aarp_opt_0           : diff error
    reject_ip6_host_disabled                : no filtertest error
(000) ldb      [0]
(001) and      #0xf0
(002) jeq      #0x60            jt 3    jf 11
(003) ld       [8]
(004) jeq      #0xfe800000      jt 5    jf 11
(005) ld       [12]
(006) jeq      #0x0             jt 7    jf 11
(007) ld       [16]
(008) jeq      #0x0             jt 9    jf 11
(009) ld       [20]
(010) jeq      #0x0             jt 22   jf 11
(011) ldb      [0]
(012) and      #0xf0
(013) jeq      #0x60            jt 14   jf 23
(014) ld       [24]
(015) jeq      #0xfe800000      jt 16   jf 23
(016) ld       [28]
(017) jeq      #0x0             jt 18   jf 23
(018) ld       [32]
(019) jeq      #0x0             jt 20   jf 23
(020) ld       [36]
(021) jeq      #0x0             jt 22   jf 23
(022) ret      #262144
(023) ret      #0
    reject_ip6_net_disabled                 : no filtertest error
(000) ldb      [0]
(001) and      #0xf0
(002) jeq      #0x60            jt 3    jf 13
(003) ld       [8]
(004) jeq      #0xfe800000      jt 5    jf 13
(005) ld       [12]
(006) jeq      #0x0             jt 7    jf 13
(007) ld       [16]
(008) and      #0x0
(009) jeq      #0x0             jt 10   jf 13
(010) ld       [20]
(011) and      #0x0
(012) jeq      #0x0             jt 26   jf 13
(013) ldb      [0]
(014) and      #0xf0
(015) jeq      #0x60            jt 16   jf 27
(016) ld       [24]
(017) jeq      #0xfe800000      jt 18   jf 27
(018) ld       [28]
(019) jeq      #0x0             jt 20   jf 27
(020) ld       [32]
(021) and      #0x0
(022) jeq      #0x0             jt 23   jf 27
(023) ld       [36]
(024) and      #0x0
(025) jeq      #0x0             jt 26   jf 27
(026) ret      #262144
(027) ret      #0

------------------------------------------------
 224 tests skipped
   2 tests failed
1248 tests passed

So this hellish Perl-script does not match any #define lines in my objects/config.h. Some issue with DOS/Windows EOL?

[infrastation]

Thank you for testing. The purpose of the timeout is to detect performance regressions. I understand config.h contains

  #define INET6   1

rather than

#define INET6   1

Then the latest revision should be able to parse that as well. Also the latest revision will refuse to run if BDEBUG is defined in config.h. For every failed test the script does output the diff.

Please test this version and post the results.

[infrastation]

For clarity, "no filtertest error" followed by some text means that the test expects a specific invalid combination of parameters in a specific environment to make filtertest exit with a specific error code and without any output on standard output, but filtertest did not return any error at all and produced some output, which is a failure to fail.

[gvanem]

Then the latest revision should be able to parse that as well.

The latest commits still shows 8 failures and some INET6!=1. Can we please have a -d option to dump the $config? 'm a noob when it comes to Perl.

My run-filtertest.bat:

@echo off
setlocal
on break quit
set CONFIG_H=f:/MinGW32/src/inet/libpcap/objects/config.h
set FILTERTEST_BIN=f:/MinGW32/src/inet/libpcap/bin/filtertest.exe
set TESTRUN_JOBS=20
set FILTERTEST_TIMEOUT=2
cd %~dp0\testprogs
perl.exe TESTrun %*

and the results: run-filtertest-result.txt

[gvanem]

and some INET6!=1.

It's a DOS vs. Unix EOL issue. If I do:

dos2unix objects\config.h
run-filtertest.bat --one reject_gateway_INET6

the test passes. Yikes!

[infrastation]

Thank you for testing, let me see what can be done quickly.

[infrastation]

Almost there.

[infrastation]

This failed test from you earlier comment looks odd:

Failed tests:
    accept_atm_vci_le_unopt_0               : filtertest error
(000) ldb      [0]
(001) jeq      #0x8c            jt 2    jf 3
(002) ret      #262144
(003) ret      #0

The filter program is from accept_arcnet_src_host_unopt_0 and accept_arcnet_src_host_unopt_1, which are 39-40 tests before that. The error from filtertest looks like a failure to open the file for writing, but in this case with 20 tester threads this would have manifested 20 tests earlier at accept_arp_gateway_name_ip_over_fc_opt_0.

Other than that, there is a new revision of the script is in the master branch, it changes the means to use a custom Perl binary (set the PERL environment variable before running make check) , and the CMake leg will try to auto-detect Perl interpreter location if not set. Also the script will refuse to run if BDEBUG is set in config.h, will correctly skip protochain tests if libpcap is built without the feature and on Windows will not attempt any tests that require ethers.

The script now supports --passed to print the list of passed tests and --config to print what it managed to parse from config.h. To that end, DOS newlines in config.h should not make any difference now.

Let me know if you see any other bugs.

[gvanem]

Seems OK now:

 200 tests skipped
   0 tests failed
1275 tests passed

But I'd wish /etc/hosts and /etc/ethers could be supported on Windows too. Here's my idea for that (and some more):

--- a/pcap/namedb.h 2025-02-16 16:50:41
+++ b/pcap/namedb.h 2025-02-18 11:46:49
@@ -56,6 +56,12 @@
        u_char addr[6];
        char name[122];
 };
+
+#ifdef _WIN32
+  PCAP_API const char *pcap_etc_subpath (const char *file);
+  #define PCAP_ETHERS_FILE pcap_etc_subpath("ethers")
+#endif
+
 #ifndef PCAP_ETHERS_FILE
   #ifdef __HAIKU__
     #define PCAP_ETHERS_FILE "/boot/system/settings/network/ethers"
@@ -99,6 +105,28 @@
 PCAP_AVAILABLE_0_9
 PCAP_API int   pcap_nametollc(const char *);

+PCAP_API struct servent *pcap_getservent (void);
+PCAP_API void            pcap_setservent (int f);
+PCAP_API void            pcap_endservent (void);
+
+PCAP_API struct netent  *pcap_getnetent (void);
+PCAP_API void            pcap_setnetent (int f);
+PCAP_API void            pcap_endnetent (void);
+
+PCAP_API struct netent  *pcap_getnetbyname (const char *name);
+
+#if !defined(LIBPCAP_NO_EXTRAS)
+  #define getservent()       pcap_getservent()
+  #define setservent(f)      pcap_setservent(f)
+  #define endservent()       pcap_endservent()
+
+  #define getnetent()        pcap_getnetent()
+  #define setnetent(f)       pcap_setnetent(f)
+  #define endnetent()        pcap_endnetent()
+
+  #define getnetbyname(name) pcap_getnetbyname (name)
+#endif
+
 /*
  * If a protocol is unknown, PROTO_UNDEF is returned.
  * Also, pcap_nametoport() returns the protocol along with the port number.

The missing stuff for /etc/ethers on Windows is already in the #ifndef HAVE_ETHER_HOSTTON section in nametoaddr.c. The other missing stuff could be in a missing/win_ethers.c or like: pcap-namedb.c.txt

[infrastation]

Thank you for preparing these changes, I see your point, but API changes such as these would need to be a separate pull request, could you open one please?

Regarding the test script, could you describe the steps to get a working test setup for libpcap on Windows? I understand there is a practicable way to have diff, so the workaround with fc on Windows in tcpdump tests could be replaced with some documentation.

[infrastation]

Is this true that if you add an entry to the hosts file on Windows, filtertest RAW 'host HOSTNAME' will successfully compile an expression for both IPv4 and IPv6 addresses, both lower case and upper case hostname?

[infrastation]

AppVeyor build includes both testprogs and Perl, so after this round of improvements make check may be ready to run in AppVeyor.

[gvanem]

Is this true that if you add an entry to the hosts file on Windows, ...

Added this to c:\Windows\System32\drivers\etc\hosts:

10.20.30.40               eth-ipv4-ipv6.host123.libpcap.test
fe80::1020:3040:5060:7080 eth-ipv4-ipv6.host123.libpcap.test

and ran:

F:\MinGW32\src\inet\libpcap> bin\filtertest.exe raw host eth-ipv4-ipv6.host123.libpcap.test
(000) ldb      [0]
(001) and      #0xf0
(002) jeq      #0x40            jt 3    jf 8
(003) ld       [12]
(004) jeq      #0xa141e28       jt 7    jf 5
(005) ld       [16]
(006) jeq      #0xa141e28       jt 7    jf 8
(007) ret      #262144
(008) ret      #0

and:

F:\MinGW32\src\inet\libpcap> bin\filtertest.exe raw host ETH-IPV4-IPV6.HOST123.LIBPCAP.TEST
(000) ldb      [0]
(001) and      #0xf0
(002) jeq      #0x40            jt 3    jf 8
(003) ld       [12]
(004) jeq      #0xa141e28       jt 7    jf 5
(005) ld       [16]
(006) jeq      #0xa141e28       jt 7    jf 8
(007) ret      #262144
(008) ret      #0

So, yes.

[gvanem]

... could you open one please?

I'm clumsy wrt to git push etc.

Regarding the test script, could you describe the steps to get a working test setup for libpcap on Windows?

I have installed Cygwin64 from https://cygwin.org/setup-x86_64.exe, ran the setup and added x:\Cygwin64 to the PATH. Piece of cake really.

[infrastation]

Thank you. The above filter program is IPv4-only. Apparently, Windows uses a slightly different format for the file, and link-local IPv6 address must use a zone ID, which would be an unnecessary complication for the tests. Before I change the test address and start updating the tests, could you retry using 2a09::1020:3040:5060:7080 as the address in the hosts file and see if the filter program is any different?

[gvanem]

could you retry using 2a09::1020:3040:5060:7080

You mean, replace fe80::1020:3040:5060:7080 eth-ipv4-ipv6.host123.libpcap.test with 2a09::1020:3040:5060:7080 eth-ipv4-ipv6.host123.libpcap.test

Seems to work:

F:\MinGW32\src\inet\libpcap> bin\filtertest.exe raw host ETH-IPV4-IPV6.HOST123.LIBPCAP.TEST
(000) ldb      [0]
(001) and      #0xf0
(002) jeq      #0x60            jt 3    jf 19
(003) ld       [8]
(004) jeq      #0x2a090000      jt 5    jf 11
(005) ld       [12]
(006) jeq      #0x0             jt 7    jf 11
(007) ld       [16]
(008) jeq      #0x10203040      jt 9    jf 11
(009) ld       [20]
(010) jeq      #0x50607080      jt 26   jf 11
(011) ld       [24]
(012) jeq      #0x2a090000      jt 13   jf 27
(013) ld       [28]
(014) jeq      #0x0             jt 15   jf 27
(015) ld       [32]
(016) jeq      #0x10203040      jt 17   jf 27
(017) ld       [36]
(018) jeq      #0x50607080      jt 26   jf 27
(019) ldb      [0]
(020) and      #0xf0
(021) jeq      #0x40            jt 22   jf 27
(022) ld       [12]
(023) jeq      #0xa141e28       jt 26   jf 24
(024) ld       [16]
(025) jeq      #0xa141e28       jt 26   jf 27
(026) ret      #262144
(027) ret      #0

but why so many more op-codes?

I also added 2001:464f:f9eb:1:e18b:8588:cec8:5f89 HP-laptop for my other laptop and did ping -6 HP-LAPTOP which works fine.

[infrastation]

Because now the filter program tests for both IPv4 and IPv6, which is the correct behaviour of host HOSTNAME when the hostname resolves to an IPv4 and an IPv6 address. Let me change the IPv6 address in the tests.

[infrastation]

Please get the latest revision, update the hosts file to use the new entries from the script and confirm whether the script works and runs the tests that depend on the hosts file.

[gvanem]

I did a git pull just now and added:

10.20.30.40                          noeth-ipv4-noipv6.host123.libpcap.test
10.20.30.40                          noeth-ipv4-ipv6.host123.libpcap.test
10.20.30.40                          eth-ipv4-noipv6.host123.libpcap.test
10.20.30.40                          eth-ipv4-ipv6.host123.libpcap.test
fd00:a1b2:c3d4::1020:3040:5060:7080  noeth-noipv4-ipv6.host123.libpcap.test
fd00:a1b2:c3d4::1020:3040:5060:7080  noeth-ipv4-ipv6.host123.libpcap.test
fd00:a1b2:c3d4::1020:3040:5060:7080  eth-noipv4-ipv6.host123.libpcap.test
fd00:a1b2:c3d4::1020:3040:5060:7080  eth-ipv4-ipv6.host123.libpcap.test

to my c:\Windows\System32\drivers\etc\hosts.

But still 200 tests skipped since I get a lot of configure the hosts file.

The perl first used reports itself as $^O == "msys"; so replacing MSWin32 with msys, it still do not work. When switching to a Cygwin Perl, bin/filtertest.exe complains about: can't open /cygdrive/c/temp/libpcap_TESTrun_JX39lbu3/job000-filter.txt: No such file or directory

[infrastation]

Please copy the lines from the Perl script exactly (with one space between the address and the name) and retry.

[gvanem]

Please copy the lines from the Perl script exactly

No difference; still 200 tests skipped. Even with MSWin32 -> msys.