PyFunceble icon indicating copy to clipboard operation
PyFunceble copied to clipboard
verified publisher icon funilrys

tests with errors

Open Nilsonfsilva opened this issue 2 years ago • 12 comments

Hey ! First congratulations on the project.

I would like to inform you that we are packaging your project to be part of our repositories.

In the construction of the package, some tests presented errors. it would be possible to help us to discover the reason.

Here are the errors:

FAIL: test_is_valid (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid) Tests the method which let us check if the given subject is valid

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 267, in test_is_valid self.assertEqual(expected, actual, subject) AssertionError: True != False : 255.45.65.0/24

====================================================================== FAIL: test_is_valid_range_v4 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_range_v4) Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 142, in test_is_valid_range_v4 self.assertEqual(expected, actual, subject) AssertionError: True != False : 255.45.65.0/24

====================================================================== FAIL: test_is_valid_range_v6 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_range_v6) Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 157, in test_is_valid_range_v6 self.assertEqual(expected, actual, subject) AssertionError: True != False : 2001:db8::/128

====================================================================== FAIL: test_is_valid_v4 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_v4) Tests the method which let us check if the given subject is valid.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 80, in test_is_valid_v4 self.assertEqual(expected, actual, subject) AssertionError: True != False : 255.45.65.0/24

====================================================================== FAIL: test_is_valid_v6 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_v6) Tests the method which let us check if the given subject is valid.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 95, in test_is_valid_v6 self.assertEqual(expected, actual, subject) AssertionError: True != False : 2001:db8:1234::/48

====================================================================== FAIL: test_is_valid (checker.syntax.test_ipv4.TestIPv4SyntaxChecker.test_is_valid) Tests the method which let us check if the given subject is valid.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv4.py", line 80, in test_is_valid self.assertEqual(expected, actual, subject) AssertionError: True != False : 255.45.65.0/24

====================================================================== FAIL: test_is_valid_range (checker.syntax.test_ipv4.TestIPv4SyntaxChecker.test_is_valid_range) Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv4.py", line 110, in test_is_valid_range self.assertEqual(expected, actual, subject) AssertionError: True != False : 255.45.65.0/24

====================================================================== FAIL: test_is_valid (checker.syntax.test_ipv6.TestIPv6SyntaxChecker.test_is_valid) Tests the method which let us check if the given subject is valid.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv6.py", line 80, in test_is_valid self.assertEqual(expected, actual, subject) AssertionError: True != False : 2001:db8:1234::/48

====================================================================== FAIL: test_is_valid_range (checker.syntax.test_ipv6.TestIPv6SyntaxChecker.test_is_valid_range) Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv6.py", line 111, in test_is_valid_range self.assertEqual(expected, actual, subject) AssertionError: True != False : 2001:db8::/128

====================================================================== FAIL: test_get_converted (converter.test_cidr2subject.TestCIDR2Subject.test_get_converted) Tests the method which let us get the converted data.

Traceback (most recent call last): File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/converter/test_cidr2subject.py", line 113, in test_get_converted self.assertTrue(Counter(expected) == Counter(actual)) AssertionError: False is not true

Nilsonfsilva avatar Jan 24 '23 09:01 Nilsonfsilva

Hi @Nilsonfsilva,

thank you! Let me know when the packaging is done so that I can link it to the documentation or the landing page!

I can't reproduce this error on my machine + 2 VMs. May I ask you which version of Python has been used? That will help a lot!

I wish you all the best in this New Year!

funilrys avatar Jan 28 '23 16:01 funilrys

Hi!

I wish you all the best in this New Year! Thank you for the congratulations and I wish you the same.

I ask for a little patience in advance. Because according to the packaging policy, debian cannot use any external repositories. To package pyfunceble it is necessary that all dependencies are already in the distribution's repositories. I suspect these errors are due to missing some dependency: answering your question about the python version is Python 3.9.13 pyfunceble is complaining about this dependency: pyf_test_dataset E ModuleNotFoundError: No module named 'pyf_test_dataset'

  • Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ ERROR tests/checker/reputation/test_base.py ERROR tests/checker/syntax/test_domain.py ERROR tests/checker/syntax/test_ip.py ERROR tests/checker/syntax/test_ipv4.py ERROR tests/checker/syntax/test_ipv6.py ERROR tests/checker/syntax/test_second_lvl_domain.py ERROR tests/checker/syntax/test_subdomain.py ERROR tests/checker/syntax/test_url.py ERROR tests/query/dnss/test_nameserver.py !!!!!!!!!!!!!!!!!!! Interrupted: 9 errors during collection !!!!!!!!!!!!!!!!!!!! ======================== 2 warnings, 9 errors in 3.98s =========================

tests/checker/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_domain.py ERROR tests/checker/availability/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_domain.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_domain_and_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/syntax/test_domain.py ERROR tests/checker/syntax/test_ip.py ERROR tests/checker/syntax/test_ipv4.py ERROR tests/checker/syntax/test_ipv6.py ERROR tests/checker/syntax/test_second_lvl_domain.py ERROR tests/checker/syntax/test_subdomain.py ERROR tests/checker/syntax/test_url.py ERROR tests/checker/utils/test_whois.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/config/test_loader.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/converter/test_internal_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/dataset/test_ipv4_reputation.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/helpers/test_download.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/test_http_status_code.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/dnss/test_nameserver.py ERROR tests/query/dnss/test_resolver.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/netinfo/test_address.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/whois/test_query_tool.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/whois/converter/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers

all these files have errors due to this absence Can you measure which repository this module is in? Thanks!

Nilsonfsilva avatar Jan 28 '23 21:01 Nilsonfsilva

Oh, you discovered a problem in the way I process tests ...

I'm testing with:

 python -m unittest discover -s tests

If I run with pytest I can reproduce an error:

python -m pytests tests
==================================================================== short test summary info ====================================================================
ERROR tests/checker/reputation/test_base.py
ERROR tests/checker/syntax/test_domain.py
ERROR tests/checker/syntax/test_ip.py
ERROR tests/checker/syntax/test_ipv4.py
ERROR tests/checker/syntax/test_ipv6.py
ERROR tests/checker/syntax/test_second_lvl_domain.py
ERROR tests/checker/syntax/test_subdomain.py
ERROR tests/checker/syntax/test_url.py
ERROR tests/query/dnss/test_nameserver.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 9 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================= 9 errors in 0.77s =======================================================================

Let me try to fix those issue ASAP.

Used version: Python 3.10.9

funilrys avatar Jan 28 '23 21:01 funilrys

@Nilsonfsilva can you test against the origin/issue-328 branch ?

I know that you are trying to package the stable version, but if that dev-based branch deliver the certainty that the issue is fixed, I will release it into the dev and later the stable version of PyFunceble.

funilrys avatar Jan 28 '23 22:01 funilrys

Hi, thanks in advance for your attention. It will be a great pleasure to bring your project to Debian.

take it easy, try to fix the bugs, and when you're ready and want to release a new version, we appreciate it. Since the debian robot will fetch the latest tag.

If you want to send me before, the .tar.gz. with corrections to check, feel free. Thank you and I'm at your disposal! a hug!

Nilsonfsilva avatar Jan 28 '23 23:01 Nilsonfsilva

@Nilsonfsilvavocê pode testar na origin/issue-328 ? ramificação

Eu sei que você está tentando empacotar o stableversão, mas se isso devramo baseado em entregar a certeza de que o problema está resolvido, vou liberá-lo para o deve mais tarde o stableversão do PyFunceble.

I believe we have evolved. But there are still other mistakes. I'm simulating in two different environments: Work environment, presents errors in 10 functions: As for that, it's quiet. Because I can escape these functions through patch

=========================== short test summary info ============================ FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid - ... FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_range_v4 FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_range_v6 FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_v4 FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_v6 FAILED tests/checker/syntax/test_ipv4.py::TestIPv4SyntaxChecker::test_is_valid FAILED tests/checker/syntax/test_ipv4.py::TestIPv4SyntaxChecker::test_is_valid_range FAILED tests/checker/syntax/test_ipv6.py::TestIPv6SyntaxChecker::test_is_valid FAILED tests/checker/syntax/test_ipv6.py::TestIPv6SyntaxChecker::test_is_valid_range FAILED tests/converter/test_cidr2subject.py::TestCIDR2Subject::test_get_converted

In the second environment, I simulate as if pyfunceble were inside the debian repositories, being built. That is, isolated without connections to external repositories. Because, for security reasons, deboian does not allow any packages to be communicated with anything external. Such errors occur:

=================== ERRORS =========================================================================================== ________________________________________________________________________ ERROR collecting tests/checker/test_base.py _________________________________________________________________________ tests/checker/test_base.py:55: in from PyFunceble.checker.base import CheckerBase, CheckerStatusBase PyFunceble/init.py:56: in from PyFunceble.checker.availability.domain import DomainAvailabilityChecker PyFunceble/checker/availability/domain.py:55: in import PyFunceble.factory PyFunceble/factory.py:55: in Requester = requester() PyFunceble/query/requests/requester.py:169: in init self.dns_query_tool = DNSQueryTool() PyFunceble/query/dns/query_tool.py:126: in init self.nameservers.guess_and_set_nameservers() PyFunceble/query/dns/nameserver.py:254: in guess_and_set_nameservers self.set_nameservers(dns.resolver.get_default_resolver().nameservers) /usr/lib/python3/dist-packages/dns/resolver.py:1166: in get_default_resolver reset_default_resolver() /usr/lib/python3/dist-packages/dns/resolver.py:1178: in reset_default_resolver default_resolver = Resolver() /usr/lib/python3/dist-packages/dns/resolver.py:758: in init self.read_resolv_conf(filename) /usr/lib/python3/dist-packages/dns/resolver.py:850: in read_resolv_conf raise NoResolverConfiguration('no nameservers') E dns.resolver.NoResolverConfiguration: no nameservers

ERROR tests/checker/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_domain.py ERROR tests/checker/availability/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/availability/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_domain.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_domain_and_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/reputation/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/checker/utils/test_whois.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/config/test_loader.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/converter/test_internal_url.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/dataset/test_ipv4_reputation.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/helpers/test_download.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/test_http_status_code.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/dnss/test_resolver.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/netinfo/test_address.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/whois/test_query_tool.py - dns.resolver.NoResolverConfiguration: no nameservers ERROR tests/query/whois/converter/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers

Nilsonfsilva avatar Jan 30 '23 04:01 Nilsonfsilva

That's an interesting output. I never thought it was possible to have a server without any DNS nameservers.

Let's me write the patch :-)

funilrys avatar Apr 29 '23 16:04 funilrys

One question for you: Are you planing to deploy both package or only pyfunceble ?

I deploy and maintain 2 packages:

  • pyfunceble (master branch)
    • -> This is one is a stable an slow package that only get a new version when pyfunceble-dev is stable enough (running several weeks on several environments with any huge bugs)
  • pyfunceble-dev (devbranch)
    • -> This is one is also stable package but it gets updated quicker. All new features and bug fixes are always available in this package first. This let us get a huge pool of feedback giver before the change are published in pyfunceble. Please also note that this package only gets updated when all tests (unit + integration + system) are passed.

funilrys avatar Apr 29 '23 16:04 funilrys

@Nilsonfsilva can you please test https://github.com/funilrys/PyFunceble/archive/refs/heads/issue-328.zip ?

Or is there a way I can see the automated tests?

I appreciate your patience.

funilrys avatar Apr 29 '23 17:04 funilrys

Reopening as other issues may come.

funilrys avatar May 01 '23 09:05 funilrys

@Nilsonfsilva Can you please test again when you get some time? Thank a lot!

funilrys avatar May 28 '23 10:05 funilrys

Hi @funilrys Firstly, I apologize for my silence regarding pyfunceble's packaging.

The problem was that pyfunceble depends on sqlalchemy 2.0 and in Debian this version took more than a year to migrate to the official repositories. And it was for this reason that pyfunceble packaging was stopped.

Now everything is back to normal, I'm back to packaging and it's already under review with Debian Developer. @mitya57

The big question is, as your project is not something simple, but rather a very well-structured project, we need to ensure that everything in it is in full working order. And testing helps us ensure that.

The Debian Developer, analyzing his project, noticed that the tests use the network to perform imports.

He asks: Would there be any possibility of importing the package without the need for a network?

As you well know, for security reasons, both Debian and Ubuntu do not allow external connections in the build process.

And as our official repositories (unstable/Testing) are always receiving daily updates, we need to ensure that the packages do not have problems when they are rebuilt.

However, feel free to do what is best for your project!

Thank you in advance for your attention and I feel very flattered to maintain your Debian/ubuntu project, if sponsored.

I will be waiting for you!

Strong hug.

.... Nilson Silva

Nilsonfsilva avatar Jul 30 '24 23:07 Nilsonfsilva