fix ImportError on local unittests

[bosd]

local unit tests are failling.

steps to reproduce:

1. go to invoice2data/tests
2. $ python test_lib.py

Traceback (most recent call last):
  File "test_lib.py", line 12, in <module>
    from .common import get_sample_files
ModuleNotFoundError: No module named '__main__.common'; '__main__' is not a package

or command python -m unittest test_cli.py

======================================================================
ERROR: test_cli (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_cli
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/bosd/Documents/invoice2data/tests/test_cli.py", line 18, in <module>
    from .common import get_sample_files
ImportError: attempted relative import with no known parent package


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

This PR fixes that..

Result:

python test_lib.py
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Sammy Maystone - For Lines testing', 'invoice_number': 'invoice_number_1', 'date': datetime.datetime(2022, 1, 1, 0, 0), 'line_items': [{'item': 'A', 'desc': 'Parts: 1 x cap_a'}, {'item': 'B', 'desc': 'Parts: 2 x shop supplies'}], 'currency': 'USD', 'desc': 'Invoice from Sammy Maystone - For Lines testing'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087', 'currency': 'INR', 'hotel_details': ' OYO 4189 Resort Nanganallur', 'date_check_in': datetime.datetime(2017, 12, 31, 0, 0), 'date_check_out': datetime.datetime(2018, 1, 1, 0, 0), 'amount_rooms': 1.0, 'booking_id': 'IBZY2087', 'payment_method': 'Cash at Hotel', 'gstin': '06AABCO6063D1ZQ', 'cin': 'U63090DL2012PTC231770', 'desc': 'Invoice from OYO'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/utils/__init__.py:130: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = tz.localize(date_obj)
{'issuer': 'Free', 'amount': 29.99, 'amount_untaxed': 24.99, 'date': datetime.datetime(2015, 7, 2, 0, 0), 'date_due': datetime.datetime(2015, 7, 5, 0, 0), 'invoice_number': '562044387', 'vat': 'FR60421938861', 'currency': 'EUR', 'line_number': 'FO10479674', 'client_id': '10577874', 'desc': 'Invoice from Free'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Amazon Web Services', 'amount': 4.11, 'amount_untaxed': 4.11, 'date': datetime.datetime(2014, 8, 3, 0, 0), 'invoice_number': '42183017', 'partner_name': 'Amazon Web Services, Inc.', 'partner_website': 'aws.amazon.com', 'currency': 'USD', 'lines': [{'description': 'AWS Data Transfer', 'price_unit': '0.01'}, {'description': 'Amazon Elastic Compute Cloud', 'price_unit': '1.87'}, {'description': 'Amazon Glacier', 'price_unit': '2.22'}, {'description': 'Amazon Simple Storage Service', 'price_unit': '0.01'}], 'desc': 'Invoice from Amazon Web Services'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'QualityHosting AG', 'amount': 34.73, 'amount_untaxed': 34.73, 'date': datetime.datetime(2014, 5, 7, 0, 0), 'invoice_number': '30064443', 'vat': 'DE 232 446 240', 'currency': 'EUR', 'lines': [{'pos': '1', 'qty': 1.0, 'desc': 'Small Business StandardExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_strukan\n01.05.14-31.05.14', 'price': 3.89}, {'pos': '2', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_schneider\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '3', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_minar\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '4', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_mayr\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '5', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_jenewein\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '6', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_jauernik\n01.05.14-31.05.14\nQualityHosting AG - Uferweg 40-42 - D-63571 Gelnhausen\niViveLabs Ltd.\n93B Sai Yu Chung\nYuen Long, N.T.\nHong Kong\nPos.            Menge      Beschreibung                                                            Rabatt %     VK-Preis     Zeilenbetrag\nOhne      Ohne MwSt.\nMwSt.', 'price': 5.39}, {'pos': '7', 'qty': 1.0, 'desc': 'Small Business StandardExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_office\n01.05.14-31.05.14\n', 'price': 3.89}], 'desc': 'Invoice from QualityHosting AG'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Flipkart', 'amount': 319.0, 'date': datetime.datetime(2015, 10, 20, 0, 0), 'invoice_number': '#BLR_WFLD20151000982590', 'order_id': 'OD304175096047380001', 'currency': 'INR', 'desc': 'Invoice from Flipkart'}
.No lines found. Start match: None. End match: <re.Match object; span=(536, 544), match='Subtotal'>
Failed to parse field line_items with parser lines
Unable to match all required fields. The required fields are: ['invoice_number', 'date', 'line_items']. Output contains the following fields: ['currency', 'invoice_number', 'issuer', 'date'].
no table body found - start None, end <re.Match object; span=(175, 185), match='Booking ID'>
no table body found - start None, end <re.Match object; span=(255, 266), match='DESCRIPTION'>
no table body found - start None, end <re.Match object; span=(896, 924), match='Oravel Stays Private Limited'>
/home/bosd/.local/lib/python3.7/site-packages/dateparser/utils/__init__.py:130: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = tz.localize(date_obj)
no table body found - start None, end <re.Match object; span=(301, 311), match='Facture n°'>
./home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Sammy Maystone - For Lines testing', 'invoice_number': 'invoice_number_1', 'date': datetime.datetime(2022, 1, 1, 0, 0), 'line_items': [{'item': 'A', 'desc': 'Parts: 1 x cap_a'}, {'item': 'B', 'desc': 'Parts: 2 x shop supplies'}], 'currency': 'USD', 'desc': 'Invoice from Sammy Maystone - For Lines testing'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087', 'currency': 'INR', 'hotel_details': ' OYO 4189 Resort Nanganallur', 'date_check_in': datetime.datetime(2017, 12, 31, 0, 0), 'date_check_out': datetime.datetime(2018, 1, 1, 0, 0), 'amount_rooms': 1.0, 'booking_id': 'IBZY2087', 'payment_method': 'Cash at Hotel', 'gstin': '06AABCO6063D1ZQ', 'cin': 'U63090DL2012PTC231770', 'desc': 'Invoice from OYO'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/utils/__init__.py:130: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = tz.localize(date_obj)
{'issuer': 'Free', 'amount': 29.99, 'amount_untaxed': 24.99, 'date': datetime.datetime(2015, 7, 2, 0, 0), 'date_due': datetime.datetime(2015, 7, 5, 0, 0), 'invoice_number': '562044387', 'vat': 'FR60421938861', 'currency': 'EUR', 'line_number': 'FO10479674', 'client_id': '10577874', 'desc': 'Invoice from Free'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Amazon Web Services', 'amount': 4.11, 'amount_untaxed': 4.11, 'date': datetime.datetime(2014, 8, 3, 0, 0), 'invoice_number': '42183017', 'partner_name': 'Amazon Web Services, Inc.', 'partner_website': 'aws.amazon.com', 'currency': 'USD', 'lines': [{'description': 'AWS Data Transfer', 'price_unit': '0.01'}, {'description': 'Amazon Elastic Compute Cloud', 'price_unit': '1.87'}, {'description': 'Amazon Glacier', 'price_unit': '2.22'}, {'description': 'Amazon Simple Storage Service', 'price_unit': '0.01'}], 'desc': 'Invoice from Amazon Web Services'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'QualityHosting AG', 'amount': 34.73, 'amount_untaxed': 34.73, 'date': datetime.datetime(2014, 5, 7, 0, 0), 'invoice_number': '30064443', 'vat': 'DE 232 446 240', 'currency': 'EUR', 'lines': [{'pos': '1', 'qty': 1.0, 'desc': 'Small Business StandardExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_strukan\n01.05.14-31.05.14', 'price': 3.89}, {'pos': '2', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_schneider\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '3', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_minar\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '4', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_mayr\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '5', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_jenewein\n01.05.14-31.05.14', 'price': 5.39}, {'pos': '6', 'qty': 1.0, 'desc': 'Small Business QualityExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_jauernik\n01.05.14-31.05.14\nQualityHosting AG - Uferweg 40-42 - D-63571 Gelnhausen\niViveLabs Ltd.\n93B Sai Yu Chung\nYuen Long, N.T.\nHong Kong\nPos.            Menge      Beschreibung                                                            Rabatt %     VK-Preis     Zeilenbetrag\nOhne      Ohne MwSt.\nMwSt.', 'price': 5.39}, {'pos': '7', 'qty': 1.0, 'desc': 'Small Business StandardExchange 2010\nGrundgebühr pro Einheit\nDienst: OUDJQ_office\n01.05.14-31.05.14\n', 'price': 3.89}], 'desc': 'Invoice from QualityHosting AG'}
/home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
{'issuer': 'Flipkart', 'amount': 319.0, 'date': datetime.datetime(2015, 10, 20, 0, 0), 'invoice_number': '#BLR_WFLD20151000982590', 'order_id': 'OD304175096047380001', 'currency': 'INR', 'desc': 'Invoice from Flipkart'}
./home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
[{'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087', 'currency': 'INR', 'hotel_details': ' OYO 4189 Resort Nanganallur', 'date_check_in': datetime.datetime(2017, 12, 31, 0, 0), 'date_check_out': datetime.datetime(2018, 1, 1, 0, 0), 'amount_rooms': 1.0, 'booking_id': 'IBZY2087', 'payment_method': 'Cash at Hotel', 'gstin': '06AABCO6063D1ZQ', 'cin': 'U63090DL2012PTC231770', 'desc': 'Invoice from OYO'}]
./home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
[{'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087', 'currency': 'INR', 'hotel_details': ' OYO 4189 Resort Nanganallur', 'date_check_in': datetime.datetime(2017, 12, 31, 0, 0), 'date_check_out': datetime.datetime(2018, 1, 1, 0, 0), 'amount_rooms': 1.0, 'booking_id': 'IBZY2087', 'payment_method': 'Cash at Hotel', 'gstin': '06AABCO6063D1ZQ', 'cin': 'U63090DL2012PTC231770', 'desc': 'Invoice from OYO'}]
./home/bosd/.local/lib/python3.7/site-packages/dateparser/date_parser.py:35: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  date_obj = stz.localize(date_obj)
[{'issuer': 'OYO', 'amount': 1939.0, 'date': datetime.datetime(2017, 12, 31, 0, 0), 'invoice_number': 'IBZY2087', 'currency': 'INR', 'hotel_details': ' OYO 4189 Resort Nanganallur', 'date_check_in': datetime.datetime(2017, 12, 31, 0, 0), 'date_check_out': datetime.datetime(2018, 1, 1, 0, 0), 'amount_rooms': 1.0, 'booking_id': 'IBZY2087', 'payment_method': 'Cash at Hotel', 'gstin': '06AABCO6063D1ZQ', 'cin': 'U63090DL2012PTC231770', 'desc': 'Invoice from OYO'}]
.Detected 23 diacritics
/usr/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 139528 is still running
  ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_lib.py:74: ResourceWarning: unclosed file <_io.BufferedReader name=3>
  if tesseract.to_text(file) is None:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Detected 9 diacritics
/usr/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 139538 is still running
  ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_lib.py:74: ResourceWarning: unclosed file <_io.BufferedReader name=3>
  if tesseract.to_text(file) is None:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Detected 8 diacritics
/usr/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 139552 is still running
  ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_lib.py:74: ResourceWarning: unclosed file <_io.BufferedReader name=3>
  if tesseract.to_text(file) is None:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Detected 57 diacritics
/usr/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 139564 is still running
  ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_lib.py:74: ResourceWarning: unclosed file <_io.BufferedReader name=3>
  if tesseract.to_text(file) is None:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
.
----------------------------------------------------------------------

Ran 7 tests in 43.237s

OK

[bosd]

This might be incompatible with CI, please check. In that case we should use:

try:
    from .common import get_sample_files
except ImportError:
    from common import get_sample_files

[bosd]

@m3nu in this pr test still say:

First-time contributors need a maintainer to approve running workflows. Learn more. 3 expected checks

Can you check?

[m3nu]

Yes, now I can see the button. 😮‍💨

[bosd]

@m3nu Did you push the button? There is no status...

[m3nu]

There was no button before, but it's there now.

[m3nu]

Instead of try-except, you could also use an absolute import all the time?

[bosd]

Instead of try-except, you could also use an absolute import all the time?

Could be. But I thought it was incopmpatible with CI. (There seems to be something going on with the paths on the CI build, see https://github.com/invoice-x/invoice2data/pull/382)

[rmilecki]

Instead of try-except, you could also use an absolute import all the time?

Just tried it in the https://github.com/invoice-x/invoice2data/pull/386 . It won't work.

[rmilecki]

I just got a bit more familiar with Python concepts using:

• Relative imports in Python 3
• Relative imports for the billionth time

If I got it right, it seems that running in-project scripts directly isn't well supported. Some consider it an anti-pattern.

Some workarounds are possible but it seems that just using an -m solution to load module is considered the cleanest solution.

So it seems to me that:

1. We should not introduce any workarounds (including one from this pull request)
2. To run tests locally python -m tests.test_cli should be used

[rmilecki]

Confirmed in https://github.com/invoice-x/invoice2data/issues/381 that -m works.