Move fourfront from pytest 2.9 to 3.10 (ALT)

[netsettler]

Substantive changes and bug fixes:

• In src/encoded/authentication.py

  • From CGAP: Be explicit about encryption algorithms in JWT handling.
  • From CGAP: Remove special-case treatment of GET and HEAD methods.

• pyproject.toml

  • Bump patch version.
  • Update pytest to version 3.10 and related libraries in some ways.
  • Update awscli, boto3, botocore, netaddr, boto3-stubs.

• In src/encoded/schema_formats.py and src/encoded/server_defaults.py:

  • Use new ACCESSION_PREFIX variable to minimize number of places code diverges.
  • Add a TEST_PREFIX variable for symmetry. Compute the letter pairs needed for accession ids by looking at the schema files.

• src/encoded/loadxl.py

  • From CGAP: Port the post_only= argument for load_all and load_all_gen.

• src/encoded/renderers.py

  • From CGAP: Port mime type selection logic in should_transform.
  • From CGAP: Port best_mime_type.

• src/encoded/tests/conftest.py

  • Change htmltestapp and anonhtmltestapp to pass HTTP_ACCEPT, which is more like what browsers would do, and which is needed for changes ported from CGAP.
  • Light PEP8.
  • Remove pytest_plugins setting, which is deprecated. This info is now in pytest.ini.

• pytest.ini

  • Change to have markers in line with cgap-portal.
  • Specify testpaths rather than a --pyargs argument being added.
  • Use -p to handle plugins, in lieu of setting pytest_plugins.

• src/encoded/tests/test_indexing.py

  • Change meta = MetaData(...); meta.reflect() to instead be meta = MetaData(..., reflect=True).

• Makefile:

  • From CGAP: Port scripts/kibana-start and scripts/psql-start scripts to support targets kibana-start-test and psql-test, respectively.
  • Pin poetry 1.1.4

• conftest.py

  • Set tempfile.tempdir to '/tmp'.

• src/encoded/commands/purge_item_type.py

  • From CGAP: Change the way transaction commits work to be more incremental.
  • From CGAP: Use set(...) to remove duplicates.

• Migrate the es_testapps to src/encoded/tests/conftest.py from src/encoded/tests/workbook.py, and update the following tests to use that test_app:

  • test_validation_errors.py
  • test_purge_item_type.py
  • test_static_page.py
  • test_indexing.py
  • test_aggregation.py
  • test_batch_download.py

• Adjust Makefile target test to split indexing and non-indexing cases the way CGAP does.

Simple changes and small bug fixes (mostly found by better warnings):

• Mark test_index_data_workbook as skip in test_search.py.

• Use str in place of basestring in src/encoded/loadxl.py and src/encoded/tests/test_loadxl.py.

• Port new file src/encoded/tests/test_renderers.py from CGAP.

• src/encoded/tests/test_static_page.py

  • Changes to fixture scopes since session-scoped fixtures don't work well here.

• src/encoded/types/page.py

  • Suppress deprecation warnings we aren't going to fix right now.

• src/encoded/tests/test_types_biosample.py

  • Unmark biosample_relation as a fixture. It's just a function.

• Add support for rewriting "~1.2.3" in setup_eb.py. Add test cases in test_setup_eb.py.

• src/encoded/commands/spreadsheet_to_json.py

  • Remove an internal import of argparse.

• src/encoded/xlreader.py

  • Add a missing module prefix (zipfile.ZipFile).
  • Light refactoring.

• Relock poetry.lock.

• Rename src/encoded/utils.py back to src/encoded/util.py for alignment with cgap-portal, and with history. Bad idea to have renamed it in the first place.

  • Port various utilities that might not be used, just in case they're needed for other ports. We can clean this up later.

  • Update callers to import from util:

    • src/encoded/types/file.py
    • src/encoded/tests/test_search.py
    • src/encoded/tests/test_types_init_collections.py
    • src/encoded/tests/test_types_protocol.py
    • src/encoded/tests/test_util.py (was test_utils.py, but renamed as well)
    • src/encoded/tests/test_validation_errors.py
    • src/encoded/tests/test_aggregation.py
    • src/encoded/tests/test_batch_download.py

• Move the definition or ORDER to `conftest_setti

• Get rid of unused config fixture on src/encoded/tests/conftest.py.

• Get rid of unused moto fixture on src/encoded/tests/conftest.py.

Cosmetic changes:

• Change .encrypt to .hash in

  • src/encoded/tests/test_access_key.py
  • src/encoded/types/access_key.py
  • src/encoded/tests/test_edw_hash.py

• Use dcicutils.misc_utils.TestApp instead of webtest.TestApp in:

  • `src/encoded/commands/add_date_created.py
  • src/encoded/commands/import_data.py
  • src/encoded/commands/load_access_keys.py

• .ebextensions/13_run_npm.config

  • Add a newline so this compares clean with CGAP.

• .gitignore

  • From CGAP: Add various ignorable files.

• src/encoded/commands/configure_kibana_index.py

  • Backport a TODO comment about {'use_ssl': True}.

• src/encoded/dev_servers.py

  • Add PEP8 hint for PyCharm.

• Also misc PEP8 in various files, including:

  • src/encoded/root.py