extra gcs and s3

[yokomotod]

Problem

• now we always have to install both s3 and gcs dependencies
• it causes
  • more lock problem, since python's dependency resolving(like Poetry) is fragile
  • unnecessary install time, disk usage, container image size

Goal

• from gokart USER
  • can separate [s3] and [gcs] extras to reduce unnecessary dependencies
    • pip install gokart: not install gcs, s3 related deps
    • pip install gokart[s3]: install with s3 related deps
    • pip install gokart[gcs]: install with gcs related deps
    • pip install gokart[s3][gcs]: install both (results same as with previous behavior)

Sub Goal

• from gokart DEVELOPER
  • can unit tests that
    • core logic works without any extra deps. no unintended access to extra libs
    • each extra logic works with its dependency
    • extra dependencies are installed correctly
  • ~usually, developers can run all unit tests without being conscious of extras~
    • ~extras are installed by default~
    • (update)
      • this goal makes confusion of pyproject.toml, so after discussion, I drop this goal
      • and developer will poetry install --extras=s3 --extras=gcs --with=dev-s3 explicitly
    • all test for extras will be executed

Discussion

• how it's confusing to need gokart[s3] or gokart[gcs] to works
• do we have to MAJOR update because this is breaking change?
  • (update) discussed and maybe YES!

ToDO

test publishing on pypy dev

[yokomotod]

other option:

• separate pypi package?
  • gokart-gcs gokart-s3
    • (update)
      • NO. it will cause combinatorial explosion if we have more extras
      • and pip extra is straight forward for this purpose

[yokomotod]

@hirosassa Hi! How do your team feel, as another well know users? 😃

especially # discussion points on description

[KAZYPinkSaurus]

unnecessary install time, disk usage, container image size

How much does it actually affect disk size and install time?