EPIC: Refactor pip package

[MalteEbner]

Current state:

Currently, we offer 3 levels of SSL parts:

• low-level: the separate building blocks such as a projection head, nn memory bank, backbone creation https://github.com/lightly-ai/lightly/blob/8dde4d68c0598758bfd05c05f225bf754d9f1ed5/lightly/models/modules/heads.py#L12
• mid-level: the model (backbone + heads + forward functions) https://github.com/lightly-ai/lightly/blob/8dde4d68c0598758bfd05c05f225bf754d9f1ed5/lightly/models/moco.py#L13
• high-level: the model, criterion, training loop put together as pytorch lightning module https://github.com/lightly-ai/lightly/blob/8dde4d68c0598758bfd05c05f225bf754d9f1ed5/docs/source/getting_started/benchmarks/cifar10_benchmark.py#L144

Target state:

We remove the mid-level representations (delete that code). We provide more examples on the high-level: One example pytorch lightning module for every implemented paper

Steps towards it

Always merge into the development branch: https://github.com/lightly-ai/lightly/tree/master-refactor-SSL-models

First step:

• [x] #494
• [x] https://github.com/lightly-ai/lightly/issues/492
• [x] #498
• [x] https://github.com/lightly-ai/lightly/issues/505
• [x] #508
• [x] #512

Second step:

• [x] #515
• [x] Remove mid- to high-level code from CLI
• [x] https://github.com/lightly-ai/lightly/issues/380

Third step:

• [ ] delete mid-level models in lightly/lightly/models

[MalteEbner]

What is still unclear for me is which versions will belong to which steps. We should have clear plans when what will deprecate. In particular, we need published python versions where deprecations warning are raised, but the old models still work.

E.g:

• versions with 1.1.xxxx: Old models work without deprecations warnings. This is the state before this issue.
• versions with 1.2.xxxx: New models are working. Old models are still there and working, but raise deprecation warnings. This equals step 1 of this EPIC. We should use the branch master-refactor-SSL-models for it.
• versions with 1.3.xxxx The old models are removed. This equals step 2 of the epic. We should use a new branch for it.

[philippmwirth]

Requirements for 1.2.0, goal: mid-level models are removed

• Documentation and tutorials are up-to-date (no mid-level models are used)
  • one issue per tutorial
  • one issue for rest of docs
• Ensure CLI tool works (doesn't use mid-level models)
• Remove mid-level models
• Refactor loss function (make symnegcosine non-symmetrical)

[guarin]

The cli still uses SelfSupervisedEmbedding objects. We should decide whether we want to continue using it or also change to low level.