conda: self-host packages

[casperdcl]

Problems

• upstream channel changes (#1866 <- #1856)
• new intel channel sometimes timing out
• running out of space at ccpi
• long installation command (lots of long -c channel_name_or_URL)

Proposal

I've mirrored all ccpi packages as well as intel ipp-* packages for all platforms (linux-64,win-64,noarch,...) at https://tomography.stfc.ac.uk/conda.

This means installation can be as simple as:

conda install -c https://tomography.stfc.ac.uk/conda cil

• if not already in the user's default channel list, add -c conda-forge
• for optional deps, add e.g. astra-toolbox=*=cuda* tigre ccpi-regulariser tomophantom ipywidgets

/CC @samtygier-stfc

• [x] #1914
• [ ] update docs
  • -c https://tomography.stfc.ac.uk/conda in lieu of -c https://software.repos.intel.com/python/conda -c ccpi
  • devs docs
    • anaconda -c ccpi (on master->dev, on tag->main)
    • conda -c https://tomography.stfc.ac.uk/conda (on tag)

[paskino]

How do we upload the packages to our mirror?

[casperdcl]

I currently manually run roughly:

ssh tomography.stfc.ac.uk 'cd conda && conda-mirror --upstream-channel {ccpi,intel} && conda index'

but we can easily:

1. make our CI upload packages to tomography.stfc.ac.uk
2. add a cron job to tomography.stfc.ac.uk

Option (1) is probably best for long-term sustainability, but option (2) is a very quick n easy stop-gap.

[gfardell]

If we replace our current instructions then we need someway to specify dev/main builds. We don't want users using the dev build unless they are doing it for a reason. Conda currently deferentiates with the channels ccp/label/dev and cil/label/main which I guess is an alias of ccpi

[casperdcl]

leaving open until we update the docs (-c https://tomography.stfc.ac.uk/conda in lieu of -c https://software.repos.intel.com/python/conda -c ccpi)

[casperdcl]

Note from #1914:

• upload to anaconda -c ccpi (on master->dev, on tag->main)
• upload to conda -c https://tomography.stfc.ac.uk/conda (on tag)