tools icon indicating copy to clipboard operation
tools copied to clipboard

Deprecate gitpod, update devcontainer.json for GitHub codespaces

Open kenibrewer opened this issue 10 months ago • 9 comments

Given that gitpod classic is going away beginning of March and the new gitpod also supports the devcontainer.json specification, we should probably start a project to migrate fully over to devcontainer.json

We should also migrate any links and references we use to "Open in Gitpod" over to "Open in Codespaces" as new gitpod will have no free tier at all while Codespaces will be maintaining its free tier allowance.

Here's a list of suggested tasks:

  • [x] Rename nf_core/gitpod/gitpod.Dockerfile to nf_core/devcontainer/devcontainer.Dockerfile
  • [x] Switch base image in devcontainer.Dockerfile from gitpod base to codespaces base (or maybe Seqera containers base)
  • [x] Delete nf_core/pipeline-template/.gitpod.yml
  • [x] Change the docker image that we publish via CI from nfcore/gitpod to nfcore/devcontainer`
  • [x] Remove "remoteUser": "gitpod", from devcontainer.json.

kenibrewer avatar Feb 24 '25 16:02 kenibrewer

I guess you don't mean deprecate, but update the devcontainer.json.

I agree with the list of suggested tasks.

Another thing that I've been wanting to do is really trim down the container if possible. This would definitely be a good time to change base too. I was wondering if having a Pixi toml to install packages on container spin up would be useful instead of installing with Conda in the image.

We would probably also need a blog post too describing the change from Gitpod Classic to using Devcontainers and it working with Gitpod Flex if desired.

mahesh-panchal avatar Feb 25 '25 08:02 mahesh-panchal

Use of the word "deprecate" also threw me here, I'll update the title.

Since @kenibrewer wrote this issue, we've done quite a lot of experimentation and development with devcontainers over at training.nextflow.io - I'm hopeful that this will be very transferrable to nf-core.

ewels avatar Apr 04 '25 22:04 ewels

Ooops. Thanks for fixing the issue title. Agreed that I like the approach we took for the github.com/nextfow-io/training repo.

kenibrewer avatar Apr 07 '25 16:04 kenibrewer

I'm getting to work on this. (Draft PR) There are a couple things we should clarify / I need clarified:

  1. What should be the new base image?
  2. Do we want to avoid installing Conda inside our container? (To decrease size)
  3. Do we still need 2 separate Dockerfiles? Should we try to merge them? (1 located in project root, 2 located now in nf_core/devcontainer/ )
  4. Why was apptainer needed in the gitpod containers and can it be dropped?

JulianFlesch avatar May 14 '25 13:05 JulianFlesch

The container image had conda, docker, and apptainer, so we could test with all 3. It was very useful when one of those modes had issues in a module test.

One question I had was if we could perhaps use Pixi in the image. The idea would be building the image would generate the lock files, and then when spinning up a container, pixi would install the packages from the lock file. Then the environment wouldn't be in the image, but it might take a little extra overhead to have a ready container. (Oops. I just saw I mentioned the Pixi thing already - on mobile)

I also just learned singularity and apptainer are on conda-forge.

mahesh-panchal avatar May 14 '25 13:05 mahesh-panchal

1. What should be the new base image?

I think it can still be gitpod

2. Do we want to avoid installing Conda inside our container? (To decrease size)

Nope, we need conda in the container for testing.

3. Do we still need 2 separate Dockerfiles? Should we try to merge them? ([1](https://github.com/JulianFlesch/nf-core-tools/blob/issues/3466-gitpod/Dockerfile?rgh-link-date=2025-05-14T13%3A22%3A24.000Z) located in project root, [2](https://github.com/JulianFlesch/nf-core-tools/blob/issues/3466-gitpod/nf_core/devcontainer/devcontainer.Dockerfile?rgh-link-date=2025-05-14T13%3A22%3A24.000Z) located now in `nf_core/devcontainer/` )

these are two different dockerfiles. One is for nf-core/tools and gets tagged and pushed to dockerhub with every release. The second one is devcontainer specific, so different purpose.

4. Why was apptainer needed in the gitpod containers and can it be dropped?

Same as with conda.

mashehu avatar May 15 '25 07:05 mashehu

Ok thanks @mashehu and @mahesh-panchal for clarifying šŸ‘ I think I misunderstood the purpose of how we want to use the devcontainer. So we need Docker-in-Docker as well to test the automatically build nf-core docker container (e.g. nfcore/tools:latest)?

One more thing: Is the testing @mahesh-panchal mentionsĀ done manually? I cannot find a workflow where the gitpod container I was previously used.

JulianFlesch avatar May 15 '25 08:05 JulianFlesch

yep, it's for people manually testing things while developing.

mashehu avatar May 15 '25 08:05 mashehu

Check out how Chris is using Gitpod here: https://youtu.be/kBoC6QBU-M0?si=eHbTnFFzaeHZLnka

mahesh-panchal avatar May 19 '25 11:05 mahesh-panchal

Closing as it was solved with Issues/3466 Move away from Gitpod #3569. Feel free to reopen is anything is still missing.

mirpedrol avatar Oct 02 '25 10:10 mirpedrol