Textual create

[ewels]

First stab at using Textual to create a TUI (text user interface) for creating nf-core pipelines. The idea is that this could replace the Questionary interactive prompts, for a nicer user experience with more space for in-depth documentation.

Command line flags would remain in place, but if some but not all are used then an error would be thrown. The idea being that non-interactive usage is still possible via the CLI, but that all interactive usage should be via the TUI.

Not much done yet. I got as far as the third screen. Main aim was to prototype how this could look and feel, plus playing with using Pydantic to hold the config generated by the wizard and apply input validations in a scalable manner.

To try this out:

pip install -e .  # Updated requirements.txt file in PR
nf-core pipelines create

To run in dev mode (CSS changes hot-reload):

pip install -r requirements-dev.txt   # one time
textual run --dev -c  nf-core pipelines create

Whilst in the app, d switches dark mode on and off and q quits. Ctrl+c also quits.

Shout if any of the code doesn't make sense! Feedback welcome 🙏🏻

Demo:

https://github.com/nf-core/tools/assets/465550/b0326f0a-b3c9-4446-a4b8-db37e4580e7c

[ewels]

Proposed as an alternative to the UI improvements started in https://github.com/nf-core/tools/pull/2398

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 82.54649% with 122 lines in your changes are missing coverage. Please review.

Project coverage is 76.11%. Comparing base (1ef694d) to head (9d6838c). Report is 23 commits behind head on dev.

Files	Patch %	Lines
nf_core/pipelines/create/githubrepo.py	44.02%	75 Missing :warning:
nf_core/pipelines/create/create.py	83.80%	17 Missing :warning:
nf_core/pipelines/create/utils.py	91.59%	10 Missing :warning:
nf_core/pipelines/create/custompipeline.py	76.00%	6 Missing :warning:
nf_core/__main__.py	89.36%	5 Missing :warning:
nf_core/pipelines/create/finaldetails.py	92.72%	4 Missing :warning:
nf_core/pipelines/create/__init__.py	94.23%	3 Missing :warning:
nf_core/pipelines/create/basicdetails.py	98.18%	1 Missing :warning:
nf_core/pipelines/create/nfcorepipeline.py	95.45%	1 Missing :warning:

Additional details and impacted files

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mirpedrol]

From @ewels during Slack discussion:

• [x] Pre-populate author name from git config --global user.name if possible
• [x] Add options at the end to "finish" the git config
  • Ask if a remote repo already exists
  • Offer to create one on github.com via the API, with appropriate fields
  • If we have either of the two above, configure the remote origin
  • If we can configure the remote, do the git push --all
• [ ] Include more boilerplate docs about developing with the community, #new-pipelines etc
• [x] Form fields for other CLI flags, such as target output directly. Pre-populate with defaults.

[ewels]

Suggestions:

• Move "do you want to create a github repo" to a separate page, before entering details
• Logging should show before we start creating a pipeline, otherwise it looks like it hangs when you press "create"

[ewels]

Suggestion: I wonder if many people will create an empty GitHub repo before running nf-core pipelines create (this is probably what I would do). Then we have to rely on people reading the help text printed by the command, which is difficult.

We could add functionality to fetch a list of the user's GH repos (maybe filtering for repos that have zero commits?) and then offer to push to one of them via a dropdown.. Might not be that difficult to do, and could help streamline that final step a little more.

[mirpedrol]

We could add functionality to fetch a list of the user's GH repos (maybe filtering for repos that have zero commits?) and then offer to push to one of them via a dropdown.. Might not be that difficult to do, and could help streamline that final step a little more.

We are already taking this into account. If a repository exists without commits, we use it. If it has commits, we throw an error. I think the dropdown is not needed, as the repository must have the proper pipeline name.

[ewels]

ok @mirpedrol - I had a go through and made some design tweaks and changes to text here and there. Nothing major though, and generally everything worked beautifully and I love it 🎉

The final thing I tried to test was actually creating a GitHub repo. This sadly didn't quite work for me. As soon as I clicked create it took me to the screen showing the log from pipeline creation. I could see some stuff briefly flashing up "behind" it (something about a GitHub username) but then nothing. It looks like the repo was created but no files pushed. I'm not sure what went wrong as I can't see any logs or messages anywhere.

It's getting late so I don't have time to look into that one sorry. But once that's cleared up, I think we're good to go 🚀

[mashehu]

@nf-core-bot fix linting

[ewels]

Summary of review items so that they don't get lost. See Slack for details + screencaps etc.

Bugs

• [x] Selecting "nf-core" as the pipeline type, then going back and changing your mind doesn't work
• [x] New repos are always created at the user's namespace, even if a different org was specified at the start
• [ ] When creating a custom pipeline, pipeline name should allow more characters
• [x] Hitting some issues pushing code to new repos still where it seems to be prompting me for username + password interactively, behind the Textual app

Feature requests

• [x] Can check if the pipeline outdir exists before continuing past "Force" option page.
• [ ] Should show the org + repo name that will be created. Better still as inputs that can be edited.
• [ ] Add a button after creating a repo to open that new repo in the web browser
• [x] Can remove toggle for "Push files" as I think the user will always want this

[mirpedrol]

I created issues to track what is missing from the TUI https://github.com/nf-core/tools/issues/2963 https://github.com/nf-core/tools/issues/2964 https://github.com/nf-core/tools/issues/2965

If there aren't major concerns, I would merge this PR and continue the development in smaller PRs.