[Discussion - PyCon] Pyrefly onboarding UX

[DEKHTIARJonathan]

Feature Idea - Onboarding UX

As with every type checkers - the onboarding experience can be absolutely brutal. As soon as you turn on any type checker for the first time - you immediately end up with 2000+ (if you're lucky) exceptions and it can be extremely overwhelming and it's a very clear limit to adoption and user experience. Their "first contact" with pyrefly (as with every other type checker) will be rigged with frustration and bad memories. Not exactly the feeling you want to give your users / early adopters.

What could pyrefly do ? What is the "dream" ?

As I said - not pyrefly specific - it's an issue every typecheckers has. However - a fresh start can give an opportunity to offer a fresh approach.

Imagine ...

pyrefly init / onboard / migrate / letsgo / gamify  # pick something fun 👍 

What does it actually do ?

We want to "smoothly ease" the user into a world of types. Not brutally attack them with 2000+ exceptions.

1. Analyze the entire codebase

2. Establish an N steps gradual transition plan (how many is left as an exercise for the reader 💯 ) A step in the plan is a partially filtered "view" of the entire codebase. The idea is to gradually enable more and more of the codebase to show "errors"/"exceptions" Gradually enabled more and more of the codebase - let the user "win".

   1. Find "islands" of the code - parts that don't import anything/much. And focus on them
   2. Compute the dependency graph - focus on the top-most (central-most) part of the codebase - Focus on them.
   3. Gradually get less and less centric.

The plan can be generated and dumped to a pyrefly-plan.toml/json

And then you can have pyrefly next-level/step

It's doesn't need to be the "most perfect / effective plan" - it just needs to be easier and less violent than flipping the switch and BOOM.

Not only it's awesome for UX - it also allows easier merges. Nobody likes merging a 2000 lines change PR. But merging 10 PRs of 200 lines. Much better :)

And if you play it well - it's very easy to "gamify"

[ndmitchell]

I like the ideas behind this! We already have pyrefly init which is meant to be the start of this process. CC @MaggieMoss who has been driving our onboarding experience.

[maggiemoss]

Hi!

I feel pretty much the same way, and am working to making this experience better. Thanks for the feedback :)

pyrefly init currently exists, and tries to pull configuration options from a mypy or pyright config to make onboarding a touch easier. I am planning on improving it over the coming weeks, and implementing some of your suggestions above.

While we continue to improve it, there are a couple tools / strategies you might want to try in the meantime.

(1) pyrefly check --summarize-errors

This will give you a high level breakdown of what kind of errors you have in your project and where they are coming from. Occasionally when onboarding a new project, it's easy to spot a handful of files that make up the bulk of the errors. Knowing this information can help you decide where to turn typing on initially. For example, if 1500 of your 2000 errors come from a single directory, you might want to add typing everywhere else, and then onboard that directory slowly.

As well, if the majority of your errors are one specific kind (.e.g, all import-errors), you know there might be an issue with the configuration file. Or you can simply ignore that category of error in your initial PR.

(2) Error Silencing

We have a script that will automatically add error suppressions to your project for you, so you don't have to do this manually.

pyrefly check --suppress-errors

This comes with an initial cost of adding type suppression comments to your code, but allows you to see at a glance where the issues are.

(3) # pyrefly: ignore-all-errors

This comment at the top of a file will ignore all type errors from within it. Can be a good way to onboard a large directory and ignore files that have a ton of errors when you get started.

---

If you give them a try, I would love to hear about your feedback or any suggestions you might have.

[DEKHTIARJonathan]

Will try - it was more a generic statement in terms of UX than really a personal frustration. I will try pyrefly when #307 is fixed. Until then can't do anything

[lolpack]

Hi @DEKHTIARJonathan have you tried this since recent updates? We've added new functionality that will suppress different sections of code based on the amount of errors it finds. Would love your feedback based on the changes made.