Ideas for the question wizard

[alexmojaki]

The question wizard was added in #181. These are some ideas for improvement:

• Add more linting as in #101
• Deal with files. Pyodide has a virtual filesystem in the browser's memory, but has no access to the host machine's files. Files also add complication to a question. Handling this requires careful patching of builtins.open.
  • When the user reads a file that doesn't exist, show the user the file path and prompt them to upload it through a standard file picker. Save the file and the path, but don't write the file to that exact path. This way if the file is modified by subsequent code, the user doesn't have to upload it again.
  • Next time the user reads from that path, load it from the uploaded files.
  • When the user writes a file, remember the path.
  • When generating the question, include the contents of text files (both written and read) below a certain size, along with the paths.
  • Maybe give users a way to download files they've written?
• Remove irrelevant code to make the question minimal.
  • Try removing statements (or chunks of statements), rerunning the code, and seeing if it produces the same output.
  • Try replacing code like foo = complicated + expression(here) with the resulting value if it's a literal, e.g. foo = [1, 2].
  • Show the user changes that seem to work. Let them decide if the change should be made, and probably let them make it themselves.
  • Ideally, the user should be able to reject a suggestion in such a way that it won't be shown again but the system can still make other suggestions.
  • Look into automated debugging resources, e.g. https://www.debuggingbook.org/
• Handle different kinds of question better. Right now there's a lot of text in the first step and it doesn't even cover everything. We could potentially ask a tree of questions like:
  • Do you have some code to ask about? If no, point them to https://stackoverflow.com/help/dont-ask like now.
  • Does the code work?
    • No: usual flow, continue to next question.
    • "Yes, but I don't understand how it works" - can check the AST for things like list comprehensions, *args, or list slicing to point them in the right direction.
    • "Yes, but I want to make it better" - cautiously suggest https://codereview.stackexchange.com/
  • Does your code involve servers, web requests, databases, etc?
    • If yes, is the question really about those things, or is that just the context? Can you isolate the core problem from those things?
      • If no, point to https://stackoverflow.com/help/minimal-reproducible-example like now.
  • Can you put all your code in one 'file', or is the nature of imports part of your question?
    • If no, maybe suggest some resources on imports, sys.path, etc.
• Handle infinite loops somehow?
• Show the values of variables and expressions like in the traceback but when there was no error.

[alexmojaki]

Comments from https://meta.stackoverflow.com/questions/411764/is-it-appropriate-to-recommend-my-own-question-wizard-tool/411779

John Kugelman:

The overall workflow wasn't entirely intuitive: that you keep hitting "Run" to get from stage to stage. It looks like a REPL site and those don't work like that. A separate "Next" button might help there.

Trilarion:

Maybe the community could in general construct a better question wizard. For example, could you provide a separate (re)search section in the wizard that must have at least some content (and guidance)? Most new questions simply do not show any research and are likely duplicates and I hope that more explicitly requiring research could improve the situation.

Putting these here because they both remind me of the "Handle different kinds of question better" point above. Here are my responses:

John Kugelman: Running the code is the only thing that makes sense most of the time. It just becomes weird when they're supposed to enter their expected output. I agree that part is a bit weird. I could also show the expected output input immediately but I didn't want to overwhelm the user. A 'Next' button only for the expected output feels weird too. What do you envision? What do you think of Trilarion's idea and the "Handle different kinds of question better" point above?

Trilarion: Can you expand on how this might look? I'm struggling to imagine what I could do that would be actually helpful.

[jkugelman]

I don't actually know what I envision. I figured I should point out my initial confusion, however I'm hesitant to suggest solutions that I've put, like, 10 minutes of thought into.

I'll give an idea or two but jeez, you've put like 1000 more hours of thought into this than I have.

I guess, for me, the page has kind of a split personality. It's a question wizard, but it looks a lot like a REPL site. Maybe the REPL part is taking too much focus away from the wizard-ness.

For instance, the two-column multi-panel UI is perhaps too complicated for the target audience of newbie questioners. The real meat is in the left half, yes? That's the actual wizard. What if you de-emphasized the right side, the code and output panes? Just spitballing here: the question wizard could take up the full page width. Instead of a "question text" text box and a "code" text box, there could just be the "question text" text box with the code inside it. The "Run" button would run the code that's embedded in the question text. Having a single text box would really streamline the editing so there aren't two separate places to edit. Also the output pane could be a temporary popup after hitting "Run", but maybe it wouldn't be permanently on screen.

I don't know if that makes sense. Okay here's a quick mockup:

If they click "Run" they'd see:

[jkugelman]

What do you think of Trilarion's idea and the "Handle different kinds of question better" point above?

Prompting askers to describe what research they've done would be awesome. I often ask people to link articles they've read to demonstrate their research. (They never actually do.) I'd love to even ask people, "What google searches did you perform? Type in the exact search terms you tried." Cause I have this strong feeling that lots of people have poor googling skills and Stack Overflow is not set up to help people learn how to google better. Lately I've started dropping comments like "This is a duplicate of {link}. I googled 'stackoverflow python list duplicate' to find that."

[alexmojaki]

Thanks for taking the time to put this together! I'll have to think about the UI for a while.

Prompting askers to describe what research they've done would be awesome.

Do you think it would actually be productive to make users enter links and search terms to finish the wizard? I'm worried it would seem weird and condescending.

Also do you think there should be a full question wizard for questions without code? Something that inserts the links and search terms into a generated question? To me that feels a bit redundant. It's interesting that you consider the left half of the UI to be the meat, I see the code part as the critical differentiator from an article giving static generic advice.

[alexmojaki]

BTW the course (the main part of the site) does actually have a page for practicing googling: https://futurecoder.io/course/#HowToFindInformationWithGoogleAndMore

[jkugelman]

Yeah it would definitely be weird and condescending.

[jkugelman]

It's interesting that you consider the left half of the UI to be the meat, I see the code part as the critical differentiator from an article giving static generic advice.

Well, from a UX perspective I think the meat is the text of the question the user is trying to build--which includes the code embedded in it. I love the REPL functionality and filling out the "actual output" section. Can that functionality be integrated into the left side so it's all one package? That's where my thoughts take me.

Again, 10 minutes' worth of thoughts.

And I know this is like when your boss comes by with a "Can't you just...?" that would take 6 months of rework... "I LOVE it, now can you make it completely different?"

Actually, to step back a bit, I was delighted when I saw your Meta post. I wish someone at Stack Exchange would read it and go "wow, we should use that!" Cause God, the question posting system right now is really broken. I've dreamt about a wizard like yours for years.