futurecoder icon indicating copy to clipboard operation
futurecoder copied to clipboard

Better tokenization of revealed solution

Open alexmojaki opened this issue 5 years ago • 0 comments

The get_solution function uses the standard Python tokenizer to split an exercise solution into tokens which the user can gradually reveal. This works pretty well, but:

  1. f-strings are considered a single massive token, which reveals too much information in a single click.
  2. There's no syntax highlighting :cry:

I think a good solution would be to use Pygments to highlight the solution and then treat each individual span as a separate token to be revealed by the user. For example, this Python code:

print(f"{name} went to {meal} with {friend}.")

gets converted into:

<span><span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> went to </span><span class="si">{</span><span class="n">meal</span><span class="si">}</span><span class="s2"> with </span><span class="si">{</span><span class="n">friend</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
</span>

Each span like <span class="nb">print</span> is a token. Any whitespace between tags (I don't see any in the example above) should be preserved, but is already 'revealed' from the beginning.

This could probably be done either by hooking into pygments, e.g. writing a custom formatter, or parsing the HTML produced by the standard formatter.

alexmojaki avatar Oct 28 '20 11:10 alexmojaki