twoslash icon indicating copy to clipboard operation
twoslash copied to clipboard

Using the protocol for other languages (in particular python)

Open Nova38 opened this issue 1 year ago • 6 comments

I find twoslash enabled doc to be really helpful when reading typescript projects that use them in their docs. I have been working on a few personal projects that use shiki for syntax highlighting some python files. I saw that there was a twoslash protocol that was created (https://github.com/twoslashes/twoslash/issues/17#issue-2118958025) to allow for things other than the typescript language server to generate the info that twoslash needs to render it. I was wondering what the steps would be to get the data generated into twoslash for rendering in to shiki?

Also if i wanted to include the docstrings for the nodes would I need to convert them into the jsdoc equivalents or what format do the strings of the types bellow need to be?

https://github.com/twoslashes/twoslash/blob/fcc90743beb3d5e26d1dda2e2cda02920bd092ec/packages/twoslash-protocol/src/types/nodes.ts#L15-L25

Nova38 avatar Dec 24 '24 03:12 Nova38

How much is the difference between Python's doc strings and jsodc? You do not have to convert them into jsdocs, it's just a data field and it's up to the renderer to parse the string to DOM. So it's possible that you provide your own patched renderer or so to render directly with docstring.

antfu avatar Dec 24 '24 04:12 antfu

Python has several different docstring standards so it is a little less unified. However their plugins for sphinx that complice them into structured data that is more standardized that I could pass to a patched renderer like you suggested

Nova38 avatar Dec 24 '24 23:12 Nova38

Is there anything for python to use the twoslash protocol?

bismitpanda avatar Mar 07 '25 11:03 bismitpanda

Not yet I haven't had a chance to work on it yet.

I have to ideas on how to go about implementing it though. One option is to extract the needed information for the typing, hover, etc... from a language server and then sterilize it into the twoslash protocol format.

Microsoft has a tool in python for extraction info from an Lsp that could be useful. This approach would potentially be scalable out to other languages using their respective lsp's. https://github.com/microsoft/multilspy

The other option is to use a library like giffe (https://mkdocstrings.github.io/griffe/) that provides a way to anylis python code and extract types, doc strings, etc.... Then either write a python script to produce it or transform the json directly.n

Nova38 avatar Mar 08 '25 06:03 Nova38

It also looks like there is a second protocol that exist on top of lsp that is ment for sterilizing some of the Lsp data to be called on by a much more limited environment. I haven't gotten far into the docs on it yet, but it might be handy for this use case.

https://microsoft.github.io/language-server-protocol/overviews/lsif/overview/

Nova38 avatar Mar 12 '25 02:03 Nova38

@Nova38 here is a working draft https://github.com/julien-blanchon/twoslash-python/tree/main

julien-blanchon avatar Mar 21 '25 20:03 julien-blanchon