downlit icon indicating copy to clipboard operation
downlit copied to clipboard

Published 20 hours ago • verified publisher icon r-lib

factor out tokenization and export it

Open thomasp85 opened this issue 10 months ago • 0 comments

As discussed outside of GitHub, this PR factors out the tokenisation that downline uses in highlight() and exports it for use by other packages. It is also being used by highlight() now.

I have not added tests since the behaviour is covered by the extensive testing of highlight(), but let me know if you want to have this

thomasp85 avatar Apr 26 '24 07:04 thomasp85