gh-132661: Document t-strings and `templatelib`

[davepeck]

[DRAFT] This has only just begun and is very much a work in progress; putting up a draft PR now for visibility.

We're keeping a TODO list of PEP 750 documentation tasks. There's lots to do.

• Issue: gh-132661

---

📚 Documentation preview 📚: https://cpython-previews--135229.org.readthedocs.build/

[davepeck]

@hugovk Thank you for the review and the helpful early feedback! Useful to know about linking.

(FWIW this is very early stage still, so I expect plenty more linting and other issues that we'll eventually resolve.)

[davepeck]

@AA-Turner @willingc @lysnikolaou Marking this as ready for review. Would love to get a bunch of eyes on it. Thanks!

[treyhunner]

@davepeck The below thoughts are largely things that could be improved after this pull request is merged, if they are improved at all. The documentation currently in this pull request is well-worth merging in my opinion. I am looking forward to seeing this feature officially documented soon!

The standard library page

I noticed that string.templatelib is not linked from the Python standard library reference page but it is linked from the slightly deeper Text Processing Services page. I'm not sure whether this has already been discussed, but that seemed a bit confusing to me. I expected to find it on that page when I searched template and then I thought maybe it would be mentioned at the top of the string page instead. It took some clicking to realize it was accessible under "Text Processing Services".

Feedback on the string.templatelib page

In the "See also" box at the top my eyes read the links as one long link rather than 3 short links:

Putting commas between them or separating them over multiple lines might improve readability.

I could see myself reading the introduction at the top of the page and assuming that "t-strings return strings":

Template strings are an extension of f-strings that allow for greater control of formatting behavior. The Template class gives you access to the static and interpolated (in curly braces) parts of a string before they are combined into a final string.

If "extension of f-strings" was "extension of the f-string syntax" that might help a bit, but the end of the final sentence ("into a final string") might lead me to that conclusion most strongly. I would consider noting "Unlike f-strings, the t-string literal syntax creates a Template object" between those two sentences or somehow more strongly implying or noting that t-strings return Template objects rather than strings.

I like that the string.templatelib.Template starts off by noting that t-strings are the usual way to make a Template object. That fact seems easy to miss and it's great that it's mentioned first.

The Template and Interpolation attributes and methods look well-documented to me. One (possibly useful?) fact that seems to be missing from the documentation I've read so far is the fact that the parts/components/elements of a Template object will always alternate back and forth between str and Interpolation with one more string than interpolation (which also means they'll start and end with a string part).

Aside thought on "parts/components/elements": I'm not sure there's an official name for the "things you get when you loop over a Template". This isn't necessarily a problem for using t-strings, but would make explaining them easier. I often find myself unsure of how to teach concepts that don't have an official (or even semi-official) name.

Input and Output tutorial page

The "Template String Literals" section under the "Input and Output" page of the tutorial lines up nicely with the f-string section and explains the purpose fairly well while also being pretty succinct.

For the final bit about returning HTML elements, I found myself wanting more with this bit:

A full implementation of this function would be quite complex and is not provided here. That said, the fact that it is possible to implement a method like parse_html() showcases the flexibility and power of t-strings.

I would appreciate seeing a working example of a function that does not return a string.

It seems like maybe a different example should be used, or an additional full example of something shorter.

You're welcome to use a shorter various of this re.compile alternative that works with t-strings, though there may be better examples that are also relatively short.

[blaisep]

... in heaven, everyone reviews code like @treyhunner

[davepeck]

@treyhunner Thank you for the amazing review!

I noticed that string.templatelib is not linked from the Python standard library reference page but it is linked from the slightly deeper Text Processing Services page

Fixed now!

In the "See also" box at the top my eyes read the links as one long link rather than 3 short links:

Fixed; thanks.

I could see myself reading the introduction at the top of the page and assuming that "t-strings return strings":

Agreed. I took another pass at the wording here. I still don't really love it; how does it work for you now?

One (possibly useful?) fact that seems to be missing from the documentation I've read so far is the fact that the parts/components/elements of a Template object will always alternate back and forth

It's in there but, yeah, it's scattered across description of Template's constructor, its .strings and .iterpolations properties, and iter(template).

The key property (which is included in the docs) is that there is always one more element in Template.strings than in Template.interpolations.

Related doc content:

• When you construct a Template instance, you can pass str and Interpolation arguments in any order and the constructor will do the work under the hood to maintain the .strings/.interpolations count property (including inserting empty strings or concatenating adjacent strings as necessary)

• When you iter(template) we skip empty strings (also described); if you want to grab the empty strings (say, for memoization) then you need to access .strings directly.

I hope this is okay? Earlier versions of the PEP explicitly mentioned alternation and made guarantees about it; the final form of the PEP mostly tries to avoid talking about it. If there's a way we can make these properties clearer sooner in the cpython docs, I'd love to find it — suggestions most welcome.

Aside thought on "parts/components/elements": I'm not sure there's an official name for the "things you get when you loop over a Template".

Hah! I've been going back and forth between "parts" and "items". You're absolutely right, we should settle on something here. I looked over the PEP, docs, and even some older discussions and "parts of a Template" seems to be the most common. I'll make updates where it makes sense.

I would appreciate seeing a working example of a function that does not return a string.

It seems like maybe a different example should be used, or an additional full example of something shorter.

You're welcome to use a shorter various of this re.compile alternative that works with t-strings, though there may be better examples that are also relatively short.

Thank you, this is useful feedback. I agree we should have something "complete" here. Will consider your cool re.compile example and circle back.

[encukou]

BTW, there's no need to update the PR if it's out of date with the main branch. GitHub just made the button for that too prominent.

[davepeck]

BTW, there's no need to update the PR if it's out of date with the main branch. GitHub just made the button for that too prominent.

Hah, sorry; I know. It's sorta become a personal bad habit on PRs against active repos. I will refrain! 😅

[davepeck]

@treyhunner I just applied a suggestion from @encukou that I think nicely clarifies the alternation. Let us know what you think. (And: thanks @encukou !)

[davepeck]

@encukou I removed the t-string tutorial for now. When I get a chance next week, I'll pop up a separate PR so we can wrangle it into shape. Onward! :-)

[davepeck]

Added @AA-Turner's documentation for string.templatelib.convert().

[hugovk]

At EuroPython sprints, @AA-Turner says he still wants to review this.

As release manager, I've given him until the first release candidate on Tuesday when I'll merge this. We can always make further updates, but let's at least have something for RC1.

[davepeck]

Thanks @picnixz for the feedback! Think I took care of all your suggestions.

On Sphinx' side, I think we should really do something for supporting !~ because it becomes annoying to lose semantics.

That would be really nice!

[AA-Turner]

@davepeck @lysnikolaou this PR is from an organisation fork (@t-strings), so we can't apply suggestions or push commits. Please would you be able to do so? Hugo wants to get this PR in before starting RC1 later today.

A

[hugovk]

Let's merge and add the updates in a followup. Thanks all! 🫖🧵📚🚀

[miss-islington-app[bot]]

Thanks @davepeck for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. 🐍🍒⛏🤖

[bedevere-app[bot]]

GH-136974 is a backport of this pull request to the 3.14 branch.

[AA-Turner]

See #137020 for the promised followup.

A