Inconsistent behavior for "c"-type ...item functions

[josephwright]

With a simple example

\tl_show:e { \tl_item:Nn \foo { 1 } }
\tl_show:e { \tl_item:cn { foo } { 1 } }

one gets an error with the first line but not with the second: more importantly, if the index is 1, you get a very strange result here.

At the very least we should document that the tl must exist, even if performance concerns mean we don't make a change. But I suspect we should hand-create the variant here with a test.

(This relates to for example https://tex.stackexchange.com/questions/717206/texlive-2024-chemnum-error-when-referencing-unknown-ids in that prop don't allow undefined storage since the move to a two-implementation approach.)

[user202729]

There is \debug_on:n, so I don't think it's too difficult to check that token lists are indeed token lists only if debug support is turned on (with no performance penalty when debug is turned off).

[josephwright]

There is \debug_on:n, so I don't think it's too difficult to check that token lists are indeed token lists only if debug support is turned on (with no performance penalty when debug is turned off).

That doesn't catch this particular case so we might need to adjust there

[user202729]

It should catch this case with correct implementation right? Because something that is implicitly defined by c-type expansion would have meaning \relax, which is not a token list (which must have meaning of a macro).

[muzimuzhi]

It seems the current convention in l3debug.dtx is to not do checks if low-level errors will always occur. https://github.com/latex3/latex3/blob/35b5295e4b2c83425962a380a00a22e9aa575b01/l3kernel/l3debug.dtx#L684-L693

[davidcarlisle]

I think c behaviour here is consistent. The fact that c locally defines the token to \relax is slightly weird but TeX is TeX, and c has done that for over 30 years and I don't think we can or should change that.

The questionable thing here is the behaviour of \tl_item:Nn if the first argument is not expandable (which arguably should never occur as it's documented as being a tl.)

\documentclass{article}

\begin{document}

\ExplSyntaxOn

\let\foo\relax
\tl_show:e { \tl_item:Nn \foo { 1 } }

\ExplSyntaxOff
\end{document}

shows \foo.

We could document that specifically for \tl_item:Nn or more generally document that in functions where the N variant is effectively a variant of an n base function, the argument passed to the base is the first level expansion of the token (which is the token itself if that token is not expandable)

[user202729]

It seems the current convention in l3debug.dtx is to not do checks if low-level errors will always occur.

Should the convention be kept anyway? (given that it isn't too necessary to optimize for performance in debugging mode) Having more informative error message can never hurt, and the behavior of "low-level TeX" can be hard to predict.

[josephwright]

@davidcarlisle Should we try to pick those up in debug mode?

[josephwright]

It seems the current convention in l3debug.dtx is to not do checks if low-level errors will always occur.

Should the convention be kept anyway? (given that it isn't too necessary to optimize for performance in debugging mode) Having more informative error message can never hurt, and the behavior of "low-level TeX" can be hard to predict.

Well at the low level we do know how things are implemented, which is the point here - for example, \int_set:Nn \l_some_undefined_int will error, so we don't need to trap it at the macro level

[davidcarlisle]

@davidcarlisle Should we try to pick those up in debug mode?

I'd be tempted to document this as a feature, both \csname defining the token as \relax and \expandafter\foo\unexpandabletoken being \foo\unexpandabletoken are basic tex functionality used all over and to catch them in debug mode would make debug mode deviate even more from the normal flow, I'm sure it's possible but that has its own risks.

For c, I think we should globally document that the token passed on will be locally let to \relax if not defined previously.

I don't think there is a general notion currently of :N being a variant of an :n base (in fact interface3 explicitly says n and N pass the arguments on with no modification), which would mean that documenting specifically for \tl_item:Nn that it is \tl_item:on although again interface3 doesn't actually say what o does if the token to be expanded once isn't expandable but we could document that in that case the token is passed as-is (rather than making it an error)

[user202729]

Well at the low level we do know how things are implemented, which is the point here - for example, \int_set:Nn \l_some_undefined_int will error, so we don't need to trap it at the macro level

Is it? It isn't too hard to imagine cases where it doesn't lead to an error

\int_set:Nn \use_none:nn {1}

interface3 doesn't actually say what o does if the token to be expanded once isn't expandable but we could document that in that case the token is passed as-is (rather than making it an error)

Personally it's hard for me to imagine a case where that behavior is expected (can anyone give me one?)

If there isn't, it seems more likely to be an user error. So I don't see much advantage in making it silently eat up the error (of course the disadvantage would be that it is harder for the user to find the error), especially because we don't need to care about performance that much in debug mode.

[josephwright]

Well at the low level we do know how things are implemented, which is the point here - for example, \int_set:Nn \l_some_undefined_int will error, so we don't need to trap it at the macro level

Is it? It isn't too hard to imagine cases where it doesn't lead to an error

\int_set:Nn \use_none:nn {1}

interface3 doesn't actually say what o does if the token to be expanded once isn't expandable but we could document that in that case the token is passed as-is (rather than making it an error)

Personally it's hard for me to imagine a case where that behavior is expected (can anyone give me one?)

If there isn't, it seems more likely to be an user error. So I don't see much advantage in making it silently eat up the error (of course the disadvantage would be that it is harder for the user to find the error), especially because we don't need to care about performance that much in debug mode.

But that is stuff that would be a hard error even outside debug mode. Remember that what l3debug is for is checking correctness of expl3 code, largely - so stuff that you can test for anyway isn't in scope. In your example, any simple test probing the functionality will pick up an issue.