Expressify and unicode

[SoongNoonien]

As mentioned in https://github.com/thofma/Hecke.jl/pull/1784 I think the expressify system should take is_unicode_allowed into account. At least for all the unicode characters translated for latex output.

[thofma]

Can you elaborate? Here is how I understand it: The latex printer turns unicode characters into latex output with some heuristic. An implementor of expressify(::MyType,..) must insert unicode if he wants to see unicode or see it handled properly by the latex printer respectively.

[SoongNoonien]

Yes, exactly. The problem I see is that if do exactly that I'll get unicode output on the console no matter what is_unicode_allowed says. But according to the documentation unicode printing should depend on is_unicode_allowed. So, I'd like to have nice automatic latex printing while still having a unicode free printing on the console if is_unicode_allowed returns false. But currently I'd have to implement this manually without using the expressify system.

[thofma]

There is probably a misunderstand. Yes, unicode should only be printed if is_unicode_allowed. An implementor of show must respect this rule. If one wants to use show via expressify, then the implementor of expressify must respect this. Here is an example from https://github.com/oscar-system/Oscar.jl/blob/ef5cf56446fd0ea165b33550075a0d4bd84bd617/src/LieTheory/RootSystem.jl#L921:

function expressify(r::RootSpaceElem; context=nothing)
  if is_unicode_allowed()
    return expressify(r, :α; context)
  else
    return expressify(r, :a; context)
  end
end

The function expr_to_string will just print whatever expression you throw at it. It will and should not replace anything depending on is_unicode_allowed.

[SoongNoonien]

Here is a concrete example related to https://github.com/thofma/Hecke.jl/pull/1784:

julia> Qx, x = QQ["x"];

julia> f = x^16 - x^15 - x^14 + 2*x^11 - x^8 - x^7 + x^6;

julia> fac = factor_cyclotomic(f)
x^6*Φ_4*Φ_2^2*Φ_3*Φ_1^4

julia> show(stdout, "text/latex", fac)
x^{6} {\Phi}_{4} {\Phi}_{2}^{2} {\Phi}_{3} {\Phi}_{1}^{4}

The Φs are translated to \Phi for the latex output but not in the console. And as far as I know there is no possiblity for me to know in my expressify method in which context it is called right now. The context flag is always nothing. So if I replace the Φs manually depending on is_unicode_allowed, the latex output will be modified too.

[thofma]

Yes, this is all working as intended. context is about capturing the IOContext and possible properties there. LaTeX output is controlled via MIME types. Both are orthogonal.

If a user wants nice LaTeX output from unicode symbols, the user must set is_unicode_allowed to true.

So in the expressify of the cyclotomic polynomials, one checks if unicode is allowed and inserts a Φ. Otherwise one inserts some non-unicode character.

[SoongNoonien]

If a user wants nice LaTeX output from unicode symbols, the user must set is_unicode_allowed to true.

Ok, I see. I think this should be highlighted in the docs. Because at least to me this is not intuitive. And I'd think most users wouldn't expect a connection either. Especially considering that unicode printing is disabled by default.

So in the expressify of the cyclotomic polynomials, one checks if unicode is allowed and inserts a Φ. Otherwise one inserts some non-unicode character.

Are there some guidelines on which non-unicode characters to use for some standard greek letters and so on?

[lgoettgens]

There is probably a misunderstand. Yes, unicode should only be printed if is_unicode_allowed. An implementor of show must respect this rule. If one wants to use show via expressify, then the implementor of expressify must respect this. Here is an example from oscar-system/Oscar.jl@ef5cf56/src/LieTheory/RootSystem.jl#L921:

function expressify(r::RootSpaceElem; context=nothing)
  if is_unicode_allowed()
    return expressify(r, :α; context)
  else
    return expressify(r, :a; context)
  end
end

The function expr_to_string will just print whatever expression you throw at it. It will and should not replace anything depending on is_unicode_allowed.

After just discussing this with @SoongNoonien, I would honestly find it way easier for downstream users if they would not need to check is_unicode_allowed() themselves, but instead could throw whatever unicode madness they want (maybe with a fixed list of allowed things) towards expressify, and expressify then decides to leave it as unicode, convert to latex, convert to ascii, etc. based on flags such as is_unicode_allowed

[thofma]

expressify then decides to leave it as unicode, convert to latex, convert to ascii, etc. based on flags such as is_unicode_allowed

Yes, this is exactly the current system. exxpressify does exactly this, because expressify is implemented for a new type, where the developer must consider the flags.

Pretty printing provides two things:

• throw an expression at it and it will simplify the expression (canonicalize)
• print an expression (expr_to_string), this is where the julia IOContext can be passed on deal with :compact flags etc.

Both things are neatly separated, which makes it easy to reason about the behavior. If you want to change the behavior, you have to throw at it different expressions. expr_to_string will not randomly change symbols.

Since julia separates MIME types and IOContext, we do the same. How about you implement a show method for the latex MIME type and insert the unicode characters you want if it is allowed?

Edit: Formatting and printing is highly opinionated, which is OK. But this means we cannot have a ruleset unicode => ascii, because people will find some random choice there unacceptable.