mcabbott
Print bug
There seems to be a glitch in the printed rows when the number of rows are 20 or greater. Notice the gap between 9-11 in the printout below.
This glitch prevents TerminalPager.jl to work nicely with the KeyedArray https://github.com/ronisbr/TerminalPager.jl/issues/75
julia> N=20; k = KeyedArray(randn(N,N), x=1:N, y=1:N)
julia> sprint(show, MIME"text/plain"(), k) |> println
2-dimensional KeyedArray(NamedDimsArray(...)) with keys:
↓ x ∈ 20-element UnitRange{Int64}
→ y ∈ 20-element UnitRange{Int64}
And data, 20×20 Matrix{Float64}:
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) (16) (17) (18) (19) (20)
(1) 1.12257 0.490402 0.487161 1.3417 -0.61508 1.00027 0.75978 -1.01997 -0.507342 2.17005 -0.171787 -0.379487 -0.990666 -0.84708 0.0829884 0.911194 0.142935 -2.65874 -0.527115 0.245108
(2) 2.4968 -0.93532 -0.397074 -0.110128 0.925698 -0.319974 -0.824232 -0.774187 0.163896 0.129388 2.79476 -0.907305 0.539264 -1.3593 0.121594 0.924924 0.619213 0.613901 -0.723006 -0.188586
(3) 0.0208007 -0.286999 0.593305 1.27435 0.712491 0.409889 0.61506 0.766101 -2.15692 -0.354933 1.62464 -0.622595 0.75396 -1.06038 0.300836 -1.09592 0.684804 -0.462004 0.0621972 -0.111631
(4) 0.808812 0.446653 -0.2332 0.76007 -1.09043 -0.0431402 -0.927826 -0.628285 0.405432 -0.29322 -1.26878 -0.236054 1.62768 0.918532 -0.0981469 1.54129 1.26642 0.977583 1.83501 0.0320194
(5) -0.273673 -0.617164 0.78719 0.693395 1.06863 -0.763251 -0.949814 0.342036 -1.32664 2.20419 0.230068 1.50768 -2.28972 0.0611954 -0.517557 0.261454 0.940659 -0.540622 0.842569 1.76228
(6) 0.477936 1.53226 1.12786 0.502254 1.3192 -0.275781 0.495464 0.10667 -0.27726 -0.227957 -0.949501 -1.05338 0.611621 0.625151 1.19027 -0.294704 0.368311 -1.1258 0.950899 -1.68034
(7) 0.570887 -0.586242 0.0604913 1.80167 -0.0126626 -0.726972 0.445168 0.569077 1.54544 0.340749 0.238241 0.807731 -0.728055 -0.166784 -1.81308 0.935487 -1.36998 2.00573 0.499047 -0.250083
(8) -0.23281 0.00270028 0.621672 2.33665 0.388242 -0.451793 -0.511436 -1.21374 0.292649 -1.44022 -1.50705 -0.381982 0.934232 0.567087 -0.0416158 -0.373854 1.26558 -0.855522 -0.24484 -1.05762
(9) -1.09506 -2.82651 -0.23011 0.169101 0.707845 1.87551 0.326911 -0.79621 0.0405532 -0.108328 -0.458916 -0.49695 -1.21077 1.40631 -0.383105 -0.00819703 -2.0816 0.479381 -1.8039 0.687658
(11) 0.402395 0.067904 0.505419 1.26774 -0.128122 -0.305771 0.518124 0.320105 0.437461 0.5606 0.142811 -0.187419 1.10411 -1.11426 -1.92751 -0.0533123 -0.71343 -0.695421 0.65695 1.796
(12) 2.75515 0.838098 0.260278 0.50885 -0.485894 3.41045 -0.189964 -1.64383 -1.23911 0.876434 -2.89447 0.89373 -1.82964 0.64817 0.669421 -1.61013 0.142432 1.05083 1.69808 1.38645
(13) 0.881214 -1.3055 -0.79923 -0.254773 0.783476 0.342084 0.390894 -0.446912 0.368134 0.441044 -0.450815 0.470341 1.6416 -1.87338 0.0531701 0.123984 -1.54473 -0.780281 -0.36103 1.97403
(14) 1.57546 -0.356563 0.224456 0.465481 -1.54371 0.034207 -2.17408 0.246078 -1.06823 1.54273 -0.432513 0.541496 -0.481721 -0.536497 -0.934225 0.391173 -0.213335 0.56148 -0.471436 -0.047096
(15) 1.19577 -0.263914 1.48525 0.801261 1.37071 -0.0483607 -0.193038 -1.10729 -0.362711 -0.290259 -0.621544 -0.024465 0.104764 0.209293 1.79168 1.12739 1.43481 -0.305706 0.544616 1.21018
(16) -0.740074 0.486808 -1.21628 1.47814 -1.4511 -0.904593 -0.374184 0.111976 0.0153674 0.0282585 -0.805838 -1.30165 -0.117036 0.585149 0.682102 1.06212 -0.498174 -0.937592 -0.235581 -0.829192
(17) -0.486374 -0.0826544 -1.88369 -0.182889 -1.04927 0.568706 -0.982996 -0.405444 0.14416 1.44949 0.939727 0.820213 -0.355105 -1.2758 0.731964 -2.28131 1.18909 0.527651 0.321574 0.131979
(18) 1.32437 -0.849305 0.354739 0.7182 1.07191 -2.73456 1.14495 0.561979 -1.1518 -1.38035 0.201228 -0.663178 -0.735969 -0.350276 -1.05175 -1.43845 0.154855 1.35661 -1.29657 0.157937
(19) -0.0537634 -0.485977 -0.483744 -0.0424855 0.206698 -1.99924 0.274828 -0.368498 -1.87434 -0.00671213 0.0724161 0.575972 0.116639 -0.619535 -0.391665 1.12729 0.934895 -0.810897 -1.77572 0.11368
(20) 0.166995 -0.390374 1.23389 0.0981188 -0.764152 -0.403233 -0.0290021 1.84925 0.136199 0.321659 -0.513166 0.505695 0.0299045 -1.22091 -1.90267 -0.233389 1.73001 0.293789 0.457946 1.93672
Seems like it just cuts the middle part on purpose, to avoid dump infinitely large arrays into terminal.
Would you expect show to print the whole array no matter what, even if it's multiple gigabytes?
It's a little odd that it skips 10 silently, in the above example.
What happens at the ordinary REPL printing (where stdout::IO contains the window size) is that either everything fits, or you get â‹®. With 25x80 window:
julia> N=15; k = KeyedArray(randn(N,N), x=1:N, y=1:N)
2-dimensional KeyedArray(NamedDimsArray(...)) with keys:
↓ x ∈ 15-element UnitRange{Int64}
→ y ∈ 15-element UnitRange{Int64}
And data, 15×15 Matrix{Float64}:
(1) (2) (3) … (14) (15)
(1) 1.47697 0.510451 0.705914 0.20488 -1.28736
(2) 1.30226 -1.46408 -1.41179 0.302384 0.23286
(3) 0.899942 -0.707045 1.81067 -1.30824 -0.393464
(4) 2.62507 -1.47469 0.428942 -0.919739 -1.11851
(5) -0.72418 0.901133 0.145809 … -1.3734 -0.210647
(6) 1.4137 0.0948597 1.98676 -0.671418 -1.86468
(7) -0.228045 0.00302542 1.57991 0.531497 -1.26064
(8) 0.860517 -0.233845 -2.18334 -0.923497 0.744353
(9) -1.01547 -2.16297 -0.0995179 -1.31932 -0.444524
(10) 0.594162 -0.959207 -0.0347569 … 0.0193206 -0.300771
(11) -1.45855 -0.779344 0.908983 0.454173 -0.0160269
(12) 1.06171 0.400011 2.17124 1.47727 0.343414
(13) -0.016208 -0.732777 -0.533414 0.310175 0.382326
(14) -1.29887 -0.0594261 -0.527276 -0.30708 1.43401
(15) 1.02568 -0.209493 0.283405 … -0.59324 0.734355
julia> N=16; k = KeyedArray(randn(N,N), x=1:N, y=1:N)
2-dimensional KeyedArray(NamedDimsArray(...)) with keys:
↓ x ∈ 16-element UnitRange{Int64}
→ y ∈ 16-element UnitRange{Int64}
And data, 16×16 Matrix{Float64}:
(1) (2) (3) … (15) (16)
(1) -1.17311 0.742278 -0.925829 -2.02316 -1.41721
(2) 0.650443 1.08115 -1.60876 -0.365797 -0.247055
(3) -1.27014 0.0617723 -1.61681 -0.624693 0.416926
(4) -0.694252 0.54337 0.160558 -2.40424 -1.0393
(5) -0.0329912 -0.140955 -0.570876 … -1.0687 0.556031
(6) 0.980979 -1.00215 0.0261444 -1.43145 -0.757553
(7) 0.179439 0.54834 1.49001 -1.34443 -1.28616
⋮ ⋱ ⋮
(10) -0.605109 -0.925501 -0.0789025 … -0.342627 -1.30461
(11) 2.33197 0.415978 0.228837 -0.0022514 -1.30805
(12) -1.47114 -0.494195 2.14628 -1.05713 -0.378415
(13) -0.0889765 -0.0289388 -0.200903 -0.175079 -0.42374
(14) 1.42185 1.10661 0.551851 0.393345 0.0778288
(15) -0.0976808 0.107977 1.52832 … -0.241538 0.311516
(16) 0.291658 0.937386 2.23261 1.04164 -0.761929
julia>
Ordinary arrays are similarly truncated, but repr(MIME"text/plain"(), arr) does not know the size and hence does not truncate:
julia> collect(1:30.0)
30-element Vector{Float64}:
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
â‹®
21.0
22.0
23.0
24.0
25.0
26.0
27.0
28.0
29.0
30.0
julia> repr(MIME"text/plain"(), ans)
"30-element Vector{Float64}:\n 1.0\n 2.0\n 3.0\n 4.0\n 5.0\n 6.0\n 7.0\n 8.0\n 9.0\n 10.0\n 11.0\n 12.0\n 13.0\n 14.0\n 15.0\n 16.0\n 17.0\n 18.0\n 19.0\n 20.0\n 21.0\n 22.0\n 23.0\n 24.0\n 25.0\n 26.0\n 27.0\n 28.0\n 29.0\n 30.0"
So maybe it's a bug that this package does truncate in the string, and doesn't put â‹® in there?
@aplavin
Would you expect show to print the whole array no matter what, even if it's multiple gigabytes?
No, but I did expect KeyedArray to work with TerminalPager (see link to issue in OP). I'm not on top of the details here, but
consider how sprint behaves on a Vector:
> sprint(show, MIME"text/plain"(), collect(1:10000), context=:limit => true) |> println # does truncate
> sprint(show, MIME"text/plain"(), collect(1:10000), context=:limit => false) |> println # does not truncate
Whereas
> N=200; k = KeyedArray(randn(N,N), x=1:N, y=1:N)
> sprint(show, MIME"text/plain"(), k, context=:limit => true) |> println # behaves as expected
> sprint(show, MIME"text/plain"(), k, context=:limit => false) |> println # weirdly also truncates
Could there be an issue with how the IOContext :limit propriety is used in here?
Ok now I remember. This package tries to re-use a lot of Base's printing logic. So the way it shows "keys" next to a vector is to make a matrix containing these and the date, fakearray here:
https://github.com/mcabbott/AxisKeys.jl/blob/c5131175216af7ffc43d1697f3ed8bc7435f7820/src/show.jl#L84-L94
At first it did this for the full size. But when someone tried it on long vector, the millions of strings turned out to be a problem. So now, it tries to make this fakearray just big enough to trigger the "..." style printing. How big depends on what displaysize returns.
And when you call sprint, Julia supplies a default size:
julia> displaysize(stdout) # my window
(33, 100)
julia> displaysize(IOContext(IOBuffer())) # the default
(24, 80)
Maybe you can tell from the IOContext that this isn't a real window size, the thing has many confusing properties. :limit does not distinguish these two, but what happens inside sprint / repr I'm not really sure:
julia> get(stdout, :limit, false)
false
julia> get(IOContext(IOBuffer()), :limit, false)
false
Although I wonder what TerminalPager.jl does about huge arrays. Converting the entire thing to a string seems like it will often be pretty expensive... and this intermediate fakearray might be worse as it's Matrix{Any} IIRC.
julia> @time randn(10^7);
0.041015 seconds (3 allocations: 76.297 MiB, 35.31% gc time)
julia> @time string(randn(10^7));
4.536410 seconds (40.00 M allocations: 5.043 GiB, 18.03% gc time)