Investigate shutdown latency on `mlr head`

[johnkerl]

Originally posted by @GiulioCentorame in https://github.com/johnkerl/miller/issues/1501#issuecomment-1963437137

I tried pulling from head and it works, thank you so much!

The interesting behaviour now is that mlr --itsv --otsv head -n 10 then cut -f f.eid [file] prints the correct 10 lines almost immediately, but then it takes a while for miller to return the exit status.

E.g. this is miller 5.10.3

$ time mlr --itsv --otsv head -n 10 then cut -f f.eid file1.tab
f.eid
[correct lines]

real    0m16.136s
user    0m16.045s
sys     0m0.035s

and this is miller from aff4b9f32dde3ee75aca260f822bfd17d91aa4a0 (interrupted manually)

$ time ./miller/mlr --itsv --otsv head -n 10 then cut -f f.eid file1.tab
f.eid
[correct lines]
^C^C^C

real    6m15.200s
user    6m0.513s
sys     1m19.823s

[johnkerl]

@GiulioCentorame can you please provide mlr version and head -n 1 file1.tab?

I am trying to reproduce this, using the data-generator from #1507, and I can't find a slowdown ...

$ time mlr --tsv head then cut -f k0000055 ~/tmp/wut.tsv
k0000055
v0000055
v0000055
v0000055
v0000055
v0000055
v0000055
v0000055
v0000055
v0000055
v0000055

real	0m0.541s
user	0m0.556s
sys	0m0.056s

$ time mlr --tsv check ~/tmp/wut.tsv

real	0m37.809s
user	0m41.230s
sys	0m1.438s

$ wc -l ~/tmp/wut.tsv
 2000000 /Users/kerl/tmp/wut.tsv

$ head -n 2 ~/tmp/wut.tsv
k0000000	k0000001	k0000002	k0000003	k0000004	k0000005	k0000006	k0000007k0000008	k0000009	k0000010	k0000011	k0000012	k0000013	k0000014	k0000015k0000016	k0000017	k0000018	k0000019	k0000020	k0000021	k0000022	k0000023k0000024	k0000025	k0000026	k0000027	k0000028	k0000029	k0000030	k0000031k0000032	k0000033	k0000034	k0000035	k0000036	k0000037	k0000038	k0000039k0000040	k0000041	k0000042	k0000043	k0000044	k0000045	k0000046	k0000047k0000048	k0000049	k0000050	k0000051	k0000052	k0000053	k0000054	k0000055k0000056	k0000057	k0000058	k0000059	k0000060	k0000061	k0000062	k0000063k0000064	k0000065	k0000066	k0000067	k0000068	k0000069	k0000070	k0000071k0000072	k0000073	k0000074	k0000075	k0000076	k0000077	k0000078	k0000079k0000080	k0000081	k0000082	k0000083	k0000084	k0000085	k0000086	k0000087k0000088	k0000089	k0000090	k0000091	k0000092	k0000093	k0000094	k0000095k0000096	k0000097	k0000098	k0000099
v0000000	v0000001	v0000002	v0000003	v0000004	v0000005	v0000006	v0000007v0000008	v0000009	v0000010	v0000011	v0000012	v0000013	v0000014	v0000015v0000016	v0000017	v0000018	v0000019	v0000020	v0000021	v0000022	v0000023v0000024	v0000025	v0000026	v0000027	v0000028	v0000029	v0000030	v0000031v0000032	v0000033	v0000034	v0000035	v0000036	v0000037	v0000038	v0000039v0000040	v0000041	v0000042	v0000043	v0000044	v0000045	v0000046	v0000047v0000048	v0000049	v0000050	v0000051	v0000052	v0000053	v0000054	v0000055v0000056	v0000057	v0000058	v0000059	v0000060	v0000061	v0000062	v0000063v0000064	v0000065	v0000066	v0000067	v0000068	v0000069	v0000070	v0000071v0000072	v0000073	v0000074	v0000075	v0000076	v0000077	v0000078	v0000079v0000080	v0000081	v0000082	v0000083	v0000084	v0000085	v0000086	v0000087v0000088	v0000089	v0000090	v0000091	v0000092	v0000093	v0000094	v0000095v0000096	v0000097	v0000098	v0000099

[GiulioCentorame]

Sorry for the slow reply!

$ mlr --version
mlr 6.12.0

Here's the tsv file: file1_header.zip