visidata icon indicating copy to clipboard operation
visidata copied to clipboard

[status-] decrease latency of status() when other threads run

Open midichef opened this issue 10 months ago • 0 comments

While benchmarking, I noticed that vd.status() is slow to finish when vd is running other very busy threads (taking ~150ms). If there are no busy threads, it's much faster, (taking ~2ms).

By recording the run time in memory, I found that it's only a matter of latency. That is, inspect.stack() takes a lot of clock time to finish, around 150 milliseconds, but very little of that time is spent in the thread. I don't know why, but I assume stack() pauses to let the busy threads run.

It turns out inspect.stack() is slow because it reads source files. And it can be made much faster by not asking for lines of source file context.

Here is a comparison of the latency, before and after this PR.

Before: Using inspect.stack(), while loading a 2 million line tsv file takes 150-190ms per call to status(): quit.vdj.txt

seq 2000111 > 2m.tsv
# load 2 million line tsv file then quit
vd -p quit.vdj
status() time spent in inspect.stack(), total time:
mean clock time:      0.152629 seconds
status() time spent in inspect.stack(), in thread:
mean time in thread:  0.002206 seconds

After: using inspect.stack(0), it takes only 40-50ms:

vd -p quit.vdj
status() time spent in inspect.stack(), total time:
mean clock time:      0.051474 seconds
status() time spent in inspect.stack(), in thread:
mean time in thread:  0.001276 seconds

midichef avatar Apr 04 '24 03:04 midichef

Hi and thanks for reporting this issue.

Can you open the exact same URL in your browser?

mnboos avatar Sep 24 '20 12:09 mnboos

Yes. I am able to open the URL in the browser. Also, If I place the JSON file on the physical drive and access i.e., C:\Users\rmaddu\Desktop\tiles.json, no issues.

This issue is with accessing the URL directly. How to resolve this? the logfile is not giving many clues...no information in EventViewer also. Please help us to resolve this issue.

Thanks for your time.

tanish-creator avatar Sep 24 '20 15:09 tanish-creator

Try http://127.0.0.1/VectorTileService/tiles.json please

mnboos avatar Sep 24 '20 15:09 mnboos

Hi, getting the same error with http://127.0.0.1/VectorTileService/tiles.json.

tanish-creator avatar Sep 24 '20 16:09 tanish-creator

Please let me know if you need any further information.

tanish-creator avatar Sep 25 '20 06:09 tanish-creator