visidata
visidata copied to clipboard
[status-] decrease latency of status() when other threads run
While benchmarking, I noticed that vd.status()
is slow to finish when vd is running other very busy threads (taking ~150ms). If there are no busy threads, it's much faster, (taking ~2ms).
By recording the run time in memory, I found that it's only a matter of latency. That is, inspect.stack()
takes a lot of clock time to finish, around 150 milliseconds, but very little of that time is spent in the thread. I don't know why, but I assume stack()
pauses to let the busy threads run.
It turns out inspect.stack()
is slow because it reads source files. And it can be made much faster by not asking for lines of source file context.
Here is a comparison of the latency, before and after this PR.
Before:
Using inspect.stack()
, while loading a 2 million line tsv file takes 150-190ms per call to status()
:
quit.vdj.txt
seq 2000111 > 2m.tsv
# load 2 million line tsv file then quit
vd -p quit.vdj
status() time spent in inspect.stack(), total time:
mean clock time: 0.152629 seconds
status() time spent in inspect.stack(), in thread:
mean time in thread: 0.002206 seconds
After:
using inspect.stack(0)
, it takes only 40-50ms:
vd -p quit.vdj
status() time spent in inspect.stack(), total time:
mean clock time: 0.051474 seconds
status() time spent in inspect.stack(), in thread:
mean time in thread: 0.001276 seconds
Hi and thanks for reporting this issue.
Can you open the exact same URL in your browser?
Yes. I am able to open the URL in the browser. Also, If I place the JSON file on the physical drive and access i.e., C:\Users\rmaddu\Desktop\tiles.json, no issues.
This issue is with accessing the URL directly. How to resolve this? the logfile is not giving many clues...no information in EventViewer also. Please help us to resolve this issue.
Thanks for your time.
Try http://127.0.0.1/VectorTileService/tiles.json please
Hi, getting the same error with http://127.0.0.1/VectorTileService/tiles.json.
Please let me know if you need any further information.