visidata icon indicating copy to clipboard operation
visidata copied to clipboard

Significantly slower selection of matching error cells in v3.0.2 vs much older version

Open NickOveracker opened this issue 4 months ago • 1 comments

Small description Yesterday I installed an old version of vd with a very outdated version of pip. I don't remember the original version number, but it wasn't version 3. The interface and plugin list matched the tutorial screenshots.

Today, I updated pip and installed the latest version of VisiData, v3.0.2.

I'm not at liberty to share the full dataset, but I can share a single screen of it below.

vd_scrsh

Expected result The cmdlog should have executed completely within a few seconds.

Actual result with screenshot The "select rows matching current cell in column" commands took a very long time to execute. For the commands executed in the screenshot below, the first several commands took around 6 seconds, and the final "select" command took 91 seconds to complete.

The cell was a Python error cell for a calculated column. I was selecting all cells with errors in order to determine the most common sources of errors.

vd_scrsh2

Steps to reproduce with sample data and a .vd The following sample dataset and cmdlog are sufficient to reproduce the issue.

faa-wildlife-strikes.csv faa-wildlife-strikes.vdj.txt

Additional context VisiData v3.0.2 Python 3.8.0

NickOveracker avatar Mar 02 '24 09:03 NickOveracker