zed
zed copied to clipboard
vector engine projection
The first step toward a working vector engine is simple projection. In this issue, we will wire the vector engine into the runtime so that it is actually invocable from a lake query. When all of the objects to be scanned have vectors (VNG files) and a projection is deemed worthwhile, the optimizer will rewrite the DAG to use a VecScan and push the projection into the scanner. VecScan will pull only the fields needed from the vcache and materialize the results into zbuf.Batches of serialized sequence data.
This will work only with vcache and lake data. zq running on VNG files will not be supported (yet).