lakeFS icon indicating copy to clipboard operation
lakeFS copied to clipboard

lakeFSFS: Define target performance and measure performance

Open talSofer opened this issue 1 year ago • 1 comments

  1. Define target performance for lakeFSFS used with Spark and Hive tables
  2. Define performance benchmarks and guide to running them
  3. Measure performance and document results

Target performance and measurements should be defined for:

  • Write a 1000-partition Parquet file (for instance).
  • Read from all objects of a 1000-partition Parquet file (for instance). A good refinement is to read all fields and also to read just one field from the file.

talSofer avatar Dec 26 '22 12:12 talSofer