parquet-dotnet icon indicating copy to clipboard operation
parquet-dotnet copied to clipboard

ReasAsTable() Inaccurate Summary Comment

Open mukunku opened this issue 5 years ago • 0 comments

I just wanted to mention that the summary on the method is misleading: image

After a quick look at the code in ParquetExtensions.cs it seems like it's actually reading all the row groups.

/// <summary>
/// Reads the first row group as a table
/// </summary>
/// <param name="reader">Open reader</param>
/// <returns></returns>
public static Table ReadAsTable(this ParquetReader reader)
{
 Table result = null;

 for(int i = 0; i < reader.RowGroupCount; i++)
 {
	using (ParquetRowGroupReader rowGroupReader = reader.OpenRowGroupReader(i))
	{
	   DataColumn[] allData = reader.Schema.GetDataFields().Select(df => rowGroupReader.ReadColumn(df)).ToArray();

	   var t = new Table(reader.Schema, allData, rowGroupReader.RowCount);

	   if(result == null)
	   {
		  result = t;
	   }
	   else
	   {
		  foreach(Row row in t)
		  {
			 result.Add(row);
		  }
	   }
	}
 }

 return result;
}

I'm using Parquet.NET v3.6.0

mukunku avatar Mar 26 '20 13:03 mukunku