parquet-dotnet icon indicating copy to clipboard operation
parquet-dotnet copied to clipboard

DateTime data columns raise exceptions

Open vladislav-stolyarov opened this issue 5 years ago • 1 comments

Version: Parquet.Net v. 3.7.4

Runtime Version: .Net Core v 2.1

OS: Windows

Expected behavior

DateTime columns work fine

Actual behavior

Exception System.AggregateException: 'One or more errors occurred. (Unable to cast object of type 'System.DateTime' to type 'System.DateTimeOffset'.)' at Parquet.Data.Concrete.DateTimeOffsetDataTypeHandler.WriteAsInt96(System.IO.BinaryWriter, System.Collections.IList)

Steps to reproduce the behavior

See simple code example below

Code snippet reproducing the behavior

var dateField = new DataField<DateTime>("Date");
            var schema2 = new Schema(new Field[] {dateField});

            using (var writer = new ParquetWriter(schema2, stream))
            {

                using (ParquetRowGroupWriter groupWriter = writer.CreateRowGroup())
                {
                    DataColumn dataColumn = new DataColumn(dateField, new []{DateTime.Now});
                    groupWriter.WriteColumn(dataColumn);
                }
            }

vladislav-stolyarov avatar May 19 '20 13:05 vladislav-stolyarov

I don't know if it's still the case but I've only been able to get it to work with DateTimeOffset's. Try using that instead of DateTime's.

E.g. var dateField = new DataField<DateTimeOffset>("Date");

mukunku avatar Jul 09 '21 02:07 mukunku