Terry Kim

By the way, this applies only to the driver side log, not the worker. So, #542 may not be a good long term solution. Integrating with `log4net` or something similar...

> Would it be possible to replace log4net with a more modern and maintained logging framework ([serilog](https://github.com/serilog/serilog) maybe)? `serilog` should be good. Do you want to work on this? If...

Cool. Here is what I am thinking: 1) Implement `ILoggerService` with `serilog`. 2) Update `SparkEnvironment` to set a logger based on an environment variable. Note that the work side should...

> It takes a lot of effort to detect the number of partitions in a dataframe What's the use case where you need to know the number of partitions? >...

@AFFogarty Do you see any reasonable place to mention Delta NuGet package? Even the main README should be fine.

Thanks @jammman for reporting this. Can you run the following and leave the outputs? ``` countriesDF.Select("Id", "PopulationCount", "Year", "CountryId").printSchema(); birthRatesDF.printSchema(); birthRatesDF.explain(true) ``` The local Spark version is also 2.4.5 right?...

Oh, thanks @jammman for trying this out. We were planning to experiment the same. :) And it looks like the `FlatMapGroupsInPandasExec.scala:155` is different from the OSS Spark, so there must...

@AFFogarty Can you help with fixing Delta tests?

> I notice in the test results that the Lists have the same content (see image below). I see "..." in the output. Can you compare elements one by one...

What I meant was if you are comparing two lists, you can compare elements in those lists one by one instead of comparing them at the list level. Does it...