Matthew Powers
Matthew Powers
@rajagurunath - the global _metadata file that Dask is currently creating is not scalable and I think we should ignore that file for purposes of this project. Assume `write_metadata_file=False`. Let's...
@rajagurunath - I actually think this is going to be super easy. Think this will work: ```python from deltalake import DeltaTable import dask.dataframe as dd dt = DeltaTable("tmp/some-delta-pyspark") ddf =...
Sounds good, keep me posted ;)
Awesome, thanks @kalbasit, appreciate the response. I don't have time to maintain this either. Maybe we could put out some messaging that we're looking for a maintainer? There is a...
@pkoplik24 - Thanks for pointing out this edge case. I think the function should error out if orderedComparison=false and ignoreColumnNames=true. We can have it return a descriptive error message that...
@snithish - Have you used `assertColumnEquality` yet? It's way better than `assertSmallDataFrameEquality` usually... Can you please help me get the `assertDoubleTypeColumnEquality ` method to work properly: https://github.com/MrPowers/spark-fast-tests/blob/master/src/main/scala/com/github/mrpowers/spark/fast/tests/ColumnComparer.scala#L50-L65 This method is...
@reecedoyle - Thanks for submitting this. I am in Costa Rica at the moment. Will take a close look when I'm back. This is really needed!
@reecedoyle - thanks again for this contribution. I'm sorry for the very late follow-up. I completely forgot about this. Can you refactor this code to add a `assertDeepSmallDatasetEquality` rather than...
@jckegelman - I dropped the ball on this one and don't work on Spark anymore, so in order to add this functionality, I'd probably need a new pull request with...
@carlsverre @nvander1 @gorros - Can you please take a look and provide thoughts on the best error message we can provide users for DataFrame inequality comparisons? Thanks!