Faisal

Results 118 comments of Faisal

> But it's locopy's code, `unload_and_copy` method If you don't use `locopy` and have concurrency scaling enabled does your query work. Keep in mind I don't have your setup or...

@maxim-lisovsky-gismart Thanks for flagging this. I'll take a closer look shortly.

@ryantaylor406 Late to the party. Not sure if this would be helpful but I maintain a library which allows you to compare to Polars DataFrames and get a report back....

> Just following up to see if there are any updates on this. I help maintain a tool that does compares if you need something. Polars is supported! https://github.com/capitalone/datacompy

@paddymul Sorry for the delay. This is great! appreciate you using and building some useful tools around datacompy. I like the idea of having something visual like you've implemented here,...

Thanks for the report. Would you be able to provide a minimal example so we can reproduce the issue? That would be really helpful to debug here.

Thanks for the details. So it seems like I can reproduce it. I also tried using Pandas and Fugue: ### Pandas ```python df_1 = pd.DataFrame([{"a": 1}]) df_2 = pd.DataFrame([{"a": 1}])...

@rupertbarton more for my understanding but could you articulate what sort of use case you have where you are just joining on a single column with nothing else to compare?

@jdawang @rupertbarton I have a WIP fix [here](https://github.com/capitalone/datacompy/compare/develop...spark-single-column-bug) Getting the following back: ```bash In [2]: print(compare.report()) ****** Column Summary ****** Number of columns in common with matching schemas: 1 Number...

This is a stale issue, and might be fixed now that we've moved to `SparkSQLCompare` over the legacy version. Comparing the following: ```python df_1 = spark.createDataFrame([{"a": 1}]) df_2 = spark.createDataFrame([{"a":...