itables icon indicating copy to clipboard operation
itables copied to clipboard

WIP: Adding Narwhals as compatibility layer between libraries

Open DeaMariaLeon opened this issue 11 months ago • 12 comments
trafficstars

Related to issue #325

If we add the proposed code, nothing changes when using a pandas dataframe. And the rest of the libraries can be handled with Narwhals. That's needed because Narwhals and Polars don't allow duplicated column names, so the pandas path has to be kept.

We also have to do this because on iTables (currently), the returned results from the function datatables_row are different when using Polars vs. pandas in some cases.

So, with the proposed changes, this function would now work for:

import pyarrow as pa 
df = pd.DataFrame({"a": [2, 1]})
datatables_rows(pa.table(df))

out:
'[[2], [1]]'

Or this:

import modin.pandas as mpd
df = pd.DataFrame({"a": [2, 1]})

data = {"a": [1, 2, 3]}
datatables_rows(mpd.DataFrame(pd.DataFrame(data))

out:
'[[1], [2], [3]]'

I could eliminate from lines 88 to 114 from file datatables_format.py and handle all with Narwhals. But the results would be different for some cases in test_databases_format.py. I can elaborate...

I would like to have some feedback before going further please. Thanks. Friendly ping @MarcoGorelli

DeaMariaLeon avatar Nov 29 '24 19:11 DeaMariaLeon