doltpy
doltpy copied to clipboard
dolthub
`write_pandas` silently discarding rows that contain NaNs
Hi there,
After a lot of debugging I just noticed that the write_pandas wrapper is silently discarding rows that contain NaN values.
Here is the point where this happens.
Is this behavior intended? Can't we toggle it providing this as additional parameter to the function?
Expected behavior:
In my case there are a lot of rows in my DataFrame that contains NaNs value but are just fine from the application logic perspective, and I would just like to store my entire DataFrame to dolt without dropping anything.
PS: even in case the drop is needed for any reason, it would make sense to print a warning to the user. In my case it was successfully concluding the write with no errors, but the fact that 50% of the content was silently dropped :)
