Iaroslav Igoshev

I tried to replace sqlalchemy-databricks to databricks-sql-python as follows ```bash --- a/pyproject.toml +++ b/pyproject.toml @@ -10,18 +10,18 @@ packages = [{include = "pandasai"}] [tool.poetry.dependencies] python = ">=3.9,3.9.7,=2.0,

My bad :) The package on pypi is databricks-sql-connector but not databricks-sql-python. I was able to generate a new lock file but pandas is not still a latest one. There...

@gventuri, is this issue planned to be fixed?

The issue is reproducible even before introducing the shared memory feature.

Reopening so we find the root cause.

Hi folks, it's been a while since opening this issue :) 3 years later Modin added support for conversion of a Modin DF to a Dask DF and vice versa....

Hi @yx367563, thanks for filing this issue! You should not wrap `process_data` into `ray.remote` decorator. Modin itself takes care of distributing computation. If you remove `ray.remote` decorator and still see...

@yx367563, can you try calling this before to_parquet? ```python import modin.config as cfg with cfg.context(NPartitions=1): df = df._repartition(axis=0) df.to_parquet(...) ``` This should write a single parquet file.

@yx367563, sorry, I didn't put the code correctly. Please see the updated comment above.

@yx367563, the operations such as storage and inserting columns should also perform well depending on the data size. It would be great if you could share the exact script and...