Takuya UESHIN

In that case, I'd also suspect `ARROW_PRE_0_15_IPC_FORMAT` is not set properly. Could you try: ```py import os os.environ.get('ARROW_PRE_0_15_IPC_FORMAT', 'None') ``` and ```py from pyspark.sql.functions import udf @udf('string') def check(x): return...

Also, what if with an older PyArrow than `0.15.0`, like `0.14.1`?

Hi @amueller, Seems like `Series.sample` supports the `frac` parameter now. - https://koalas.readthedocs.io/en/latest/reference/api/databricks.koalas.Series.sample.html For #1893, now it's stuck by a performance concern (https://github.com/databricks/koalas/pull/1893#discussion_r521869698). Could you kindly advice us if you have...

Thanks for escalating the issue here! It seems related to Spark UDT and Arrow. Actually we can have a workaround to convert from/to pandas DataFrame/Series including Spark UDT objects, but...

Hi @vkrot-exos, thanks for the suggestion! It sounds a good idea. Would you mind submitting the PR to modify the error message? Thanks!

Does the function `toposort` have a return type annotation? If not, Koalas collects some amount of data into the driver to infer the return type. See also: - https://koalas.readthedocs.io/en/latest/reference/api/databricks.koalas.groupby.GroupBy.apply.html -...

FYI: for the read path, it was resolved at #1695.

@shakirshakeelzargar Unfortunately, Spark doesn't support such operations, and neither does Koalas, at least so far. If the file is small enough, you can use pandas and convert it to Koalas....

Seems like pandas doesn't support it as well: ```py >>> import pandas as pd >>> pdf = pd.DataFrame({'A': [1, 1, 2, 2], 'B': ['x', 'x', 'x', 'y']}, columns=['A', 'B']) >>>...

ah, I see. we might want to support it. cc @HyukjinKwon