koalas icon indicating copy to clipboard operation
koalas copied to clipboard

ValueError: Cannot describe a DataFrame without columns

Open FK7 opened this issue 4 years ago • 5 comments

Hi,

I am using koalas in Cloudera Spark 2.4.0 and when I describe dataframe, I get "ValueError: Cannot describe a DataFrame without columns".

`

type(ekko_df)
databricks.koalas.frame.DataFrame

ekko_df.columns Index(['client', 'purchasing_document', 'company_code', 'purch_doc_category', 'purchasing_doc_type', 'control_indicator', 'deletion_indicator', 'status', 'created_on', 'created_by', ... 'budget_type', 'otb_check_status', 'otb_reason_reason', 'type_of_otb_check', 'otb_relevant_contract', 'indicator_level_for_contracts', 'distrib_using_target_value_or_item_data', 'swap_contract', 'externalsystem', 'externalreferenceid'], dtype='object', length=156)

ekko_df.describe()
Traceback (most recent call last): File "", line 1, in File "/opt/anaconda3/envs/python_3_6_environment/lib/python3.6/site-packages/databricks/koalas/frame.py", line 7552, in describe raise ValueError("Cannot describe a DataFrame without columns") ValueError: Cannot describe a DataFrame without columns

`

FK7 avatar Nov 04 '20 19:11 FK7

Can you show the types of your DataFrame? It's possible that your DataFrame does not have any numeric types:

>>> df = ks.DataFrame({'a': ["a", "b", "c"]})
>>> df.describe()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../koalas/databricks/koalas/frame.py", line 7582, in describe
    raise ValueError("Cannot describe a DataFrame without columns")
ValueError: Cannot describe a DataFrame without columns

HyukjinKwon avatar Nov 05 '20 12:11 HyukjinKwon

It works in pandas so we should probably fix too.

HyukjinKwon avatar Nov 05 '20 12:11 HyukjinKwon

Let me take a look at this one.

itholic avatar Nov 16 '20 06:11 itholic

What is the status on this?

lgallindo avatar Dec 15 '21 14:12 lgallindo

@lgallindo We currently focus on pandas API on Spark, which is ported Koalas into PySpark.

I just created related ticket here, and will fix it in pandas API on Spark first.

I'd recommend you to use PySpark as we actively update features and bug fix there, and now the Koalas repository is now in maintenance mode.

itholic avatar Dec 16 '21 02:12 itholic