eds-scikit icon indicating copy to clipboard operation
eds-scikit copied to clipboard

Fix: Switch default `to_koalas` in `HiveData._read_table`. to False

Open Vincent-Maladiere opened this issue 1 year ago • 0 comments

Description

The to_koalas argument has been deprecated in _read_table since it wasn't used. Returning koalas DataFrame during this step allows for more predictable behavior and fewer worries due to OOM errors.

When to_koalas is set to True, we raise a warning to indicate that this argument is not used. Since this argument is True by default, we raise this warning for every table to be read, which is unpleasant for the user.

Therefore, this PR suggests setting to_koalas to False by default.

Checklist

  • [x] If this PR is a bug fix, the bug is documented in the test suite.
  • [x] Changes were documented in the changelog (pending section).
  • [x] If necessary, changes were made to the documentation (eg new pipeline).

Vincent-Maladiere avatar Mar 30 '23 14:03 Vincent-Maladiere