HypEx icon indicating copy to clipboard operation
HypEx copied to clipboard

[BUG] future warning pandas

Open tikhomirovd opened this issue 9 months ago • 0 comments

⚠️ Issue: Remove FutureWarning in Pandas Backend

Description

A FutureWarning is raised when using .fillna() in pandas_backend.py. This warning indicates that downcasting object dtype arrays on .fillna(), .ffill(), and .bfill() is deprecated and will change in future versions of Pandas.

Steps To Reproduce

  1. Run HypEx with dataset processing that involves missing value handling.
  2. Observe the warning in the logs:
    /Users/tikhomirov/Documents/Sber/HypEx/hypex/dataset/backends/pandas_backend.py:444: FutureWarning: 
    Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. 
    Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set 
    `pd.set_option('future.no_silent_downcasting', True)`
    return self.data.fillna(value=values, **kwargs)
    

Expected Behavior

  • The warning should be resolved by explicitly handling the dtype conversion.
  • Suggested fix: Use .infer_objects(copy=False) after .fillna() to ensure compatibility with future Pandas versions.

Suggested Solution

  • Modify the relevant line in pandas_backend.py:
    result = self.data.fillna(value=values, **kwargs)
    result = result.infer_objects(copy=False)  # Ensure correct dtype handling
    return result
    

Environment

  • HypEx Version: 1.0.0
  • Python Version: 3/9+

Checklist

  • [ ] Identify all occurrences of .fillna() usage.
  • [ ] Apply .infer_objects(copy=False) where needed.
  • [ ] Ensure compatibility with existing HypEx workflows.
  • [ ] Test and verify that the warning no longer appears.

Related Issues

  • May be affected by changes in Pandas versioning. Consider checking for related deprecations.

Let’s fix this before the next Pandas update breaks compatibility! 🚀

tikhomirovd avatar Mar 13 '25 12:03 tikhomirovd