cudf icon indicating copy to clipboard operation
cudf copied to clipboard

[BUG] cudf.pandas wrapped numpy arrays not compatible with numba

Open AjayThorve opened this issue 2 months ago • 4 comments

Describe the bug When I try to use cudf.pandas with datashader, I get an error Cannot determine Numba type of <class 'cudf.pandas._wrappers.numpy.ndarray'>, full repro below. Datashader actually works directly with cudf, and a cudf.DataFrame is an exceptable data format. But using cudf as a no-code-change accelerator for pandas, this seems to fail.

Steps/Code to reproduce bug

import cudf.pandas
cudf.pandas.install()

import pandas as pd
import numpy as np
import datashader as ds
import datashader.transfer_functions as tf
from datashader.colors import inferno

# Create a small dataset
np.random.seed(0)
n = 1000
df = pd.DataFrame({
    'x': np.random.normal(0, 1, n),
    'y': np.random.normal(0, 1, n)
})

# Create a canvas to render the plot
cvs = ds.Canvas(plot_width=400, plot_height=400)

# Aggregate the points in the canvas
agg = cvs.points(df, 'x', 'y')

# Render the plot using a transfer function
img = tf.shade(agg, cmap=inferno, how='eq_hist')

# Display the plot
img

Output

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at [/home/ajay/miniconda3/envs/rapids-24.06/lib/python3.11/site-packages/datashader/glyphs/glyph.py](http://localhost:8888/lab/tree/dev/miniconda3/envs/rapids-24.06/lib/python3.11/site-packages/datashader/glyphs/glyph.py) (66)

File ".[./miniconda3/envs/rapids-24.06/lib/python3.11/site-packages/datashader/glyphs/glyph.py", line 66](http://localhost:8888/lab/tree/dev/miniconda3/envs/rapids-24.06/lib/python3.11/site-packages/datashader/glyphs/glyph.py#line=65):
    def _compute_bounds(s):
        <source elided>

    @staticmethod
    ^ 

This error may have been caused by the following argument(s):
- argument 0: Cannot determine Numba type of <class 'cudf.pandas._wrappers.numpy.ndarray'>

Expected behavior Ideally same output as a cudf or a pandas dataframe.

Environment overview (please complete the following information)

  • Environment location: Ubuntu
  • Method of cuDF install: Conda

AjayThorve avatar May 07 '24 19:05 AjayThorve