[BUG] Convert cuDF to Triton Object does not work with null values

[bschifferer]

Bug description

If I use convert_df_to_triton_input with a dataframe containing Null values, then I get an error:

ValueError: Column must have no nulls.

We could replace .values_host with .to_pandas().values. However, that will change the nan value to -2147483648

Expected behavior

I can convert DataFrames with Nulls

[karlhigley]

I don't think this is expected to work, since I don't think Triton supports nulls in tensors. Could you post the full stack trace though, so we can double-check where the issue is surfacing and what (if anything) we can do about it?

[bschifferer]

I dont have the example available, right now. It happens in this line: https://github.com/NVIDIA-Merlin/systems/blob/main/merlin/systems/triton/init.py#L55

I don't think Triton supports nulls in tensors - This is an issue when we deploy NVTabular workflows to Triton. We support Operators like FillMissing and FillMedian. A user can define a training pipeline, which imputes the missing values. When we deploy it to Triton Inference Server, the user cannot provide the same input data and therefore, he/she will get other predictions.

@EvenOldridge FYI

[karlhigley]

Okay, if this is a thing we were already doing that stopped working, could you fill me in on what example this breaks? I understand that there's an issue here, but I don't yet have much to go on for reproducing or troubleshooting it.

[viswa-nvidia]

retiring this bug. PR is merged.

[bschifferer]

@viswa-nvidia why do you retire this bug? The PR is a short-term workaround. I run into the same bug, right now.

[karlhigley]

@bschifferer Could you provide more information on this issue? Like a minimal repro and a full stack trace? (It's been...116 days since I asked for more info to help me troubleshoot, so it doesn't seem unreasonable to close this issue as stale at this point.)

[bschifferer]

@karlhigley I am sorry, I didnt know that this was missing.

import cudf
import tritonclient.grpc as grpcclient

from merlin.systems.triton import convert_df_to_triton_input

df = cudf.DataFrame({
    'col1': [0,1,None,2,3,None],
    'col2': [0.0, 1.0, None, 2.0, 3.0, None]
})

convert_df_to_triton_input(['col1', 'col2'], df, grpcclient.InferInput)

I think this is more a high-level questions - do we want to support FillNa / FillMedian ops in NVTabular? Because we are not able to send data with NA values to Triton

[bschifferer]

@viswa-nvidia - Yes, this is still relevant