Edward Kim
Edward Kim
@ssubbayya Can you please share more information on how you arrived at that warning? A minimal reproducible code would be great. I'm particularly confused because you are using the `merlin-pytorch`...
I'm working on updating the `merlin-tensorflow` image to `23.06` here: https://github.com/NVIDIA-Merlin/Merlin/pull/1040. After bumping the image version to `23.06` and updating the processing workflow in `train.py` to reflect recent changes, and...
@wei-m-teh Apologies for the delay. It's in review at the moment, but I updated #1040 with a workaround I found for making the notebook work with the latest `23.08` image.
@bschifferer Why did we decide to split the dataset in the multi-gpu example instead of [repartitioning the dataset](https://github.com/NVIDIA-Merlin/models/blob/d4453cb599ef7ace289da758dff2c0ce11e69700/tests/unit/tf/horovod/test_horovod.py#L47-L52) with `row_group_size`? Did repartitioning not work?
> I dont think repartitioning is an option. If you have 1TB dataset, does that work? How long will repartition take? We are using dask dataframes so it's out of...
@jperez999 Is there a way to produce equal number of batches so that the workload is balanced across workers? Although nvtabular seems to produce equal-sized batches in [tf_trainer.py](https://github.com/NVIDIA-Merlin/NVTabular/blob/main/examples/multi-gpu-movielens/tf_trainer.py), the number...
Could this be related to https://github.com/NVIDIA-Merlin/dataloader/issues/76? It sounds like calling `loader.stop()` or better yet the context manager could help release the memory properly.
@Ilyushin Thanks for reporting the issue. Can you provide more details so we can reproduce the issue on our end? - Did you use our merlin containers, e.g., `nvcr.io/nvidia/merlin/merlin-pytorch:22.11` or...
This seems to be due to the version of `cudf` in the `nvcr.io/nvidia/pytorch:22.06-py3` container. In the older version of `cudf` (prior to 22.04), the `keep_index` parameter was not available in...
Even with #132, the device assignment still doesn't work for list columns: ```python import os import pandas as pd from merlin.dataloader.torch import Loader from merlin.io.dataset import Dataset #dataset = Dataset(pd.DataFrame({"a":...