NVIDIA-Merlin
Add a protocol for expected interface of a model and save input schema to metadata directory
Fixes #676 and #677
Goals :soccer:
- Improve consistency and clarity of models API.
- adding protocol to document expected methods on a Model.
- Enable use of Merlin Model artifacts to be used in Serving concepts more easily.
- Allow additional metadata to be stored about inputs by storing Merlin Schema alongside model artifacts
Implementation Details :construction:
- Provide a protocol that is runtime checkable and serves as a documentation reference of what is expected of a model across the backend implementations.
- Saving input and output Melrin Schema of model (where available) to a Merlin Metadata directory.
- There is more to do here, like dealing with models where we don't always have a schema available. This is a first step in that direction.
Testing Details :mag:
Click to view CI Results
GitHub pull request #680 of commit 83887aba95435064fd64c2139267812428755a8a, no merge conflicts.
Running as SYSTEM
Setting status of 83887aba95435064fd64c2139267812428755a8a to PENDING with url https://10.20.13.93:8080/job/merlin_models/1038/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 83887aba95435064fd64c2139267812428755a8a^{commit} # timeout=10
Checking out Revision 83887aba95435064fd64c2139267812428755a8a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 83887aba95435064fd64c2139267812428755a8a # timeout=10
Commit message: "Add a runtime checkable protocol for expected interface of a model"
> git rev-list --no-walk fad2ca4319158d6d3fe921b40e84586abdc96d91 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3409052551654140212.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.6)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.3.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.9.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.4)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.2.1)
Requirement already satisfied: tornado>=6.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 658 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 3%]
tests/unit/datasets/test_synthetic.py ..... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 4%]
tests/unit/tf/test_core.py ...... [ 5%]
tests/unit/tf/test_dataset.py ................ [ 7%]
tests/unit/tf/test_public_api.py . [ 7%]
tests/unit/tf/blocks/test_cross.py ........... [ 9%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 10%]
tests/unit/tf/blocks/test_interactions.py . [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 15%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 20%]
..................... [ 24%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 24%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 24%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ........... [ 26%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 26%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 26%]
tests/unit/tf/core/test_aggregation.py ......... [ 27%]
tests/unit/tf/core/test_base.py .. [ 28%]
tests/unit/tf/core/test_combinators.py s................... [ 31%]
tests/unit/tf/core/test_index.py ... [ 31%]
tests/unit/tf/core/test_prediction.py .. [ 32%]
tests/unit/tf/core/test_tabular.py .... [ 32%]
tests/unit/tf/core/test_transformations.py s............................ [ 37%]
... [ 37%]
tests/unit/tf/data_augmentation/test_misc.py . [ 37%]
tests/unit/tf/data_augmentation/test_negative_sampling.py .......... [ 39%]
tests/unit/tf/data_augmentation/test_noise.py ..... [ 39%]
tests/unit/tf/examples/test_01_getting_started.py . [ 40%]
tests/unit/tf/examples/test_02_dataschema.py . [ 40%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 40%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 40%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 40%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 40%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 41%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 41%]
tests/unit/tf/inputs/test_continuous.py ..... [ 41%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 46%]
.. [ 47%]
tests/unit/tf/inputs/test_tabular.py .................. [ 50%]
tests/unit/tf/layers/test_queue.py .............. [ 52%]
tests/unit/tf/losses/test_losses.py ....................... [ 55%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 56%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 59%]
tests/unit/tf/models/test_base.py s................. [ 62%]
tests/unit/tf/models/test_benchmark.py .. [ 62%]
tests/unit/tf/models/test_ranking.py .......................... [ 66%]
tests/unit/tf/models/test_retrieval.py ................................ [ 71%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 72%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 74%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 75%]
tests/unit/tf/prediction_tasks/test_regression.py .. [ 75%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 75%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 76%]
tests/unit/tf/predictions/test_base.py ..... [ 77%]
tests/unit/tf/predictions/test_classification.py ....... [ 78%]
tests/unit/tf/predictions/test_dot_product.py ........ [ 79%]
tests/unit/tf/predictions/test_regression.py .. [ 79%]
tests/unit/tf/predictions/test_sampling.py .... [ 80%]
tests/unit/tf/utils/test_batch.py .... [ 81%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 81%]
tests/unit/torch/test_dataset.py ......... [ 83%]
tests/unit/torch/test_public_api.py . [ 83%]
tests/unit/torch/block/test_base.py .... [ 84%]
tests/unit/torch/block/test_mlp.py . [ 84%]
tests/unit/torch/features/test_continuous.py .. [ 84%]
tests/unit/torch/features/test_embedding.py .............. [ 86%]
tests/unit/torch/features/test_tabular.py .... [ 87%]
tests/unit/torch/model/test_head.py ............ [ 89%]
tests/unit/torch/model/test_model.py .. [ 89%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 90%]
tests/unit/torch/tabular/test_tabular.py ... [ 91%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 96%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.11) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]
tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]
tests/unit/tf/test_dataset.py::test_tf_catname_ordering
tests/unit/tf/test_dataset.py::test_tf_map
/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:384: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:807: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/predictions/test_classification.py: 12 warnings
tests/unit/tf/predictions/test_dot_product.py: 2 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file_dh8e1k.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag_.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.7]
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/transformations.py:835: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [4] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========== 647 passed, 11 skipped, 121 warnings in 895.75s (0:14:55) ===========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins3939254575721996199.sh
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Click to view CI Results
GitHub pull request #680 of commit 9e4741bbe7b89f767928b54b29335173dee6cde6, no merge conflicts.
Running as SYSTEM
Setting status of 9e4741bbe7b89f767928b54b29335173dee6cde6 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1450/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 9e4741bbe7b89f767928b54b29335173dee6cde6^{commit} # timeout=10
Checking out Revision 9e4741bbe7b89f767928b54b29335173dee6cde6 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9e4741bbe7b89f767928b54b29335173dee6cde6 # timeout=10
Commit message: "Add save and load methods to Tensorflow Model"
> git rev-list --no-walk 90aabc5dbe367d071a8039b436a9feff86e2a449 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins12322895966040005303.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 661 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 3%]
tests/unit/datasets/test_synthetic.py ..... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 4%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_dataset.py ................ [ 7%]
tests/unit/tf/test_public_api.py . [ 7%]
tests/unit/tf/blocks/test_cross.py ........... [ 9%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 10%]
tests/unit/tf/blocks/test_interactions.py . [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 15%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 20%]
..................... [ 24%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 24%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 24%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ........... [ 26%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 26%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 26%]
tests/unit/tf/core/test_aggregation.py ......... [ 27%]
tests/unit/tf/core/test_base.py .. [ 28%]
tests/unit/tf/core/test_combinators.py s................... [ 31%]
tests/unit/tf/core/test_index.py ... [ 31%]
tests/unit/tf/core/test_prediction.py .. [ 31%]
tests/unit/tf/core/test_tabular.py .... [ 32%]
tests/unit/tf/core/test_transformations.py s............................ [ 36%]
... [ 37%]
tests/unit/tf/data_augmentation/test_misc.py . [ 37%]
tests/unit/tf/data_augmentation/test_negative_sampling.py .......... [ 39%]
tests/unit/tf/data_augmentation/test_noise.py ..... [ 39%]
tests/unit/tf/examples/test_01_getting_started.py . [ 39%]
tests/unit/tf/examples/test_02_dataschema.py . [ 40%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 40%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 40%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 40%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 40%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 40%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 40%]
tests/unit/tf/inputs/test_continuous.py ..... [ 41%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 46%]
.. [ 47%]
tests/unit/tf/inputs/test_tabular.py .................. [ 49%]
tests/unit/tf/layers/test_queue.py .............. [ 51%]
tests/unit/tf/losses/test_losses.py ....................... [ 55%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 56%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 59%]
tests/unit/tf/models/test_base.py s................... [ 62%]
tests/unit/tf/models/test_benchmark.py .. [ 62%]
tests/unit/tf/models/test_ranking.py .......................... [ 66%]
tests/unit/tf/models/test_retrieval.py ................................ [ 71%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 72%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 74%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 75%]
tests/unit/tf/prediction_tasks/test_regression.py .. [ 75%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 75%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 76%]
tests/unit/tf/predictions/test_base.py ..... [ 77%]
tests/unit/tf/predictions/test_classification.py ....... [ 78%]
tests/unit/tf/predictions/test_dot_product.py ........ [ 79%]
tests/unit/tf/predictions/test_regression.py .. [ 79%]
tests/unit/tf/predictions/test_sampling.py .... [ 80%]
tests/unit/tf/utils/test_batch.py .... [ 81%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 81%]
tests/unit/torch/test_dataset.py ......... [ 83%]
tests/unit/torch/test_public_api.py . [ 83%]
tests/unit/torch/block/test_base.py .... [ 83%]
tests/unit/torch/block/test_mlp.py . [ 84%]
tests/unit/torch/features/test_continuous.py .. [ 84%]
tests/unit/torch/features/test_embedding.py .............. [ 86%]
tests/unit/torch/features/test_tabular.py .... [ 87%]
tests/unit/torch/model/test_head.py ............ [ 88%]
tests/unit/torch/model/test_model.py .. [ 89%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 90%]
tests/unit/torch/tabular/test_tabular.py ... [ 90%]
tests/unit/torch/tabular/test_transformations.py ....... [ 91%]
tests/unit/utils/test_schema_utils.py ................................ [ 96%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_dataset.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 10 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 10 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/core/test_transformations.py: 13 warnings
tests/unit/tf/data_augmentation/test_negative_sampling.py: 10 warnings
tests/unit/tf/data_augmentation/test_noise.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 20 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 30 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/predictions/test_base.py: 5 warnings
tests/unit/tf/predictions/test_classification.py: 7 warnings
tests/unit/tf/predictions/test_dot_product.py: 8 warnings
tests/unit/tf/predictions/test_regression.py: 2 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_dataset.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 10 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 10 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/core/test_transformations.py: 10 warnings
tests/unit/tf/data_augmentation/test_negative_sampling.py: 10 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 20 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 28 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 2 warnings
tests/unit/tf/predictions/test_base.py: 5 warnings
tests/unit/tf/predictions/test_classification.py: 7 warnings
tests/unit/tf/predictions/test_dot_product.py: 8 warnings
tests/unit/tf/predictions/test_regression.py: 2 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_dataset.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 10 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/data_augmentation/test_negative_sampling.py: 9 warnings
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 16 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:807: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/predictions/test_classification.py: 12 warnings
tests/unit/tf/predictions/test_dot_product.py: 2 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filesrzbk7gx.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/data_augmentation/test_noise.py::test_stochastic_swap_noise[0.7]
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/transformations.py:835: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [4] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========== 650 passed, 11 skipped, 1024 warnings in 910.86s (0:15:10) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins15284713450606103461.sh
Click to view CI Results
GitHub pull request #680 of commit 6d5a8cf188a3b20818ffd055e25b87c4a00579d3, no merge conflicts.
Running as SYSTEM
Setting status of 6d5a8cf188a3b20818ffd055e25b87c4a00579d3 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1452/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 6d5a8cf188a3b20818ffd055e25b87c4a00579d3^{commit} # timeout=10
Checking out Revision 6d5a8cf188a3b20818ffd055e25b87c4a00579d3 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6d5a8cf188a3b20818ffd055e25b87c4a00579d3 # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 4a8cb1e5184a4b2b10e15b4481323312df995a0c # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4238284117329630891.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 726 items / 1 error
==================================== ERRORS ====================================
______________ ERROR collecting tests/unit/tf/models/test_base.py ______________
tests/unit/tf/models/test_base.py:675: in
@pytest.mark.parametrize("load_fn", [ml.Model.load, load_model])
E NameError: name 'ml' is not defined
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 7 warnings, 1 error in 4.57s =========================
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins6322494041598647194.sh
Click to view CI Results
GitHub pull request #680 of commit 0ad32ae6149f7ca515607ef97050d3a4cfa381a3, no merge conflicts.
Running as SYSTEM
Setting status of 0ad32ae6149f7ca515607ef97050d3a4cfa381a3 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1453/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 0ad32ae6149f7ca515607ef97050d3a4cfa381a3^{commit} # timeout=10
Checking out Revision 0ad32ae6149f7ca515607ef97050d3a4cfa381a3 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0ad32ae6149f7ca515607ef97050d3a4cfa381a3 # timeout=10
Commit message: "Use `mm` instead of `ml` in `test_reload`"
> git rev-list --no-walk 6d5a8cf188a3b20818ffd055e25b87c4a00579d3 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3071704405061529730.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 746 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py . [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py F [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py F [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s................... [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 82%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 83%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 85%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 91%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
___________________ test_example_04_exporting_ranking_models ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f19872cb6d0>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", execute=False)
def test_example_04_exporting_ranking_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
tb.execute()
tests/unit/tf/examples/test_04_export_ranking_models.py:17:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f19872cb6d0>
cell = {'cell_type': 'code', 'execution_count': 14, 'id': 'f999a063', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':... "\x1b[0;31mFileExistsError\x1b[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'"]}], 'source': 'model.save("dlrm")'}
cell_index = 26
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': '1ecbfd72-f87d...e, 'engine': '1ecbfd72-f87d-4a4a-a003-2caffe04138b', 'started': '2022-10-05T09:46:18.195941Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [14], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mdlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:891[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 882[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 883[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 884[0m [38;5;124;03m"""[39;00m
E [1;32m 885[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 886[0m export_path,
E [1;32m 887[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 888[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 889[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 890[0m )
E [0;32m--> 891[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-05 09:46:02.679631: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-05 09:46:04.760448: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-05 09:46:04.760615: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-05 09:46:04.761339: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-05 09:46:04.761396: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
__________________ test_example_06_defining_own_architecture ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f1987307610>
@testbook(
REPO_ROOT / "examples/06-Define-your-own-architecture-with-Merlin-Models.ipynb", execute=False
)
def test_example_06_defining_own_architecture(tb):
tb.inject(
"""
from unittest.mock import patch
from merlin.datasets.synthetic import generate_data
mock_train, mock_valid = generate_data(
input="movielens-1m",
num_rows=1000,
set_sizes=(0.8, 0.2)
)
p1 = patch(
"merlin.datasets.entertainment.get_movielens",
return_value=[mock_train, mock_valid]
)
p1.start()
"""
)
tb.execute()
tests/unit/tf/examples/test_06_advanced_own_architecture.py:28:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f1987307610>
cell = {'cell_type': 'code', 'execution_count': 21, 'id': 'fb89d842', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...leExistsError\x1b[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'"]}], 'source': 'model.save("custom_dlrm")'}
cell_index = 61
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ae7c1d5b-2ae4...e, 'engine': 'ae7c1d5b-2ae4-4b24-bc2e-a1b0824b5c51', 'started': '2022-10-05T09:47:13.864775Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("custom_dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [21], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mcustom_dlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:891[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 882[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 883[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 884[0m [38;5;124;03m"""[39;00m
E [1;32m 885[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 886[0m export_path,
E [1;32m 887[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 888[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 889[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 890[0m )
E [0;32m--> 891[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-05 09:47:05.180984: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-05 09:47:06.550569: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-05 09:47:06.550741: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-05 09:47:06.551491: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-05 09:47:06.551547: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 19 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 19 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:910: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filew6dd463q.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 2 failed, 732 passed, 12 skipped, 1137 warnings in 1133.27s (0:18:53) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins5332098502875795283.sh
Click to view CI Results
GitHub pull request #680 of commit a75a33611b939c4bd7e42878840398d5295ab7de, no merge conflicts.
Running as SYSTEM
Setting status of a75a33611b939c4bd7e42878840398d5295ab7de to PENDING with url https://10.20.13.93:8080/job/merlin_models/1454/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse a75a33611b939c4bd7e42878840398d5295ab7de^{commit} # timeout=10
Checking out Revision a75a33611b939c4bd7e42878840398d5295ab7de (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f a75a33611b939c4bd7e42878840398d5295ab7de # timeout=10
Commit message: "Use InputBlockV2 in model `test_reload`"
> git rev-list --no-walk 0ad32ae6149f7ca515607ef97050d3a4cfa381a3 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins13492200406122966608.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 746 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py . [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py F [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py F [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s................... [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 82%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 83%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 85%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 91%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
___________________ test_example_04_exporting_ranking_models ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7faf90963a60>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", execute=False)
def test_example_04_exporting_ranking_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
tb.execute()
tests/unit/tf/examples/test_04_export_ranking_models.py:17:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7faf90963a60>
cell = {'cell_type': 'code', 'execution_count': 14, 'id': 'f999a063', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':... "\x1b[0;31mFileExistsError\x1b[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'"]}], 'source': 'model.save("dlrm")'}
cell_index = 26
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': '555c9b0c-a2d5...e, 'engine': '555c9b0c-a2d5-4fc7-a09c-aea1b7e02e8d', 'started': '2022-10-05T10:05:27.358707Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [14], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mdlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:891[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 882[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 883[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 884[0m [38;5;124;03m"""[39;00m
E [1;32m 885[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 886[0m export_path,
E [1;32m 887[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 888[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 889[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 890[0m )
E [0;32m--> 891[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-05 10:05:11.851347: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-05 10:05:13.924578: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-05 10:05:13.924748: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-05 10:05:13.925505: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-05 10:05:13.925559: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
__________________ test_example_06_defining_own_architecture ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7faf9099f9a0>
@testbook(
REPO_ROOT / "examples/06-Define-your-own-architecture-with-Merlin-Models.ipynb", execute=False
)
def test_example_06_defining_own_architecture(tb):
tb.inject(
"""
from unittest.mock import patch
from merlin.datasets.synthetic import generate_data
mock_train, mock_valid = generate_data(
input="movielens-1m",
num_rows=1000,
set_sizes=(0.8, 0.2)
)
p1 = patch(
"merlin.datasets.entertainment.get_movielens",
return_value=[mock_train, mock_valid]
)
p1.start()
"""
)
tb.execute()
tests/unit/tf/examples/test_06_advanced_own_architecture.py:28:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7faf9099f9a0>
cell = {'cell_type': 'code', 'execution_count': 21, 'id': 'fb89d842', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...leExistsError\x1b[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'"]}], 'source': 'model.save("custom_dlrm")'}
cell_index = 61
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ec37a5e2-0785...e, 'engine': 'ec37a5e2-0785-421d-b421-a9956777aa88', 'started': '2022-10-05T10:06:22.988593Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("custom_dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [21], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mcustom_dlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:891[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 882[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 883[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 884[0m [38;5;124;03m"""[39;00m
E [1;32m 885[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 886[0m export_path,
E [1;32m 887[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 888[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 889[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 890[0m )
E [0;32m--> 891[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-05 10:06:14.316924: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-05 10:06:15.698467: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-05 10:06:15.698632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-05 10:06:15.699373: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-05 10:06:15.699431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 19 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 19 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:910: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileb01n3rot.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 2 failed, 732 passed, 12 skipped, 1137 warnings in 1125.72s (0:18:45) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins9230330769398840830.sh
Click to view CI Results
GitHub pull request #680 of commit f875c4aacd3f61c9243da41bd7646f7622cd3f79, no merge conflicts.
Running as SYSTEM
Setting status of f875c4aacd3f61c9243da41bd7646f7622cd3f79 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1476/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse f875c4aacd3f61c9243da41bd7646f7622cd3f79^{commit} # timeout=10
Checking out Revision f875c4aacd3f61c9243da41bd7646f7622cd3f79 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f f875c4aacd3f61c9243da41bd7646f7622cd3f79 # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk b385f6f3e019e7dcac6c91efa4468e69d318d3a7 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins15839137233641358601.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 750 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py F [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py F [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s..................... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
___________________ test_example_04_exporting_ranking_models ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7fb5b86dae50>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", execute=False)
def test_example_04_exporting_ranking_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
tb.execute()
tests/unit/tf/examples/test_04_export_ranking_models.py:17:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7fb5b86dae50>
cell = {'cell_type': 'code', 'execution_count': 14, 'id': 'f999a063', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':... "\x1b[0;31mFileExistsError\x1b[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'"]}], 'source': 'model.save("dlrm")'}
cell_index = 26
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'cf17bbf9-8b3b...e, 'engine': 'cf17bbf9-8b3b-4637-ad8e-a4cb890893cd', 'started': '2022-10-07T14:30:47.906352Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [14], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mdlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:893[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 884[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 885[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 886[0m [38;5;124;03m"""[39;00m
E [1;32m 887[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 888[0m export_path,
E [1;32m 889[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 890[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 891[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 892[0m )
E [0;32m--> 893[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-07 14:30:32.310602: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-07 14:30:34.420308: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-07 14:30:34.420485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-07 14:30:34.421164: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-07 14:30:34.421215: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
__________________ test_example_06_defining_own_architecture ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7fb5a3ce8d60>
@testbook(
REPO_ROOT / "examples/06-Define-your-own-architecture-with-Merlin-Models.ipynb", execute=False
)
def test_example_06_defining_own_architecture(tb):
tb.inject(
"""
from unittest.mock import patch
from merlin.datasets.synthetic import generate_data
mock_train, mock_valid = generate_data(
input="movielens-1m",
num_rows=1000,
set_sizes=(0.8, 0.2)
)
p1 = patch(
"merlin.datasets.entertainment.get_movielens",
return_value=[mock_train, mock_valid]
)
p1.start()
"""
)
tb.execute()
tests/unit/tf/examples/test_06_advanced_own_architecture.py:28:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7fb5a3ce8d60>
cell = {'cell_type': 'code', 'execution_count': 21, 'id': 'fb89d842', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...leExistsError\x1b[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'"]}], 'source': 'model.save("custom_dlrm")'}
cell_index = 61
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'e3208b0c-dd93...e, 'engine': 'e3208b0c-dd93-47d8-86a2-bb6dea441c27', 'started': '2022-10-07T14:31:44.358594Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("custom_dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [21], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mcustom_dlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:893[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 884[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 885[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 886[0m [38;5;124;03m"""[39;00m
E [1;32m 887[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 888[0m export_path,
E [1;32m 889[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 890[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 891[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 892[0m )
E [0;32m--> 893[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-07 14:31:35.619879: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-07 14:31:37.024419: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-07 14:31:37.024579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-07 14:31:37.025302: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-07 14:31:37.025357: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filewpv10twc.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 2 failed, 736 passed, 12 skipped, 1158 warnings in 1205.02s (0:20:05) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins18149913997330376966.sh
Click to view CI Results
GitHub pull request #680 of commit e45f25bcc59d6365a69b161c8782549d12e496d3, no merge conflicts.
Running as SYSTEM
Setting status of e45f25bcc59d6365a69b161c8782549d12e496d3 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1483/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse e45f25bcc59d6365a69b161c8782549d12e496d3^{commit} # timeout=10
Checking out Revision e45f25bcc59d6365a69b161c8782549d12e496d3 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f e45f25bcc59d6365a69b161c8782549d12e496d3 # timeout=10
Commit message: "Check signatures in save and load Model test"
> git rev-list --no-walk a2c3054becb2f9702388dc5f7099e4d1958ab665 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins238362659723620508.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 750 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py F [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py F [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s..................... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
___________________ test_example_04_exporting_ranking_models ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f91312b5e20>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", execute=False)
def test_example_04_exporting_ranking_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
tb.execute()
tests/unit/tf/examples/test_04_export_ranking_models.py:17:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f91312b5e20>
cell = {'cell_type': 'code', 'execution_count': 14, 'id': 'f999a063', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':... "\x1b[0;31mFileExistsError\x1b[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'"]}], 'source': 'model.save("dlrm")'}
cell_index = 26
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': '073b2a55-14f8...e, 'engine': '073b2a55-14f8-48ef-b05a-d7a694447a46', 'started': '2022-10-07T21:02:45.234909Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [14], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mdlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:893[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 884[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 885[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 886[0m [38;5;124;03m"""[39;00m
E [1;32m 887[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 888[0m export_path,
E [1;32m 889[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 890[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 891[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 892[0m )
E [0;32m--> 893[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m _MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-07 21:02:29.692479: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-07 21:02:31.778643: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-07 21:02:31.778814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-07 21:02:31.779625: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-07 21:02:31.779682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
__________________ test_example_06_defining_own_architecture ___________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f91308efd30>
@testbook(
REPO_ROOT / "examples/06-Define-your-own-architecture-with-Merlin-Models.ipynb", execute=False
)
def test_example_06_defining_own_architecture(tb):
tb.inject(
"""
from unittest.mock import patch
from merlin.datasets.synthetic import generate_data
mock_train, mock_valid = generate_data(
input="movielens-1m",
num_rows=1000,
set_sizes=(0.8, 0.2)
)
p1 = patch(
"merlin.datasets.entertainment.get_movielens",
return_value=[mock_train, mock_valid]
)
p1.start()
"""
)
tb.execute()
tests/unit/tf/examples/test_06_advanced_own_architecture.py:28:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f91308efd30>
cell = {'cell_type': 'code', 'execution_count': 21, 'id': 'fb89d842', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...leExistsError\x1b[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'"]}], 'source': 'model.save("custom_dlrm")'}
cell_index = 61
exec_reply = {'buffers': [], 'content': {'ename': 'FileExistsError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'cb7e67f3-2cd1...e, 'engine': 'cb7e67f3-2cd1-4e0a-b832-bffd2bdb74a9', 'started': '2022-10-07T21:03:41.953106Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model.save("custom_dlrm")
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mFileExistsError[0m Traceback (most recent call last)
E Cell [0;32mIn [21], line 1[0m
E [0;32m----> 1[0m [43mmodel[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mcustom_dlrm[39;49m[38;5;124;43m"[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:893[0m, in [0;36mModel.save[0;34m(self, export_path, include_optimizer, save_traces)[0m
E [1;32m 884[0m [38;5;124;03m"""Saves the model to export_path as a Tensorflow Saved Model.[39;00m
E [1;32m 885[0m [38;5;124;03mAlong with merlin model metadata.[39;00m
E [1;32m 886[0m [38;5;124;03m"""[39;00m
E [1;32m 887[0m [38;5;28msuper[39m()[38;5;241m.[39msave(
E [1;32m 888[0m export_path,
E [1;32m 889[0m include_optimizer[38;5;241m=[39minclude_optimizer,
E [1;32m 890[0m save_traces[38;5;241m=[39msave_traces,
E [1;32m 891[0m save_format[38;5;241m=[39m[38;5;124m"[39m[38;5;124mtf[39m[38;5;124m"[39m,
E [1;32m 892[0m )
E [0;32m--> 893[0m [43msave_merlin_metadata[49m[43m([49m[43mexport_path[49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/io.py:39[0m, in [0;36msave_merlin_metadata[0;34m(export_path, model, input_schema, output_schema)[0m
E [1;32m 37[0m export_path [38;5;241m=[39m pathlib[38;5;241m.[39mPath(export_path)
E [1;32m 38[0m merlin_metadata_dir [38;5;241m=[39m export_path [38;5;241m/[39m _MERLIN_METADATA_DIR_NAME
E [0;32m---> 39[0m [43mmerlin_metadata_dir[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[43m)[49m
E [1;32m 40[0m model_metadata [38;5;241m=[39m [38;5;28mdict[39m(
E [1;32m 41[0m model_module_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__module__[39m,
E [1;32m 42[0m model_class_name[38;5;241m=[39mmodel[38;5;241m.[39m[38;5;18m__class__[39m[38;5;241m.[39m[38;5;18m__name__[39m,
E [1;32m 43[0m )
E [1;32m 44[0m [38;5;28;01mwith[39;00m [38;5;28mopen[39m(merlin_metadata_dir [38;5;241m/[39m [38;5;124m"[39m[38;5;124mmodel.json[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mw[39m[38;5;124m"[39m) [38;5;28;01mas[39;00m f:
E
E File [0;32m/usr/lib/python3.8/pathlib.py:1288[0m, in [0;36mPath.mkdir[0;34m(self, mode, parents, exist_ok)[0m
E [1;32m 1286[0m [38;5;28mself[39m[38;5;241m.[39m_raise_closed()
E [1;32m 1287[0m [38;5;28;01mtry[39;00m:
E [0;32m-> 1288[0m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_accessor[49m[38;5;241;43m.[39;49m[43mmkdir[49m[43m([49m[38;5;28;43mself[39;49m[43m,[49m[43m [49m[43mmode[49m[43m)[49m
E [1;32m 1289[0m [38;5;28;01mexcept[39;00m [38;5;167;01mFileNotFoundError[39;00m:
E [1;32m 1290[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m parents [38;5;129;01mor[39;00m [38;5;28mself[39m[38;5;241m.[39mparent [38;5;241m==[39m [38;5;28mself[39m:
E
E [0;31mFileExistsError[0m: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
E FileExistsError: [Errno 17] File exists: 'custom_dlrm/merlin_metadata'
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-07 21:03:33.136859: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-07 21:03:34.536566: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-07 21:03:34.536735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-07 21:03:34.537441: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-07 21:03:34.537499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filefknvodrd.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 2 failed, 736 passed, 12 skipped, 1158 warnings in 1220.56s (0:20:20) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins15589381945779919199.sh
Click to view CI Results
GitHub pull request #680 of commit 6b875a6116a8c9fce6f835129a678f73e72ba65f, no merge conflicts.
Running as SYSTEM
Setting status of 6b875a6116a8c9fce6f835129a678f73e72ba65f to PENDING with url https://10.20.13.93:8080/job/merlin_models/1500/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 6b875a6116a8c9fce6f835129a678f73e72ba65f^{commit} # timeout=10
Checking out Revision 6b875a6116a8c9fce6f835129a678f73e72ba65f (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6b875a6116a8c9fce6f835129a678f73e72ba65f # timeout=10
Commit message: "Remove targets before passing to input block in test_save_and_load"
> git rev-list --no-walk ea0f19a86188c899317609ff406be9937ab0118c # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins12911897325223396684.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 751 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
.................FF.. [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s........F........... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py F [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py F [ 31%]
tests/unit/tf/inputs/test_base.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py .......................FF........ [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .......FFFF....... [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s.............F....... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py ..........FFFFFFFF......FFFFFF.... [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ...........FF. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
__________________ test_lazy_adam_for_large_embeddings[True] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa1e018c10>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_lazy_adam_for_large_embeddings[False] __________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa30edf7f0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________________ test_parallel_block_select_by_tags ______________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1d693940>
def test_parallel_block_select_by_tags(music_streaming_data):
continuous_block = mm.Filter(Tags.CONTINUOUS)
embedding_block = mm.Embeddings(
schema=music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL)
)
branches = {"continuous": continuous_block, "embeddings": embedding_block}
parallel_block = mm.ParallelBlock(branches, schema=music_streaming_data.schema)
continuous_inputs = parallel_block.select_by_tag(Tags.CONTINUOUS)
assert isinstance(continuous_inputs, mm.ParallelBlock)
assert sorted(continuous_inputs.schema.column_names) == [
"item_recency",
"position",
"user_age",
]
categorical_inputs = parallel_block.select_by_tag(Tags.CATEGORICAL)
assert sorted(categorical_inputs.schema.column_names) == [
"country",
"item_category",
"item_genres",
"item_id",
"session_id",
"user_genres",
"user_id",
]
# Forward pass a batch and compare the input features to output features
batch = mm.sample_batch(music_streaming_data, batch_size=10, include_targets=False)
continuous_outputs = continuous_inputs(batch)
assert sorted(continuous_outputs.keys()) == sorted(continuous_inputs.schema.column_names)
categorical_outputs = categorical_inputs(batch)
assert sorted(categorical_outputs.keys()) == sorted(categorical_inputs.schema.column_names)
# There are no sequence or time features in the dataset.
assert parallel_block.select_by_tag([Tags.SEQUENCE, Tags.TIME]) is None
# InputBlock is also a ParallelBlock.
input_block = mm.InputBlockV2(music_streaming_data.schema)
tests/unit/tf/core/test_combinators.py:140:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_example_03_exploring_different_models __________________
self = <testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:
args = (<testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>, {'cell_type': 'code', 'execution_count': 24, 'id':...cks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}, 55)
kwargs = {}
def wrapped(*args, **kwargs):
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85:
coro = <coroutine object NotebookClient.async_execute_cell at 0x7efa60b25240>
def just_run(coro: Awaitable) -> Any:
"""Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop is None:
had_running_loop = False
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
else:
had_running_loop = True
if had_running_loop:
# if there is a running loop, we patch using nest_asyncio
# to have reentrant event loops
check_ipython()
import nest_asyncio
nest_asyncio.apply()
check_patch_tornado()
return loop.run_until_complete(coro)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60:
self = <_UnixSelectorEventLoop running=False closed=False debug=False>
future = <Task finished name='Task-351' coro=<NotebookClient.async_execute_cell() done, defined at /usr/local/lib/python3.8/dis...should not contain any target features. Target columns can be removed withscheme.remove_by_tag(Tags.TARGET)`. \n')>
def run_until_complete(self, future):
"""Run until the Future is done.
If the argument is a coroutine, it is wrapped in a Task.
WARNING: It would be disastrous to call run_until_complete()
with the same coroutine twice -- it would wrap it in two
different Tasks and that can't be good.
Return the Future's result, or raise its exception.
"""
self._check_closed()
self._check_running()
new_task = not futures.isfuture(future)
future = tasks.ensure_future(future, loop=self)
if new_task:
# An exception is raised if the future didn't complete, so there
# is no need to log the "destroy pending task" message
future._log_destroy_pending = False
future.add_done_callback(_run_until_complete_cb)
try:
self.run_forever()
except:
if new_task and future.done() and not future.cancelled():
# The coroutine raised a BaseException. Consume the exception
# to not log a warning, the caller doesn't have access to the
# local task.
future.exception()
raise
finally:
future.remove_done_callback(_run_until_complete_cb)
if not future.done():
raise RuntimeError('Event loop stopped before Future completed.')
return future.result()
/usr/lib/python3.8/asyncio/base_events.py:616:
self = <testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
exec_reply = await self.task_poll_for_reply
except asyncio.CancelledError:
# can only be cancelled by task_poll_kernel_alive when the kernel is dead
task_poll_output_msg.cancel()
raise DeadKernelError("Kernel died")
except Exception as e:
# Best effort to cancel request if it hasn't been resolved
try:
# Check if the task_poll_output is doing the raising for us
if not isinstance(e, CellControlSignal):
task_poll_output_msg.cancel()
finally:
raise
if execution_count:
cell['execution_count'] = execution_count
await run_hook(
self.on_cell_executed, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
await self._check_raise_for_error(cell, cell_index, exec_reply)
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025:
self = <testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': '67f14918-40e4-4292...e, 'engine': '67f14918-40e4-4292-80f7-d685e8a756b9', 'started': '2022-10-10T15:21:24.977978Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>
@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", execute=False)
def test_example_03_exploring_different_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
NUM_OF_CELLS = len(tb.cells)
tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))
tests/unit/tf/examples/test_03_exploring_different_models.py:18:
self = <testbook.client.TestbookNotebookClient object at 0x7efa60b7c1c0>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
except CellExecutionError as ce:
raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))
E testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError
----------------------------- Captured stderr call -----------------------------
2022-10-10 15:21:11.268478: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:21:13.366363: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 15:21:13.366537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 15:21:13.367256: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 15:21:13.367315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
______________________ test_usecase_pretrained_embeddings ______________________
tb = <testbook.client.TestbookNotebookClient object at 0x7efa60f93820>
@testbook(
REPO_ROOT / "examples/usecases/entertainment-with-pretrained-embeddings.ipynb", execute=False
)
def test_usecase_pretrained_embeddings(tb):
tb.execute()
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py:10:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7efa60f93820>
cell = {'cell_type': 'code', 'execution_count': 11, 'id': '4d8df690-5d0d-413a-b5dc-7124a10378d2', 'metadata': {'execution': {...movieId': False},\n dim=embed_dims,\n)\ninput_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)'}
cell_index = 17
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'b9de170d-df87-4a6a...e, 'engine': 'b9de170d-df87-4a6a-8226-06f3b52edd0e', 'started': '2022-10-10T15:24:20.392511Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E embed_dims = {}
E embed_dims["movieId"] = pretrained_movie_embs.shape[1]
E
E embeddings_init={
E "movieId": mm.TensorInitializer(pretrained_movie_embs),
E }
E
E embeddings_block = mm.Embeddings(
E train.schema.select_by_tag(Tags.CATEGORICAL),
E infer_embedding_sizes=True,
E embeddings_initializer=embeddings_init,
E trainable={'movieId': False},
E dim=embed_dims,
E )
E input_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [11], line 15[0m
E [1;32m 4[0m embeddings_init[38;5;241m=[39m{
E [1;32m 5[0m [38;5;124m"[39m[38;5;124mmovieId[39m[38;5;124m"[39m: mm[38;5;241m.[39mTensorInitializer(pretrained_movie_embs),
E [1;32m 6[0m }
E [1;32m 8[0m embeddings_block [38;5;241m=[39m mm[38;5;241m.[39mEmbeddings(
E [1;32m 9[0m train[38;5;241m.[39mschema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mCATEGORICAL),
E [1;32m 10[0m infer_embedding_sizes[38;5;241m=[39m[38;5;28;01mTrue[39;00m,
E [0;32m (...)[0m
E [1;32m 13[0m dim[38;5;241m=[39membed_dims,
E [1;32m 14[0m )
E [0;32m---> 15[0m input_block [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mInputBlockV2[49m[43m([49m[43mtrain[49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[43mcategorical[49m[38;5;241;43m=[39;49m[43membeddings_block[49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-10 15:24:17.513408: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:24:19.591172: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 15:24:19.591319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 15:24:19.592154: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 15:24:19.592208: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
___________________ test_pretrained_from_InputBlockV2[True] ____________________
trainable = True
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0d1c8be0>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_pretrained_from_InputBlockV2[False] ___________________
trainable = False
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1ec529a0>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0fb3f670>
run_eagerly = True, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-False] ________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1f4f3f40>
run_eagerly = False, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________ test_tabular_features_yoochoose_model_inputblockv2[128-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa06502d30>
run_eagerly = True, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[128-False] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1c68bbe0>
run_eagerly = False, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________________ test_freeze_parallel_block[True] _______________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa0fad4f10>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True])
def test_freeze_parallel_block(ecommerce_data, run_eagerly):
# Train all parameters at first then freeze some layers
test_case = TestCase()
schema = ecommerce_data.schema.select_by_name(
names=["user_categories", "item_category", "click"]
)
input_block = mm.InputBlockV2(schema)
tests/unit/tf/models/test_base.py:321:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-True] ___________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa63139610>
stacked = True, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0fadf0d0>
stacked = False, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[False-True] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1d9df6a0>
stacked = True, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________________________ test_dcn_model[False-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1c0c3bb0>
stacked = False, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_deepfm_model_only_categ_feats[True] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0c22a970>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_deepfm_model_only_categ_feats[False] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0c4c9c40>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_deepfm_model_categ_and_continuous_feats[True] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1ea546d0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_deepfm_model_categ_and_continuous_feats[False] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa285ba6d0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.ITEM: 'item'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM_ID: 'item_id'>, <Tags...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[True] ______________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa1ddedd30>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[False] _____________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa1eb99ee0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[True] ____________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa1ef7daf0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[False] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7efa1ecc1310>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________ test_wide_deep_embedding_custom_inputblock[True] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa1cc60700>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_wide_deep_embedding_custom_inputblock[False] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7efa0fddc130>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.CATEGORICAL: 'categ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[True] _________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7efa1c5b0dc0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.SEQUENCE: 'sequence'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: 'item_id'>, <Tags.C...et'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[False] ________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7efa0e613040>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.SEQUENCE: 'sequence'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: 'item_id'>, <Tags.C...et'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7efb390b35e0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filesz0acpp3.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 28 failed, 711 passed, 12 skipped, 1147 warnings in 1108.19s (0:18:28) ====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins1258327866649227019.sh
Click to view CI Results
GitHub pull request #680 of commit 6c5c49603714614a6b20bef5336ce8ba7d8dcf06, no merge conflicts.
Running as SYSTEM
Setting status of 6c5c49603714614a6b20bef5336ce8ba7d8dcf06 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1501/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 6c5c49603714614a6b20bef5336ce8ba7d8dcf06^{commit} # timeout=10
Checking out Revision 6c5c49603714614a6b20bef5336ce8ba7d8dcf06 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6c5c49603714614a6b20bef5336ce8ba7d8dcf06 # timeout=10
Commit message: "Add Optional typehint reflecting valid inputs to save_merlin_metadata"
> git rev-list --no-walk 6b875a6116a8c9fce6f835129a678f73e72ba65f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1530107333701760455.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 751 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
.................FF.. [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s........F........... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py F [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py F [ 31%]
tests/unit/tf/inputs/test_base.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py .......................FF........ [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .......FFFF....... [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s.............F....... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py ..........FFFFFFFF......FFFFFF.... [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ...........FF. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 92%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py ..................... [100%]
=================================== FAILURES ===================================
__________________ test_lazy_adam_for_large_embeddings[True] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f1443587940>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_lazy_adam_for_large_embeddings[False] __________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f14438db580>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________________ test_parallel_block_select_by_tags ______________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1452eccd00>
def test_parallel_block_select_by_tags(music_streaming_data):
continuous_block = mm.Filter(Tags.CONTINUOUS)
embedding_block = mm.Embeddings(
schema=music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL)
)
branches = {"continuous": continuous_block, "embeddings": embedding_block}
parallel_block = mm.ParallelBlock(branches, schema=music_streaming_data.schema)
continuous_inputs = parallel_block.select_by_tag(Tags.CONTINUOUS)
assert isinstance(continuous_inputs, mm.ParallelBlock)
assert sorted(continuous_inputs.schema.column_names) == [
"item_recency",
"position",
"user_age",
]
categorical_inputs = parallel_block.select_by_tag(Tags.CATEGORICAL)
assert sorted(categorical_inputs.schema.column_names) == [
"country",
"item_category",
"item_genres",
"item_id",
"session_id",
"user_genres",
"user_id",
]
# Forward pass a batch and compare the input features to output features
batch = mm.sample_batch(music_streaming_data, batch_size=10, include_targets=False)
continuous_outputs = continuous_inputs(batch)
assert sorted(continuous_outputs.keys()) == sorted(continuous_inputs.schema.column_names)
categorical_outputs = categorical_inputs(batch)
assert sorted(categorical_outputs.keys()) == sorted(categorical_inputs.schema.column_names)
# There are no sequence or time features in the dataset.
assert parallel_block.select_by_tag([Tags.SEQUENCE, Tags.TIME]) is None
# InputBlock is also a ParallelBlock.
input_block = mm.InputBlockV2(music_streaming_data.schema)
tests/unit/tf/core/test_combinators.py:140:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_example_03_exploring_different_models __________________
self = <testbook.client.TestbookNotebookClient object at 0x7f148ff62280>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:
args = (<testbook.client.TestbookNotebookClient object at 0x7f148ff62280>, {'cell_type': 'code', 'execution_count': 24, 'id':...cks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}, 55)
kwargs = {}
def wrapped(*args, **kwargs):
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85:
coro = <coroutine object NotebookClient.async_execute_cell at 0x7f1441b82140>
def just_run(coro: Awaitable) -> Any:
"""Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop is None:
had_running_loop = False
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
else:
had_running_loop = True
if had_running_loop:
# if there is a running loop, we patch using nest_asyncio
# to have reentrant event loops
check_ipython()
import nest_asyncio
nest_asyncio.apply()
check_patch_tornado()
return loop.run_until_complete(coro)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60:
self = <_UnixSelectorEventLoop running=False closed=False debug=False>
future = <Task finished name='Task-351' coro=<NotebookClient.async_execute_cell() done, defined at /usr/local/lib/python3.8/dis...should not contain any target features. Target columns can be removed withscheme.remove_by_tag(Tags.TARGET)`. \n')>
def run_until_complete(self, future):
"""Run until the Future is done.
If the argument is a coroutine, it is wrapped in a Task.
WARNING: It would be disastrous to call run_until_complete()
with the same coroutine twice -- it would wrap it in two
different Tasks and that can't be good.
Return the Future's result, or raise its exception.
"""
self._check_closed()
self._check_running()
new_task = not futures.isfuture(future)
future = tasks.ensure_future(future, loop=self)
if new_task:
# An exception is raised if the future didn't complete, so there
# is no need to log the "destroy pending task" message
future._log_destroy_pending = False
future.add_done_callback(_run_until_complete_cb)
try:
self.run_forever()
except:
if new_task and future.done() and not future.cancelled():
# The coroutine raised a BaseException. Consume the exception
# to not log a warning, the caller doesn't have access to the
# local task.
future.exception()
raise
finally:
future.remove_done_callback(_run_until_complete_cb)
if not future.done():
raise RuntimeError('Event loop stopped before Future completed.')
return future.result()
/usr/lib/python3.8/asyncio/base_events.py:616:
self = <testbook.client.TestbookNotebookClient object at 0x7f148ff62280>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
exec_reply = await self.task_poll_for_reply
except asyncio.CancelledError:
# can only be cancelled by task_poll_kernel_alive when the kernel is dead
task_poll_output_msg.cancel()
raise DeadKernelError("Kernel died")
except Exception as e:
# Best effort to cancel request if it hasn't been resolved
try:
# Check if the task_poll_output is doing the raising for us
if not isinstance(e, CellControlSignal):
task_poll_output_msg.cancel()
finally:
raise
if execution_count:
cell['execution_count'] = execution_count
await run_hook(
self.on_cell_executed, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
await self._check_raise_for_error(cell, cell_index, exec_reply)
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025:
self = <testbook.client.TestbookNotebookClient object at 0x7f148ff62280>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'a490ef6a-f9a0-4069...e, 'engine': 'a490ef6a-f9a0-4069-aee8-4506775318df', 'started': '2022-10-10T15:40:11.380552Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7f148ff62280>
@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", execute=False)
def test_example_03_exploring_different_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
NUM_OF_CELLS = len(tb.cells)
tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))
tests/unit/tf/examples/test_03_exploring_different_models.py:18:
self = <testbook.client.TestbookNotebookClient object at 0x7f148ff62280>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
except CellExecutionError as ce:
raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))
E testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError
----------------------------- Captured stderr call -----------------------------
2022-10-10 15:39:57.722411: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:39:59.820389: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 15:39:59.820558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 15:39:59.821282: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 15:39:59.821338: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
______________________ test_usecase_pretrained_embeddings ______________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f14940e28e0>
@testbook(
REPO_ROOT / "examples/usecases/entertainment-with-pretrained-embeddings.ipynb", execute=False
)
def test_usecase_pretrained_embeddings(tb):
tb.execute()
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py:10:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f14940e28e0>
cell = {'cell_type': 'code', 'execution_count': 11, 'id': '4d8df690-5d0d-413a-b5dc-7124a10378d2', 'metadata': {'execution': {...movieId': False},\n dim=embed_dims,\n)\ninput_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)'}
cell_index = 17
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'd4b7229b-17da-495d...e, 'engine': 'd4b7229b-17da-495d-8f16-ab7b20c5b512', 'started': '2022-10-10T15:43:04.801566Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E embed_dims = {}
E embed_dims["movieId"] = pretrained_movie_embs.shape[1]
E
E embeddings_init={
E "movieId": mm.TensorInitializer(pretrained_movie_embs),
E }
E
E embeddings_block = mm.Embeddings(
E train.schema.select_by_tag(Tags.CATEGORICAL),
E infer_embedding_sizes=True,
E embeddings_initializer=embeddings_init,
E trainable={'movieId': False},
E dim=embed_dims,
E )
E input_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [11], line 15[0m
E [1;32m 4[0m embeddings_init[38;5;241m=[39m{
E [1;32m 5[0m [38;5;124m"[39m[38;5;124mmovieId[39m[38;5;124m"[39m: mm[38;5;241m.[39mTensorInitializer(pretrained_movie_embs),
E [1;32m 6[0m }
E [1;32m 8[0m embeddings_block [38;5;241m=[39m mm[38;5;241m.[39mEmbeddings(
E [1;32m 9[0m train[38;5;241m.[39mschema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mCATEGORICAL),
E [1;32m 10[0m infer_embedding_sizes[38;5;241m=[39m[38;5;28;01mTrue[39;00m,
E [0;32m (...)[0m
E [1;32m 13[0m dim[38;5;241m=[39membed_dims,
E [1;32m 14[0m )
E [0;32m---> 15[0m input_block [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mInputBlockV2[49m[43m([49m[43mtrain[49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[43mcategorical[49m[38;5;241;43m=[39;49m[43membeddings_block[49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-10 15:43:01.874795: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:43:03.967660: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 15:43:03.967805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 15:43:03.968644: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 15:43:03.968701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
___________________ test_pretrained_from_InputBlockV2[True] ____________________
trainable = True
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f145ac4e130>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_pretrained_from_InputBlockV2[False] ___________________
trainable = False
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1443d037f0>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f14585156d0>
run_eagerly = True, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-False] ________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f144377e3d0>
run_eagerly = False, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________ test_tabular_features_yoochoose_model_inputblockv2[128-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1453ba8310>
run_eagerly = True, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[128-False] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1450b6b4f0>
run_eagerly = False, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________________ test_freeze_parallel_block[True] _______________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f14438d4eb0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True])
def test_freeze_parallel_block(ecommerce_data, run_eagerly):
# Train all parameters at first then freeze some layers
test_case = TestCase()
schema = ecommerce_data.schema.select_by_name(
names=["user_categories", "item_category", "click"]
)
input_block = mm.InputBlockV2(schema)
tests/unit/tf/models/test_base.py:321:
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-True] ___________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f14680c1490>
stacked = True, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f145251b250>
stacked = False, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[False-True] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f145a14f130>
stacked = True, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________________________ test_dcn_model[False-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f145940b520>
stacked = False, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_deepfm_model_only_categ_feats[True] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1450b83310>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_deepfm_model_only_categ_feats[False] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f14584367f0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_deepfm_model_categ_and_continuous_feats[True] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1459564430>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_deepfm_model_categ_and_continuous_feats[False] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1452f6ee20>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.ITEM_ID: ...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[True] ______________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f1443d08fa0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[False] _____________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f14523c9eb0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[True] ____________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f1458c82c40>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[False] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f1443b9ce50>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.USER: 'user'>, <Tags.CATEGORICAL: 'categorical'>}, 'properties': {'domain'...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ion'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________ test_wide_deep_embedding_custom_inputblock[True] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f145ad859d0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_wide_deep_embedding_custom_inputblock[False] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f1443b76f40>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.SESSION_ID: 'session_id'>, <Tags.SESSION: 'session'>, <Tags.ID: 'id'>, <Tags.CA...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.TARGET: 'target'>, <Tags.BINARY_CLASSIFICATION: 'binary_classification'>}, 'properti...ICATION: 'binary_classification'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[True] _________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f1451c6ad00>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.SEQUENCE: 'sequence'>, <Tags.CATEGORICAL...er'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.TARGET: 'target'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[False] ________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f14416bea90>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.ID: 'id'>, <Tags.ITEM: 'item'>, <Tags.SEQUENCE: 'sequence'>, <Tags.CATEGORICAL...er'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f1573e4bca0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.TARGET: 'target'>, <Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 19 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file7a6r7iq7.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 15 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 13 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 28 failed, 711 passed, 12 skipped, 1147 warnings in 1109.99s (0:18:29) ====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins5373409557900363035.sh
Click to view CI Results
GitHub pull request #680 of commit 27d165c092850efdb2a4b3658cc0b0016cf6c9f6, no merge conflicts.
Running as SYSTEM
Setting status of 27d165c092850efdb2a4b3658cc0b0016cf6c9f6 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1502/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 27d165c092850efdb2a4b3658cc0b0016cf6c9f6^{commit} # timeout=10
Checking out Revision 27d165c092850efdb2a4b3658cc0b0016cf6c9f6 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 27d165c092850efdb2a4b3658cc0b0016cf6c9f6 # timeout=10
Commit message: "Remove `load_model` function from this PR"
> git rev-list --no-walk 6c5c49603714614a6b20bef5336ce8ba7d8dcf06 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins9320044947243623889.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 749 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
.................FF.. [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s........F........... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py F [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py F [ 31%]
tests/unit/tf/inputs/test_base.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 32%]
tests/unit/tf/inputs/test_embedding.py .......................FF........ [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .......FFFF....... [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s.............F...... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py ..........FFFFFFFF......FFFFFF.... [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ...........FF. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
__________________ test_lazy_adam_for_large_embeddings[True] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c1f322280>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_lazy_adam_for_large_embeddings[False] __________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c1e6ce5e0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_lazy_adam_for_large_embeddings(ecommerce_data, run_eagerly):
schema = ecommerce_data.schema
embeddings = ml.Embeddings(schema.select_by_tag(Tags.CATEGORICAL))
large_embeddings, small_embeddings = ml.split_embeddings_on_size(embeddings, threshold=1000)
input = ml.InputBlockV2(schema, categorical=embeddings)
tests/unit/tf/blocks/test_optimizer.py:595:
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(user_categories): EmbeddingTable(
(features): Dict(
(user...n': 0, 'max': 4}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________________ test_parallel_block_select_by_tags ______________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1ec43070>
def test_parallel_block_select_by_tags(music_streaming_data):
continuous_block = mm.Filter(Tags.CONTINUOUS)
embedding_block = mm.Embeddings(
schema=music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL)
)
branches = {"continuous": continuous_block, "embeddings": embedding_block}
parallel_block = mm.ParallelBlock(branches, schema=music_streaming_data.schema)
continuous_inputs = parallel_block.select_by_tag(Tags.CONTINUOUS)
assert isinstance(continuous_inputs, mm.ParallelBlock)
assert sorted(continuous_inputs.schema.column_names) == [
"item_recency",
"position",
"user_age",
]
categorical_inputs = parallel_block.select_by_tag(Tags.CATEGORICAL)
assert sorted(categorical_inputs.schema.column_names) == [
"country",
"item_category",
"item_genres",
"item_id",
"session_id",
"user_genres",
"user_id",
]
# Forward pass a batch and compare the input features to output features
batch = mm.sample_batch(music_streaming_data, batch_size=10, include_targets=False)
continuous_outputs = continuous_inputs(batch)
assert sorted(continuous_outputs.keys()) == sorted(continuous_inputs.schema.column_names)
categorical_outputs = categorical_inputs(batch)
assert sorted(categorical_outputs.keys()) == sorted(categorical_inputs.schema.column_names)
# There are no sequence or time features in the dataset.
assert parallel_block.select_by_tag([Tags.SEQUENCE, Tags.TIME]) is None
# InputBlock is also a ParallelBlock.
input_block = mm.InputBlockV2(music_streaming_data.schema)
tests/unit/tf/core/test_combinators.py:140:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_example_03_exploring_different_models __________________
self = <testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:
args = (<testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>, {'cell_type': 'code', 'execution_count': 24, 'id':...cks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}, 55)
kwargs = {}
def wrapped(*args, **kwargs):
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85:
coro = <coroutine object NotebookClient.async_execute_cell at 0x7f0c1de045c0>
def just_run(coro: Awaitable) -> Any:
"""Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop is None:
had_running_loop = False
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
else:
had_running_loop = True
if had_running_loop:
# if there is a running loop, we patch using nest_asyncio
# to have reentrant event loops
check_ipython()
import nest_asyncio
nest_asyncio.apply()
check_patch_tornado()
return loop.run_until_complete(coro)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60:
self = <_UnixSelectorEventLoop running=False closed=False debug=False>
future = <Task finished name='Task-351' coro=<NotebookClient.async_execute_cell() done, defined at /usr/local/lib/python3.8/dis...should not contain any target features. Target columns can be removed withscheme.remove_by_tag(Tags.TARGET)`. \n')>
def run_until_complete(self, future):
"""Run until the Future is done.
If the argument is a coroutine, it is wrapped in a Task.
WARNING: It would be disastrous to call run_until_complete()
with the same coroutine twice -- it would wrap it in two
different Tasks and that can't be good.
Return the Future's result, or raise its exception.
"""
self._check_closed()
self._check_running()
new_task = not futures.isfuture(future)
future = tasks.ensure_future(future, loop=self)
if new_task:
# An exception is raised if the future didn't complete, so there
# is no need to log the "destroy pending task" message
future._log_destroy_pending = False
future.add_done_callback(_run_until_complete_cb)
try:
self.run_forever()
except:
if new_task and future.done() and not future.cancelled():
# The coroutine raised a BaseException. Consume the exception
# to not log a warning, the caller doesn't have access to the
# local task.
future.exception()
raise
finally:
future.remove_done_callback(_run_until_complete_cb)
if not future.done():
raise RuntimeError('Event loop stopped before Future completed.')
return future.result()
/usr/lib/python3.8/asyncio/base_events.py:616:
self = <testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
exec_reply = await self.task_poll_for_reply
except asyncio.CancelledError:
# can only be cancelled by task_poll_kernel_alive when the kernel is dead
task_poll_output_msg.cancel()
raise DeadKernelError("Kernel died")
except Exception as e:
# Best effort to cancel request if it hasn't been resolved
try:
# Check if the task_poll_output is doing the raising for us
if not isinstance(e, CellControlSignal):
task_poll_output_msg.cancel()
finally:
raise
if execution_count:
cell['execution_count'] = execution_count
await run_hook(
self.on_cell_executed, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
await self._check_raise_for_error(cell, cell_index, exec_reply)
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025:
self = <testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>
cell = {'cell_type': 'code', 'execution_count': 24, 'id': 'c4700682', 'metadata': {'execution': {'iopub.status.busy': '2022-1...g_blocks,\n deep_block=deep_part,\n prediction_tasks=mm.BinaryClassificationTask(target_column),\n )'}
cell_index = 55
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'd719875c-8346-4d04...e, 'engine': 'd719875c-8346-4d04-b042-e44afa2c42ca', 'started': '2022-10-10T15:58:57.314472Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>
@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", execute=False)
def test_example_03_exploring_different_models(tb):
tb.inject(
"""
import os
os.environ["DATA_FOLDER"] = "/tmp/data/"
os.environ["NUM_ROWS"] = "999"
"""
)
NUM_OF_CELLS = len(tb.cells)
tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))
tests/unit/tf/examples/test_03_exploring_different_models.py:18:
self = <testbook.client.TestbookNotebookClient object at 0x7f0c607c93a0>
cell = {'cell_type': 'markdown', 'id': '694da73f', 'metadata': {}, 'source': '#### Putting it all together'}
kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]
executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...d': '2dd02301', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': '## Feature Engineering with NVTabular'}, ...]
idx = 55
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
except CellExecutionError as ce:
raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))
E testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:
E ------------------
E model = mm.WideAndDeepModel(
E schema,
E wide_schema=cat_schema,
E deep_schema=schema,
E wide_preprocess=wide_preprocessing_blocks,
E deep_block=deep_part,
E prediction_tasks=mm.BinaryClassificationTask(target_column),
E )
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [24], line 1[0m
E [0;32m----> 1[0m model [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mWideAndDeepModel[49m[43m([49m
E [1;32m 2[0m [43m [49m[43mschema[49m[43m,[49m
E [1;32m 3[0m [43m [49m[43mwide_schema[49m[38;5;241;43m=[39;49m[43mcat_schema[49m[43m,[49m
E [1;32m 4[0m [43m [49m[43mdeep_schema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m,[49m
E [1;32m 5[0m [43m [49m[43mwide_preprocess[49m[38;5;241;43m=[39;49m[43mwide_preprocessing_blocks[49m[43m,[49m
E [1;32m 6[0m [43m [49m[43mdeep_block[49m[38;5;241;43m=[39;49m[43mdeep_part[49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mprediction_tasks[49m[38;5;241;43m=[39;49m[43mmm[49m[38;5;241;43m.[39;49m[43mBinaryClassificationTask[49m[43m([49m[43mtarget_column[49m[43m)[49m[43m,[49m
E [1;32m 8[0m [43m [49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/ranking.py:485[0m, in [0;36mWideAndDeepModel[0;34m(schema, deep_block, wide_schema, deep_schema, wide_preprocess, deep_input_block, wide_input_block, deep_regularizer, wide_regularizer, deep_dropout, wide_dropout, prediction_tasks, **wide_body_kwargs)[0m
E [1;32m 483[0m [38;5;28;01mif[39;00m [38;5;129;01mnot[39;00m deep_input_block:
E [1;32m 484[0m [38;5;28;01mif[39;00m deep_schema [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m [38;5;129;01mand[39;00m [38;5;28mlen[39m(deep_schema) [38;5;241m>[39m [38;5;241m0[39m:
E [0;32m--> 485[0m deep_input_block [38;5;241m=[39m [43mInputBlockV2[49m[43m([49m[43mdeep_schema[49m[43m)[49m
E [1;32m 486[0m [38;5;28;01mif[39;00m deep_input_block:
E [1;32m 487[0m deep_body [38;5;241m=[39m deep_input_block[38;5;241m.[39mconnect(deep_block)[38;5;241m.[39mconnect(
E [1;32m 488[0m MLPBlock(
E [1;32m 489[0m [[38;5;241m1[39m],
E [0;32m (...)[0m
E [1;32m 494[0m )
E [1;32m 495[0m )
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError
----------------------------- Captured stderr call -----------------------------
2022-10-10 15:58:43.654687: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:58:45.743874: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 15:58:45.744053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 15:58:45.744834: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 15:58:45.744889: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
______________________ test_usecase_pretrained_embeddings ______________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f0c608c7a00>
@testbook(
REPO_ROOT / "examples/usecases/entertainment-with-pretrained-embeddings.ipynb", execute=False
)
def test_usecase_pretrained_embeddings(tb):
tb.execute()
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py:10:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f0c608c7a00>
cell = {'cell_type': 'code', 'execution_count': 11, 'id': '4d8df690-5d0d-413a-b5dc-7124a10378d2', 'metadata': {'execution': {...movieId': False},\n dim=embed_dims,\n)\ninput_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)'}
cell_index = 17
exec_reply = {'buffers': [], 'content': {'ename': 'ValueError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'b5d6cc76-9bfc-48e0...e, 'engine': 'b5d6cc76-9bfc-48e0-a4a8-36ff4e8675d8', 'started': '2022-10-10T16:01:51.712783Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E embed_dims = {}
E embed_dims["movieId"] = pretrained_movie_embs.shape[1]
E
E embeddings_init={
E "movieId": mm.TensorInitializer(pretrained_movie_embs),
E }
E
E embeddings_block = mm.Embeddings(
E train.schema.select_by_tag(Tags.CATEGORICAL),
E infer_embedding_sizes=True,
E embeddings_initializer=embeddings_init,
E trainable={'movieId': False},
E dim=embed_dims,
E )
E input_block = mm.InputBlockV2(train.schema, categorical=embeddings_block)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mValueError[0m Traceback (most recent call last)
E Cell [0;32mIn [11], line 15[0m
E [1;32m 4[0m embeddings_init[38;5;241m=[39m{
E [1;32m 5[0m [38;5;124m"[39m[38;5;124mmovieId[39m[38;5;124m"[39m: mm[38;5;241m.[39mTensorInitializer(pretrained_movie_embs),
E [1;32m 6[0m }
E [1;32m 8[0m embeddings_block [38;5;241m=[39m mm[38;5;241m.[39mEmbeddings(
E [1;32m 9[0m train[38;5;241m.[39mschema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mCATEGORICAL),
E [1;32m 10[0m infer_embedding_sizes[38;5;241m=[39m[38;5;28;01mTrue[39;00m,
E [0;32m (...)[0m
E [1;32m 13[0m dim[38;5;241m=[39membed_dims,
E [1;32m 14[0m )
E [0;32m---> 15[0m input_block [38;5;241m=[39m [43mmm[49m[38;5;241;43m.[39;49m[43mInputBlockV2[49m[43m([49m[43mtrain[49m[38;5;241;43m.[39;49m[43mschema[49m[43m,[49m[43m [49m[43mcategorical[49m[38;5;241;43m=[39;49m[43membeddings_block[49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/inputs/base.py:295[0m, in [0;36mInputBlockV2[0;34m(schema, categorical, continuous, pre, post, aggregation, tag_to_block, **branches)[0m
E [1;32m 293[0m target_schema [38;5;241m=[39m schema[38;5;241m.[39mselect_by_tag(Tags[38;5;241m.[39mTARGET)
E [1;32m 294[0m [38;5;28;01mif[39;00m target_schema[38;5;241m.[39mcolumn_names:
E [0;32m--> 295[0m [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
E [1;32m 296[0m [38;5;124m"[39m[38;5;124mschema should not contain any target features. [39m[38;5;124m"[39m
E [1;32m 297[0m [38;5;124m"[39m[38;5;124mTarget columns can be removed with scheme.remove_by_tag(Tags.TARGET). [39m[38;5;124m"[39m
E [1;32m 298[0m )
E [1;32m 300[0m unparsed [38;5;241m=[39m {[38;5;124m"[39m[38;5;124mcategorical[39m[38;5;124m"[39m: categorical, [38;5;124m"[39m[38;5;124mcontinuous[39m[38;5;124m"[39m: continuous, [38;5;241m[39m[38;5;241m[39mbranches}
E [1;32m 301[0m parsed [38;5;241m=[39m {}
E
E [0;31mValueError[0m: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-10 16:01:48.821250: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 16:01:50.916981: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-10 16:01:50.917133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-10 16:01:50.917940: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-10 16:01:50.917994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14500 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
___________________ test_pretrained_from_InputBlockV2[True] ____________________
trainable = True
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c16288dc0>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_pretrained_from_InputBlockV2[False] ___________________
trainable = False
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1e492400>
@pytest.mark.parametrize("trainable", [True, False])
def test_pretrained_from_InputBlockV2(trainable, music_streaming_data: Dataset):
vocab_size = music_streaming_data.schema.column_schemas["item_id"].int_domain.max + 1
embedding_dim = 32
weights = np.random.rand(vocab_size, embedding_dim)
pre_trained_weights_df = pd.DataFrame(weights)
embed_dims = {}
embed_dims["item_id"] = pre_trained_weights_df.shape[1]
embeddings_init = {
"item_id": mm.TensorInitializer(weights),
}
embeddings_block = mm.Embeddings(
music_streaming_data.schema.select_by_tag(Tags.CATEGORICAL),
embeddings_initializer=embeddings_init,
trainable={"item_id": trainable},
dim=embed_dims,
)
input_block = mm.InputBlockV2(music_streaming_data.schema, categorical=embeddings_block)
tests/unit/tf/inputs/test_embedding.py:339:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c14537160>
run_eagerly = True, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[None-False] ________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c16fa5a00>
run_eagerly = False, continuous_projection = None
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________ test_tabular_features_yoochoose_model_inputblockv2[128-True] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c16f18df0>
run_eagerly = True, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________ test_tabular_features_yoochoose_model_inputblockv2[128-False] _________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c16fd92b0>
run_eagerly = False, continuous_projection = 128
@testing_utils.mark_run_eagerly_modes
@pytest.mark.parametrize("continuous_projection", [None, 128])
def test_tabular_features_yoochoose_model_inputblockv2(
music_streaming_data: Dataset, run_eagerly, continuous_projection
):
kwargs = {}
if continuous_projection:
kwargs["continuous"] = ml.ContinuousProjection(
music_streaming_data.schema.select_by_tag(Tags.CONTINUOUS),
ml.MLPBlock([continuous_projection]),
)
inputs = ml.InputBlockV2(music_streaming_data.schema, aggregation="concat", **kwargs)
tests/unit/tf/inputs/test_tabular.py:88:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = SequentialBlock(
(layers): List(
(0): Continuous(
(feature_names): List(
(0): 'item_recency'
... List(
(0): _Dense(
(dense): Dense(128, activation=relu, use_bias=True)
)
)
)
)
)
pre = None, post = None, aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________________ test_freeze_parallel_block[True] _______________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c1659b460>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True])
def test_freeze_parallel_block(ecommerce_data, run_eagerly):
# Train all parameters at first then freeze some layers
test_case = TestCase()
schema = ecommerce_data.schema.select_by_name(
names=["user_categories", "item_category", "click"]
)
input_block = mm.InputBlockV2(schema)
tests/unit/tf/models/test_base.py:320:
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-True] ___________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1e189700>
stacked = True, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[True-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c0fdd3640>
stacked = False, run_eagerly = True
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________________ test_dcn_model[False-True] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1cd6e610>
stacked = True, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_________________________ test_dcn_model[False-False] __________________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c0f87e1c0>
stacked = False, run_eagerly = False
@pytest.mark.parametrize("stacked", [True, False])
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_dcn_model(music_streaming_data, stacked, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "user_age", "click"]
)
model = mm.DCNModel(
music_streaming_data.schema,
depth=1,
deep_block=mm.MLPBlock([2]),
stacked=stacked,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:148:
merlin/models/tf/models/ranking.py:140: in DCNModel
input_block = input_block or InputBlockV2(schema, **kwargs)
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_deepfm_model_only_categ_feats[True] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c14e95640>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
__________________ test_deepfm_model_only_categ_feats[False] ___________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1595f430>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_only_categ_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:164:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_deepfm_model_categ_and_continuous_feats[True] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1dd150a0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_deepfm_model_categ_and_continuous_feats[False] ______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c14caf400>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_deepfm_model_categ_and_continuous_feats(music_streaming_data, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(
["item_id", "item_category", "user_id", "user_age", "click"]
)
model = mm.DeepFMModel(
music_streaming_data.schema,
embedding_dim=16,
deep_block=mm.MLPBlock([16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:179:
merlin/models/tf/models/ranking.py:221: in DeepFMModel
input_block = input_block or InputBlockV2(
schema = [{'name': 'item_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.ID: 'id'>, <Tags.ITEM_ID: ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(item_id): EmbeddingTable(
(features): Dict(
(item_id): Co...0, 'max': 10000}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = None
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[True] ______________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c1eb1db50>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_____________ test_wide_deep_model_wide_categorical_one_hot[False] _____________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c0e9cb610>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_wide_categorical_one_hot(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.CategoryEncoding(wide_schema, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:242:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[True] ____________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c155c62e0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
___________________ test_wide_deep_model_hashed_cross[False] ___________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f0c159dcaf0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_model_hashed_cross(ecommerce_data, run_eagerly):
wide_schema = ecommerce_data.schema.select_by_name(names=["user_categories", "item_category"])
deep_schema = ecommerce_data.schema
model = mm.WideAndDeepModel(
ecommerce_data.schema,
wide_schema=wide_schema,
deep_schema=deep_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:260:
merlin/models/tf/models/ranking.py:485: in WideAndDeepModel
deep_input_block = InputBlockV2(deep_schema)
schema = [{'name': 'user_categories', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>}, 'properties': {'domain'...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...get'>}, 'properties': {'domain': {'min': 0, 'max': 0}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
_______________ test_wide_deep_embedding_custom_inputblock[True] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1d087640>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
______________ test_wide_deep_embedding_custom_inputblock[False] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f0c1e5a4d90>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_wide_deep_embedding_custom_inputblock(music_streaming_data, run_eagerly):
schema = music_streaming_data.schema
# prepare wide_schema
wide_schema = schema.select_by_name(["country", "user_age"])
deep_embedding = mm.Embeddings(schema.select_by_tag(Tags.CATEGORICAL), dim=16)
model = mm.WideAndDeepModel(
schema,
deep_input_block=mm.InputBlockV2(schema=schema, categorical=deep_embedding),
wide_schema=wide_schema,
wide_preprocess=mm.HashedCross(wide_schema, 1000, sparse=True),
deep_block=mm.MLPBlock([32, 16]),
deep_regularizer=regularizers.l2(1e-5),
wide_regularizer=regularizers.l2(1e-5),
deep_dropout=0.1,
wide_dropout=0.2,
prediction_tasks=mm.BinaryClassificationTask("click"),
)
tests/unit/tf/models/test_ranking.py:282:
schema = [{'name': 'session_id', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ID: 'id'>, <Tags.SESSION_ID: 'session_id'>, ...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
categorical = ParallelBlock(
(parallel_layers): Dict(
(session_id): EmbeddingTable(
(features): Dict(
(session_i...: 0, 'max': 100}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {}
target_schema = [{'name': 'click', 'tags': {<Tags.BINARY_CLASSIFICATION: 'binary_classification'>, <Tags.TARGET: 'target'>}, 'properti...cation'>, <Tags.TARGET: 'target'>}, 'properties': {}, 'dtype': dtype('float64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[True] _________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f0c06308220>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.SEQUENCE: 'sequence'>,...et'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
________________ test_transformer_as_classfication_model[False] ________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f0bff33c220>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_transformer_as_classfication_model(sequence_testing_data: Dataset, run_eagerly):
EMBED_DIM = 48
loader, schema = classification_loader(sequence_testing_data)
model = mm.Model(
mm.InputBlockV2(
schema,
embeddings=mm.Embeddings(schema, sequence_combiner=None),
),
BertBlock(
d_model=EMBED_DIM,
n_head=8,
n_layer=2,
transformer_post="pooler_output",
),
mm.CategoricalOutput(
to_call=schema["user_country"],
),
)
tests/unit/tf/transformers/test_block.py:92:
schema = [{'name': 'item_id_seq', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.ITEM: 'item'>, <Tags.SEQUENCE: 'sequence'>,...et'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
categorical = <Tags.CATEGORICAL: 'categorical'>
continuous = <Tags.CONTINUOUS: 'continuous'>, pre = None, post = None
aggregation = 'concat'
tag_to_block = {<Tags.CONTINUOUS: 'continuous'>: <class 'merlin.models.tf.inputs.continuous.Continuous'>, <Tags.CATEGORICAL: 'categorical'>: <function Embeddings at 0x7f0d382f1ca0>}
branches = {'embeddings': ParallelBlock(
(parallel_layers): Dict(
(item_id_seq): EmbeddingTable(
(features): Dict(
...: 1, 'max': 62}}, dtype=dtype('int64'), is_list=False, is_ragged=False)
)
(table): Embedding()
)
)
)}
target_schema = [{'name': 'user_country', 'tags': {<Tags.CATEGORICAL: 'categorical'>, <Tags.USER: 'user'>, <Tags.TARGET: 'target'>}, 'properties': {'domain': {'min': 1, 'max': 62}}, 'dtype': dtype('int64'), 'is_list': False, 'is_ragged': False}]
def InputBlockV2(
schema: Optional[Schema] = None,
categorical: Union[Tags, Layer] = Tags.CATEGORICAL,
continuous: Union[Tags, Layer] = Tags.CONTINUOUS,
pre: Optional[BlockType] = None,
post: Optional[BlockType] = None,
aggregation: Optional[TabularAggregationType] = "concat",
tag_to_block=INPUT_TAG_TO_BLOCK,
**branches,
) -> ParallelBlock:
"""The entry block of the model to process input features from a schema.
This is a new version of InputBlock, which is more flexible for accepting
the external definition of `embeddings` block. After `22.10` this will become the default.
Simple Usage::
inputs = InputBlockV2(schema)
Custom Embeddings::
inputs = InputBlockV2(
schema,
categorical=Embeddings(schema, dim=32)
)
Sparse outputs for one-hot::
inputs = InputBlockV2(
schema,
categorical=CategoryEncoding(schema, sparse=True),
post=ToSparse()
)
Add continuous projection::
inputs = InputBlockV2(
schema,
continuous=ContinuousProjection(continuous_schema, MLPBlock([32])),
)
Merge 2D and 3D (for session-based)::
inputs = InputBlockV2(
schema,
post=BroadcastToSequence(context_schema, sequence_schema)
)
Parameters
----------
schema : Schema
Schema of the input data. This Schema object will be automatically generated using
[NVTabular](https://nvidia-merlin.github.io/NVTabular/main/Introduction.html).
Next to this, it's also possible to construct it manually.
categorical : Union[Tags, Layer], defaults to `Tags.CATEGORICAL`
A block or column-selector to use for categorical-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Embeddings` to infer the embedding tables from the column-selector.
continuous : Union[Tags, Layer], defaults to `Tags.CONTINUOUS`
A block to use for continuous-features.
If a column-selector is provided (either a schema or tags), the selector
will be passed to `Continuous` to infer the features from the column-selector.
pre : Optional[BlockType], optional
Transformation block to apply before the embeddings lookup, by default None
post : Optional[BlockType], optional
Transformation block to apply after the embeddings lookup, by default None
aggregation : Optional[TabularAggregationType], optional
Transformation block to apply for aggregating the inputs, by default "concat"
tag_to_block : Dict[str, Callable[[Schema], Layer]], optional
Mapping from tag to block-type, by default:
Tags.CONTINUOUS -> Continuous
Tags.CATEGORICAL -> Embeddings
**branches : dict
Extra branches to add to the input block.
Returns
-------
ParallelBlock
Returns a ParallelBlock with a Dict with two branches:
continuous and embeddings
"""
target_schema = schema.select_by_tag(Tags.TARGET)
if target_schema.column_names:
raise ValueError(
"`schema` should not contain any target features. "
"Target columns can be removed with `scheme.remove_by_tag(Tags.TARGET)`. "
)
E ValueError: schema should not contain any target features. Target columns can be removed with scheme.remove_by_tag(Tags.TARGET).
merlin/models/tf/inputs/base.py:295: ValueError
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file4qw3_j46.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 28 failed, 709 passed, 12 skipped, 1140 warnings in 1101.97s (0:18:21) ====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins15216691388723892377.sh
Click to view CI Results
GitHub pull request #680 of commit ac87c58a385aa8ae3e6e40786789af19dbfb77d8, no merge conflicts.
Running as SYSTEM
Setting status of ac87c58a385aa8ae3e6e40786789af19dbfb77d8 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1506/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse ac87c58a385aa8ae3e6e40786789af19dbfb77d8^{commit} # timeout=10
Checking out Revision ac87c58a385aa8ae3e6e40786789af19dbfb77d8 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f ac87c58a385aa8ae3e6e40786789af19dbfb77d8 # timeout=10
Commit message: "Revert change to check targets in InputBlock"
> git rev-list --no-walk 059bf5ad8b6ea74f23bcdf1ee1994302c38b5648 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4854672158706312692.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 748 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 31%]
[ 31%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s.................... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filetmvljgp7.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 736 passed, 12 skipped, 1151 warnings in 1188.84s (0:19:48) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins1736912590910819654.sh
Click to view CI Results
GitHub pull request #680 of commit 167bc9e88411ec0c0114fb34a04e015d2c2756d1, no merge conflicts.
Running as SYSTEM
Setting status of 167bc9e88411ec0c0114fb34a04e015d2c2756d1 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1507/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 167bc9e88411ec0c0114fb34a04e015d2c2756d1^{commit} # timeout=10
Checking out Revision 167bc9e88411ec0c0114fb34a04e015d2c2756d1 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 167bc9e88411ec0c0114fb34a04e015d2c2756d1 # timeout=10
Commit message: "Restore newline in InputBlockV2"
> git rev-list --no-walk ac87c58a385aa8ae3e6e40786789af19dbfb77d8 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7927257454225441963.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 748 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 31%]
[ 31%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s.................... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 55%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 63%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 85%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 23 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileebyqiivd.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 736 passed, 12 skipped, 1151 warnings in 1193.15s (0:19:53) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins14004372042058152527.sh
Click to view CI Results
GitHub pull request #680 of commit d8ab1f0132e57a3d135a0e70628be717f7b2527a, no merge conflicts.
Running as SYSTEM
Setting status of d8ab1f0132e57a3d135a0e70628be717f7b2527a to PENDING with url https://10.20.13.93:8080/job/merlin_models/1509/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse d8ab1f0132e57a3d135a0e70628be717f7b2527a^{commit} # timeout=10
Checking out Revision d8ab1f0132e57a3d135a0e70628be717f7b2527a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f d8ab1f0132e57a3d135a0e70628be717f7b2527a # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 2c9a1547a439af393f6d63e22485ed21cb6adfd5 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1155320238188614029.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 750 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 10%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 22%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 23%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 25%]
tests/unit/tf/core/test_base.py .. [ 25%]
tests/unit/tf/core/test_combinators.py s.................... [ 28%]
tests/unit/tf/core/test_encoder.py .. [ 28%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 29%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 30%]
tests/unit/tf/examples/test_02_dataschema.py . [ 30%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 30%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 30%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 30%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 30%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 31%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 31%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]
...... [ 37%]
tests/unit/tf/inputs/test_tabular.py .................. [ 39%]
tests/unit/tf/layers/test_queue.py .............. [ 41%]
tests/unit/tf/losses/test_losses.py ....................... [ 44%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 45%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 48%]
tests/unit/tf/models/test_base.py s...................... [ 51%]
tests/unit/tf/models/test_benchmark.py .. [ 51%]
tests/unit/tf/models/test_ranking.py .................................. [ 56%]
tests/unit/tf/models/test_retrieval.py ................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 60%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .............. [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 82%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 88%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 25 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 25 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileadvpmou6.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 738 passed, 12 skipped, 1155 warnings in 1182.11s (0:19:42) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins13021938845246517905.sh
Click to view CI Results
GitHub pull request #680 of commit 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34, no merge conflicts.
Running as SYSTEM
Setting status of 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1525/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34^{commit} # timeout=10
Checking out Revision 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34 # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3164963490817653839.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 773 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 30%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 46%]
tests/unit/tf/models/test_base.py s........................ [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 50%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................ [ 58%]
tests/unit/tf/outputs/test_base.py ..... [ 59%]
tests/unit/tf/outputs/test_classification.py ...... [ 60%]
tests/unit/tf/outputs/test_contrastive.py ............. [ 61%]
tests/unit/tf/outputs/test_regression.py .. [ 62%]
tests/unit/tf/outputs/test_sampling.py .... [ 62%]
tests/unit/tf/outputs/test_topk.py . [ 62%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 63%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 65%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 65%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 66%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 67%]
tests/unit/tf/transformers/test_block.py .................... [ 69%]
tests/unit/tf/transformers/test_transforms.py ...... [ 70%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 74%]
....................s...... [ 78%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 79%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filetg9x_k07.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 761 passed, 12 skipped, 1205 warnings in 1496.82s (0:24:56) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins5306383322990356331.sh
Click to view CI Results
GitHub pull request #680 of commit cb01a27da433afe092f26446d8f96755d87f3b5a, no merge conflicts.
Running as SYSTEM
Setting status of cb01a27da433afe092f26446d8f96755d87f3b5a to PENDING with url https://10.20.13.93:8080/job/merlin_models/1529/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse cb01a27da433afe092f26446d8f96755d87f3b5a^{commit} # timeout=10
Checking out Revision cb01a27da433afe092f26446d8f96755d87f3b5a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f cb01a27da433afe092f26446d8f96755d87f3b5a # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 98b88014d80b3f366da4261b27d04cda70c4d301 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins13141315325995497407.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 773 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 30%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 46%]
tests/unit/tf/models/test_base.py s........................ [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 50%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................ [ 58%]
tests/unit/tf/outputs/test_base.py ..... [ 59%]
tests/unit/tf/outputs/test_classification.py ...... [ 60%]
tests/unit/tf/outputs/test_contrastive.py ............. [ 61%]
tests/unit/tf/outputs/test_regression.py .. [ 62%]
tests/unit/tf/outputs/test_sampling.py .... [ 62%]
tests/unit/tf/outputs/test_topk.py . [ 62%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 63%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 65%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 65%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 66%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 67%]
tests/unit/tf/transformers/test_block.py .................... [ 69%]
tests/unit/tf/transformers/test_transforms.py ...... [ 70%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 74%]
....................s...... [ 78%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 79%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file79yb3ad2.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 761 passed, 12 skipped, 1205 warnings in 1497.99s (0:24:57) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins793679268986863756.sh
Click to view CI Results
GitHub pull request #680 of commit 4425c882444cf954d47b9f6cc7840f769bc4485e, no merge conflicts.
Running as SYSTEM
Setting status of 4425c882444cf954d47b9f6cc7840f769bc4485e to PENDING with url https://10.20.13.93:8080/job/merlin_models/1563/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 4425c882444cf954d47b9f6cc7840f769bc4485e^{commit} # timeout=10
Checking out Revision 4425c882444cf954d47b9f6cc7840f769bc4485e (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 4425c882444cf954d47b9f6cc7840f769bc4485e # timeout=10
Commit message: "Update merlin metadata directory name to `.merlin`"
> git rev-list --no-walk dfa9b14801cf419be74f05474830f1d4c610d0ee # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins16653348676732720922.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 775 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 30%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
....... [ 36%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 46%]
tests/unit/tf/models/test_base.py s........................ [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 50%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................ [ 58%]
tests/unit/tf/outputs/test_base.py ..... [ 59%]
tests/unit/tf/outputs/test_classification.py ...... [ 60%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 62%]
tests/unit/tf/outputs/test_regression.py .. [ 62%]
tests/unit/tf/outputs/test_sampling.py .... [ 62%]
tests/unit/tf/outputs/test_topk.py . [ 62%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 63%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 65%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 65%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 66%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 67%]
tests/unit/tf/transformers/test_block.py .................... [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 70%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 74%]
....................s...... [ 78%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 79%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filenucq08cr.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 763 passed, 12 skipped, 1211 warnings in 1513.98s (0:25:13) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins447096621519452533.sh
Click to view CI Results
GitHub pull request #680 of commit 85566e424dd9f231214d0c717da67fad808db2d7, no merge conflicts.
Running as SYSTEM
Setting status of 85566e424dd9f231214d0c717da67fad808db2d7 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1564/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 85566e424dd9f231214d0c717da67fad808db2d7^{commit} # timeout=10
Checking out Revision 85566e424dd9f231214d0c717da67fad808db2d7 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 85566e424dd9f231214d0c717da67fad808db2d7 # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 4425c882444cf954d47b9f6cc7840f769bc4485e # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins5877863177796572011.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 776 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 30%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
....... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 46%]
tests/unit/tf/models/test_base.py s........................ [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 50%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................ [ 58%]
tests/unit/tf/outputs/test_base.py ...... [ 59%]
tests/unit/tf/outputs/test_classification.py ...... [ 60%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 62%]
tests/unit/tf/outputs/test_regression.py .. [ 62%]
tests/unit/tf/outputs/test_sampling.py .... [ 62%]
tests/unit/tf/outputs/test_topk.py . [ 63%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 63%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 65%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 65%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 66%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 67%]
tests/unit/tf/transformers/test_block.py .................... [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 70%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 78%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 79%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file1x4iy31d.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 764 passed, 12 skipped, 1213 warnings in 1493.01s (0:24:53) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins9116323796544861146.sh
Click to view CI Results
GitHub pull request #680 of commit 9e46d3de11311884f1bf2a232b783186b9f7f09f, no merge conflicts.
Running as SYSTEM
Setting status of 9e46d3de11311884f1bf2a232b783186b9f7f09f to PENDING with url https://10.20.13.93:8080/job/merlin_models/1568/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 9e46d3de11311884f1bf2a232b783186b9f7f09f^{commit} # timeout=10
Checking out Revision 9e46d3de11311884f1bf2a232b783186b9f7f09f (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9e46d3de11311884f1bf2a232b783186b9f7f09f # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk 34e5138053f6de313403e55bfe31532484a0284b # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins14170587462476034921.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 748 items / 2 errors
==================================== ERRORS ====================================
_________________ ERROR collecting tests/unit/tf/test_core.py __________________
tests/unit/tf/test_core.py:4: in
import merlin.models.tf as ml
merlin/models/tf/init.py:105: in
from merlin.models.tf.models.retrieval import (
merlin/models/tf/models/retrieval.py:14: in
from merlin.models.tf.prediction_tasks.retrieval import ItemRetrievalTask
merlin/models/tf/prediction_tasks/retrieval.py:33: in
class ItemRetrievalTask(MultiClassClassificationTask):
merlin/models/tf/prediction_tasks/retrieval.py:65: in ItemRetrievalTask
DEFAULT_METRICS = TopKMetricsAggregator.default_metrics(top_ks=[10])
merlin/models/tf/metrics/topk.py:463: in default_metrics
metrics.extend([RecallAt(k), MRRAt(k), NDCGAt(k), AvgPrecisionAt(k), PrecisionAt(k)])
merlin/models/tf/metrics/topk.py:340: in init
super().init(recall_at, k=k, pre_sorted=pre_sorted, name=name, **kwargs)
merlin/models/tf/metrics/topk.py:216: in init
super().init(name=name, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/dtensor/utils.py:141: in _wrap_function
init_method(instance, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/metrics/base_metric.py:578: in init
super(Mean, self).init(
/usr/local/lib/python3.8/dist-packages/keras/metrics/base_metric.py:413: in init
self.total = self.add_weight(
/usr/local/lib/python3.8/dist-packages/keras/metrics/base_metric.py:351: in add_weight
return super(Metric, self).add_weight(
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:665: in add_weight
variable = self._add_variable_with_custom_getter(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/training/tracking/base.py:873: in _add_variable_with_custom_getter
new_variable = getter(
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer_utils.py:126: in make_variable
return tf.compat.v1.Variable(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:153: in error_handler
raise e.with_traceback(filtered_tb) from None
/usr/local/lib/python3.8/dist-packages/keras/initializers/initializers_v2.py:152: in call
return tf.zeros(shape, dtype)
E tensorflow.python.framework.errors_impl.FailedPreconditionError: Failed to allocate scratch buffer for device 0
------------------------------- Captured stderr --------------------------------
2022-10-24 16:28:16.841507: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-24 16:28:19.058869: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-24 16:28:19.059045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-24 16:28:19.059739: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-24 16:28:19.059798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14476 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
2022-10-24 16:28:19.081464: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1028 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-24 16:28:19.081520: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 0
MaxInUse: 0
NumAllocs: 0
MaxAllocSize: 0
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-24 16:28:19.081529: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
_____________ ERROR collecting tests/unit/tf/losses/test_losses.py _____________
tests/unit/tf/losses/test_losses.py:28: in
ml.losses.BPRLoss(),
E AttributeError: module 'merlin.models.tf' has no attribute 'losses'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
======================== 7 warnings, 2 errors in 5.10s =========================
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins8462640987711217641.sh
Click to view CI Results
GitHub pull request #680 of commit 9e46d3de11311884f1bf2a232b783186b9f7f09f, no merge conflicts.
Running as SYSTEM
Setting status of 9e46d3de11311884f1bf2a232b783186b9f7f09f to PENDING with url https://10.20.13.93:8080/job/merlin_models/1574/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 9e46d3de11311884f1bf2a232b783186b9f7f09f^{commit} # timeout=10
Checking out Revision 9e46d3de11311884f1bf2a232b783186b9f7f09f (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9e46d3de11311884f1bf2a232b783186b9f7f09f # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk d18f7f4006f42c47d0f31ffb89c3a859232beced # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7056710828112936188.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 777 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
........ [ 36%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 46%]
tests/unit/tf/models/test_base.py s........................ [ 50%]
tests/unit/tf/models/test_benchmark.py .. [ 50%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................ [ 58%]
tests/unit/tf/outputs/test_base.py ...... [ 59%]
tests/unit/tf/outputs/test_classification.py ...... [ 60%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 62%]
tests/unit/tf/outputs/test_regression.py .. [ 62%]
tests/unit/tf/outputs/test_sampling.py .... [ 62%]
tests/unit/tf/outputs/test_topk.py . [ 63%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 63%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 65%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 66%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 66%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 67%]
tests/unit/tf/transformers/test_block.py .................... [ 70%]
tests/unit/tf/transformers/test_transforms.py ...... [ 70%]
tests/unit/tf/transforms/test_bias.py .. [ 71%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 78%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 79%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 32 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 4 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filesficthmc.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 765 passed, 12 skipped, 1213 warnings in 1486.41s (0:24:46) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins7723843817912108644.sh
Click to view CI Results
GitHub pull request #680 of commit 3f5da6d1e50ee87771a864374d98f57d3ceb3d3f, no merge conflicts.
Running as SYSTEM
Setting status of 3f5da6d1e50ee87771a864374d98f57d3ceb3d3f to PENDING with url https://10.20.13.93:8080/job/merlin_models/1674/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/680/*:refs/remotes/origin/pr/680/* # timeout=10
> git rev-parse 3f5da6d1e50ee87771a864374d98f57d3ceb3d3f^{commit} # timeout=10
Checking out Revision 3f5da6d1e50ee87771a864374d98f57d3ceb3d3f (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3f5da6d1e50ee87771a864374d98f57d3ceb3d3f # timeout=10
Commit message: "Merge branch 'main' into model-protocol"
> git rev-list --no-walk c078dbdd0292bdb972417406c6ba88bcf9f3295a # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7791525283155167860.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
GLOB sdist-make: /var/jenkins_home/workspace/merlin_models/models/setup.py
test-gpu inst-nodeps: /var/jenkins_home/workspace/merlin_models/models/.tox/.tmp/package/1/merlin-models-0.9.0+45.g3f5da6d1.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,appdirs==1.4.4,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.26.4,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==20.8b1,bleach==5.0.1,boto3==1.24.75,botocore==1.28.4,Brotli==1.0.9,build==0.9.0,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,check-manifest==0.48,clang==5.0,click==8.1.3,cliff==4.0.0,cloudpickle==2.2.0,cmaes==0.8.2,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flake8==5.0.4,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,interrogate==1.5.0,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,isort==5.10.1,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter==1.0.0,jupyter-cache==0.4.3,jupyter-console==6.4.4,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.3,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mccabe==0.7.0,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.9.0+45.g3f5da6d1,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy==0.982,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathspec==0.10.1,pathtools==0.1.2,pbr==5.11.0,pep517==0.13.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycodestyle==2.9.1,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,pyflakes==2.5.0,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,qtconsole==5.3.2,QtPy==2.2.1,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.42,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.0,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,tensorflow-ranking==0.5.1,tensorflow-serving-api==2.9.2,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typed-ast==1.5.4,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='318182881'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-ao9aw8sg
Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-ao9aw8sg
Resolved https://github.com/NVIDIA-Merlin/core.git to commit eda153c663aa864da66927c7a0a9d4e64c073120
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+4.geda153c) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+4.geda153c) (2022.5.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+4.geda153c) (0.55.1)
Requirement already satisfied: betterproto=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+4.geda153c) (4.64.1)
Requirement already satisfied: pandas=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+4.geda153c) (1.3.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+4.geda153c) (21.3)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+4.geda153c) (1.10.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+4.geda153c) (7.0.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+4.geda153c) (2022.3.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+4.geda153c) (2022.3.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterprotomerlin-core==0.8.0+4.geda153c) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterprotomerlin-core==0.8.0+4.geda153c) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.8.0+4.geda153c) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+4.geda153c) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.8.0+4.geda153c) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+4.geda153c) (2.2.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (5.8.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (1.0.4)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/test-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (6.2)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (3.1.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (2.0.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (1.7.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (8.1.3)
Requirement already satisfied: llvmlite=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+4.geda153c) (0.38.1)
Requirement already satisfied: numpy=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+4.geda153c) (1.20.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+4.geda153c) (65.4.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.8.0+4.geda153c) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas=1.2.0->merlin-core==0.8.0+4.geda153c) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas=1.2.0->merlin-core==0.8.0+4.geda153c) (2022.2.1)
Requirement already satisfied: googleapis-common-protos=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+4.geda153c) (1.52.0)
Requirement already satisfied: absl-py=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+4.geda153c) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.8.0+4.geda153c) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas=1.2.0->merlin-core==0.8.0+4.geda153c) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterprotomerlin-core==0.8.0+4.geda153c) (6.0.2)
Requirement already satisfied: h2=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterprotomerlin-core==0.8.0+4.geda153c) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.8.0+4.geda153c) (2.0.1)
Requirement already satisfied: hpack=4.0 in /usr/local/lib/python3.8/dist-packages (from h2=3.1.0->grpclib->betterprotomerlin-core==0.8.0+4.geda153c) (4.0.0)
Requirement already satisfied: hyperframe=6.0 in /usr/local/lib/python3.8/dist-packages (from h2=3.1.0->grpclib->betterprotomerlin-core==0.8.0+4.geda153c) (6.0.1)
[notice] A new release of pip available: 22.2.2 -> 22.3
[notice] To update, run: pip install --upgrade pip
test-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/nvtabular.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/nvtabular.git
Cloning https://github.com/NVIDIA-Merlin/nvtabular.git to /tmp/pip-req-build-xq451et1
Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/nvtabular.git /tmp/pip-req-build-xq451et1
Resolved https://github.com/NVIDIA-Merlin/nvtabular.git to commit c0636deadefebcf238935267239406bf7cc6803f
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from nvtabular==1.6.0+1.gc0636dea) (0.8.0+4.geda153c)
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+1.gc0636dea) (1.8.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2022.5.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (0.55.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.2.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (4.64.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.3.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (21.3)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.10.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (7.0.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2022.3.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2022.3.0)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (from scipy->nvtabular==1.6.0+1.gc0636dea) (1.20.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2.2.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (5.8.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.0.4)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/test-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (6.2)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (3.1.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2.0.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.7.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (8.1.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (65.4.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+1.gc0636dea) (6.0.1)
[notice] A new release of pip available: 22.2.2 -> 22.3
[notice] To update, run: pip install --upgrade pip
test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/test-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-3.0.2, cov-4.0.0
collected 829 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 22%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 27%]
[ 27%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_incremental_training_layer_freezing.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 37%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ......................... [ 44%]
tests/unit/tf/models/test_base.py s........................ [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 85%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19
/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/var/jenkins_home/workspace/merlin_models/models/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 27 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/var/jenkins_home/workspace/merlin_models/models/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/var/jenkins_home/workspace/merlin_models/models/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file0jhw6dz4.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/var/jenkins_home/workspace/merlin_models/models/.tox/test-gpu/lib/python3.8/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover
merlin/datasets/init.py 2 0 100%
merlin/datasets/advertising/init.py 2 0 100%
merlin/datasets/advertising/criteo/init.py 0 0 100%
merlin/datasets/advertising/criteo/dataset.py 79 60 24%
merlin/datasets/advertising/criteo/transformed/init.py 0 0 100%
merlin/datasets/ecommerce/init.py 3 0 100%
merlin/datasets/ecommerce/aliccp/init.py 0 0 100%
merlin/datasets/ecommerce/aliccp/dataset.py 141 87 38%
merlin/datasets/ecommerce/aliccp/raw/init.py 0 0 100%
merlin/datasets/ecommerce/aliccp/transformed/init.py 0 0 100%
merlin/datasets/ecommerce/dressipi/init.py 0 0 100%
merlin/datasets/ecommerce/dressipi/dataset.py 45 37 18%
merlin/datasets/ecommerce/dressipi/preprocessed/init.py 0 0 100%
merlin/datasets/ecommerce/large/init.py 0 0 100%
merlin/datasets/ecommerce/small/init.py 0 0 100%
merlin/datasets/entertainment/init.py 2 0 100%
merlin/datasets/entertainment/movielens/1m-raw/init.py 0 0 100%
merlin/datasets/entertainment/movielens/1m-raw/ratings/init.py 0 0 100%
merlin/datasets/entertainment/movielens/1m/init.py 0 0 100%
merlin/datasets/entertainment/movielens/25m/init.py 0 0 100%
merlin/datasets/entertainment/movielens/100k/init.py 0 0 100%
merlin/datasets/entertainment/movielens/init.py 0 0 100%
merlin/datasets/entertainment/movielens/dataset.py 152 122 20%
merlin/datasets/entertainment/music_streaming/init.py 0 0 100%
merlin/datasets/social/init.py 0 0 100%
merlin/datasets/synthetic.py 147 15 90%
merlin/datasets/testing/init.py 0 0 100%
merlin/datasets/testing/sequence_testing/init.py 0 0 100%
merlin/models/init.py 2 0 100%
merlin/models/_version.py 354 205 42%
merlin/models/api.py 14 5 64%
merlin/models/config/init.py 0 0 100%
merlin/models/config/schema.py 62 0 100%
merlin/models/implicit/init.py 27 4 85%
merlin/models/io.py 15 0 100%
merlin/models/lightfm/init.py 23 0 100%
merlin/models/loader/init.py 0 0 100%
merlin/models/loader/backend.py 379 42 89%
merlin/models/loader/dataframe_iter.py 21 17 19%
merlin/models/loader/tf_utils.py 57 27 53%
merlin/models/loader/utils.py 40 15 62%
merlin/models/tf/init.py 70 0 100%
merlin/models/tf/blocks/init.py 0 0 100%
merlin/models/tf/blocks/cross.py 44 0 100%
merlin/models/tf/blocks/dlrm.py 49 2 96%
merlin/models/tf/blocks/experts.py 99 17 83%
merlin/models/tf/blocks/interaction.py 108 40 63%
merlin/models/tf/blocks/mlp.py 114 10 91%
merlin/models/tf/blocks/optimizer.py 173 13 92%
merlin/models/tf/blocks/retrieval/init.py 0 0 100%
merlin/models/tf/blocks/retrieval/base.py 169 62 63%
merlin/models/tf/blocks/retrieval/matrix_factorization.py 35 1 97%
merlin/models/tf/blocks/retrieval/two_tower.py 30 0 100%
merlin/models/tf/blocks/sampling/init.py 0 0 100%
merlin/models/tf/blocks/sampling/base.py 29 2 93%
merlin/models/tf/blocks/sampling/cross_batch.py 46 2 96%
merlin/models/tf/blocks/sampling/in_batch.py 35 0 100%
merlin/models/tf/blocks/sampling/queue.py 115 12 90%
merlin/models/tf/core/init.py 0 0 100%
merlin/models/tf/core/aggregation.py 245 49 80%
merlin/models/tf/core/base.py 244 55 77%
merlin/models/tf/core/combinators.py 421 54 87%
merlin/models/tf/core/encoder.py 170 29 83%
merlin/models/tf/core/index.py 104 16 85%
merlin/models/tf/core/prediction.py 50 1 98%
merlin/models/tf/core/tabular.py 288 30 90%
merlin/models/tf/inputs/init.py 0 0 100%
merlin/models/tf/inputs/base.py 60 10 83%
merlin/models/tf/inputs/continuous.py 39 3 92%
merlin/models/tf/inputs/embedding.py 452 31 93%
merlin/models/tf/loader.py 243 73 70%
merlin/models/tf/losses/init.py 4 0 100%
merlin/models/tf/losses/base.py 9 0 100%
merlin/models/tf/losses/listwise.py 13 0 100%
merlin/models/tf/losses/pairwise.py 115 1 99%
merlin/models/tf/metrics/init.py 2 0 100%
merlin/models/tf/metrics/evaluation.py 105 48 54%
merlin/models/tf/metrics/topk.py 198 48 76%
merlin/models/tf/models/init.py 0 0 100%
merlin/models/tf/models/base.py 707 75 89%
merlin/models/tf/models/benchmark.py 16 0 100%
merlin/models/tf/models/ranking.py 67 3 96%
merlin/models/tf/models/retrieval.py 78 4 95%
merlin/models/tf/models/utils.py 10 1 90%
merlin/models/tf/outputs/init.py 0 0 100%
merlin/models/tf/outputs/base.py 122 17 86%
merlin/models/tf/outputs/classification.py 91 1 99%
merlin/models/tf/outputs/contrastive.py 147 10 93%
merlin/models/tf/outputs/regression.py 9 0 100%
merlin/models/tf/outputs/sampling/init.py 0 0 100%
merlin/models/tf/outputs/sampling/base.py 78 21 73%
merlin/models/tf/outputs/sampling/in_batch.py 37 1 97%
merlin/models/tf/outputs/sampling/popularity.py 27 1 96%
merlin/models/tf/outputs/topk.py 97 6 94%
merlin/models/tf/prediction_tasks/init.py 0 0 100%
merlin/models/tf/prediction_tasks/base.py 207 37 82%
merlin/models/tf/prediction_tasks/classification.py 68 17 75%
merlin/models/tf/prediction_tasks/multi.py 7 0 100%
merlin/models/tf/prediction_tasks/next_item.py 59 6 90%
merlin/models/tf/prediction_tasks/regression.py 35 2 94%
merlin/models/tf/prediction_tasks/retrieval.py 73 3 96%
merlin/models/tf/transformers/init.py 0 0 100%
merlin/models/tf/transformers/block.py 101 2 98%
merlin/models/tf/transformers/transforms.py 63 0 100%
merlin/models/tf/transforms/init.py 0 0 100%
merlin/models/tf/transforms/bias.py 111 9 92%
merlin/models/tf/transforms/features.py 422 34 92%
merlin/models/tf/transforms/negative_sampling.py 77 3 96%
merlin/models/tf/transforms/noise.py 43 1 98%
merlin/models/tf/transforms/regularization.py 17 1 94%
merlin/models/tf/transforms/sequence.py 282 42 85%
merlin/models/tf/transforms/tensor.py 107 2 98%
merlin/models/tf/typing.py 7 0 100%
merlin/models/tf/utils/init.py 0 0 100%
merlin/models/tf/utils/batch_utils.py 85 4 95%
merlin/models/tf/utils/repr_utils.py 69 4 94%
merlin/models/tf/utils/search_utils.py 34 22 35%
merlin/models/tf/utils/testing_utils.py 206 36 83%
merlin/models/tf/utils/tf_utils.py 209 42 80%
merlin/models/torch/init.py 12 0 100%
merlin/models/torch/block/init.py 0 0 100%
merlin/models/torch/block/base.py 167 32 81%
merlin/models/torch/block/mlp.py 38 5 87%
merlin/models/torch/dataset.py 68 5 93%
merlin/models/torch/features/init.py 0 0 100%
merlin/models/torch/features/base.py 4 0 100%
merlin/models/torch/features/continuous.py 22 0 100%
merlin/models/torch/features/embedding.py 165 12 93%
merlin/models/torch/features/tabular.py 65 8 88%
merlin/models/torch/losses.py 28 28 0%
merlin/models/torch/model/init.py 0 0 100%
merlin/models/torch/model/base.py 286 65 77%
merlin/models/torch/model/prediction_task.py 24 0 100%
merlin/models/torch/tabular/init.py 0 0 100%
merlin/models/torch/tabular/aggregation.py 75 0 100%
merlin/models/torch/tabular/base.py 247 39 84%
merlin/models/torch/tabular/transformations.py 67 3 96%
merlin/models/torch/typing.py 6 0 100%
merlin/models/torch/utils/init.py 0 0 100%
merlin/models/torch/utils/data_utils.py 117 117 0%
merlin/models/torch/utils/examples_utils.py 39 39 0%
merlin/models/torch/utils/torch_utils.py 80 22 72%
merlin/models/utils/init.py 0 0 100%
merlin/models/utils/constants.py 3 0 100%
merlin/models/utils/dataset.py 38 4 89%
merlin/models/utils/dependencies.py 26 19 27%
merlin/models/utils/doc_utils.py 10 0 100%
merlin/models/utils/example_utils.py 31 2 94%
merlin/models/utils/misc_utils.py 118 90 24%
merlin/models/utils/nvt_utils.py 27 24 11%
merlin/models/utils/registry.py 101 29 71%
merlin/models/utils/schema_utils.py 90 19 79%
merlin/models/xgb/init.py 124 4 97%
TOTAL 11291 2218 80%
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../.local/lib/python3.8/site-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 817 passed, 12 skipped, 1347 warnings in 1542.58s (0:25:42) ==========
___________________________________ summary ____________________________________
test-gpu: commands succeeded
congratulations :)
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins10419969593119354337.sh