NVIDIA-Merlin
Retrieval models definition using new RetrievalModelV2, InputBlockV2 and ModelOutput classes
Fixes #611 Fixes #805 Uses a workaround in the integration test to avoid the issue #144
Goals :soccer:
-
The objective of this PR is to implement a new version of high-level retrieval models API (
MatrixFactorizationModelV2,TwoTowerModelV2, andYoutubeDNNRetrievalModelV2) using the new core API (such as RetrievalModelV2, InputBlockV2, and ModelOutput) -
The tested functionalities for the three models are: - Saving/loading retrieval models - Saving /loading Top-k encoders defined from the trained retrieval models.
- Computing top-k evaluation of retrieval models - Exporting item embeddings and the query tower.
Implementation Details :construction:
- [x] Introduce new retrieval methods: MatrixFactorizationModelV2, TwoTowerModelV2, and YoutubeDNNRetrievalModelV2
- [x] Ensure embeddings export is working for MatrixFactorizationModelV2 and TwoTowerModelV2
- [x] Link Top-K encoder to retrieval models.
Testing Details :mag:
- [x] Update all unit tests of retrieval models with the new API.
- [x] Test export towers and embeddings works for MatrixFactorizationModelV2 and TwoTowerModelV2.
- [x] Test top-k recommendation and evaluation of MatrixFactorizationModelV2 and TwoTowerModelV2.
- [x] Test top-k recommendation and evaluation of YoutubeDNNRetrievalModelV2.
- [x] Test MatrixFactorizationV2 and TwoTowerModelV2 with Binary and Regression tasks.
- [x] Add integration tests for MatrixFactorizationV2 and TwoTowerModelV2.
Notes:
-
This PR does not refactor the old retrieval API
-
While exporting
candidates embeddingsis working, exporting the query embeddings fromYoutubeDNNRetrievalModelV2raises an error related to thebatch-predictstep. The error (more details can be found in the issue #801) is related to the apply method of dask-df does not support a list column with an unspecified dtype. -
Integration tests showed a mismatch between the evaluation scores of the old and new API. After debugging the API, I found three differences that needed to be fixed in the new API (embedding regularization (fixed here), log-q correction (fixed here), and logits scaling (fixed here) )
Click to view CI Results
GitHub pull request #790 of commit e104e9507802f4224265baa939cd05a7ecdd6c38, no merge conflicts.
Running as SYSTEM
Setting status of e104e9507802f4224265baa939cd05a7ecdd6c38 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1473/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse e104e9507802f4224265baa939cd05a7ecdd6c38^{commit} # timeout=10
Checking out Revision e104e9507802f4224265baa939cd05a7ecdd6c38 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f e104e9507802f4224265baa939cd05a7ecdd6c38 # timeout=10
Commit message: "Add retrieval models using new RetrievalModelV2 and ModelOutput classes"
> git rev-list --no-walk 0f168feed2c18a3bd7b4a43393f1b7e6fc4a90a8 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3469859236073264739.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 770 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 8%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 14%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 21%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 21%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 23%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 28%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 29%]
tests/unit/tf/examples/test_01_getting_started.py . [ 29%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 30%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]
[ 30%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 30%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 31%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
...... [ 36%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 40%]
tests/unit/tf/losses/test_losses.py ....................... [ 43%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 46%]
tests/unit/tf/models/test_base.py s................... [ 49%]
tests/unit/tf/models/test_benchmark.py .. [ 49%]
tests/unit/tf/models/test_ranking.py .................................. [ 54%]
tests/unit/tf/models/test_retrieval.py ................................. [ 58%]
...................... [ 61%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 65%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 69%]
tests/unit/tf/transformers/test_block.py .............. [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 81%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 83%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 91%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 90 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 60 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 8 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileahp15cpc.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:497: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:603: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 758 passed, 12 skipped, 1218 warnings in 1337.03s (0:22:17) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins17929154612076238633.sh
I don't think we can close this issue https://github.com/NVIDIA-Merlin/models/issues/498 (in the sense that the issue is referring to the specific version of TwoTowerModel that will continue to have the loading error even with this new version?). So that issue might exist until we've removed the old TwoTowerModel and migrated all the examples to use the new one?
I don't think we can close this issue #498 (in the sense that the issue is referring to the specific version of
TwoTowerModelthat will continue to have the loading error even with this new version?). So that issue might exist until we've removed the oldTwoTowerModeland migrated all the examples to use the new one?
That's a good point! The focus of this PR is to implement a new version of retrieval models, such as TwoTowerModelV2. Removing the old API and updating the examples should happen in a separate PR. I removed the issue from the PR description and just add a comment about using TwoTowerModelV2 as an alternative solution.
Click to view CI Results
GitHub pull request #790 of commit 14df5ba3418907994f0d446e8ba52abdce3230a2, no merge conflicts.
Running as SYSTEM
Setting status of 14df5ba3418907994f0d446e8ba52abdce3230a2 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1479/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 14df5ba3418907994f0d446e8ba52abdce3230a2^{commit} # timeout=10
Checking out Revision 14df5ba3418907994f0d446e8ba52abdce3230a2 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 14df5ba3418907994f0d446e8ba52abdce3230a2 # timeout=10
Commit message: "add more tests"
> git rev-list --no-walk f96eb21a097a2a40d7a71dce617ac61550c6b26f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4139311124539209891.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 777 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 39%]
tests/unit/tf/losses/test_losses.py ....................... [ 42%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 46%]
tests/unit/tf/models/test_base.py s................... [ 49%]
tests/unit/tf/models/test_benchmark.py .. [ 49%]
tests/unit/tf/models/test_ranking.py .................................. [ 53%]
tests/unit/tf/models/test_retrieval.py .....FF..........FF.............. [ 57%]
............................. [ 61%]
tests/unit/tf/outputs/test_base.py ..... [ 62%]
tests/unit/tf/outputs/test_classification.py ...... [ 63%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 65%]
tests/unit/tf/outputs/test_topk.py . [ 65%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 69%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 69%]
tests/unit/tf/transformers/test_block.py .............. [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 80%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 81%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
_______________ test_matrix_factorization_topk_evaluation[True] ________________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f87841e7580>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_matrix_factorization_topk_evaluation(music_streaming_data: Dataset, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(["user_id", "item_id"])
model = mm.MatrixFactorizationModelV2(
music_streaming_data.schema, negative_samplers="in-batch", dim=4
)
model.compile(optimizer="adam", run_eagerly=run_eagerly)
_, losses = testing_utils.model_test(model, music_streaming_data, reload_model=False)
# Top-K evaluation
candidates = model.candidate_embeddings(
music_streaming_data, batch_size=10, index=Tags.ITEM_ID
).compute()
topk_model = mm.TopKEncoder(model.query_encoder, candidates=candidates)
tests/unit/tf/models/test_retrieval.py:79:
merlin/models/tf/core/encoder.py:294: in init
topk_output = TopKOutput(to_call=topk_layer, candidates=candidates, k=k, **kwargs)
merlin/models/tf/outputs/topk.py:272: in init
to_call = block_registry.parse(to_call).index(candidates)
merlin/models/tf/outputs/topk.py:133: in index
tf.rank(candidates), 2, f"candidates must be 2-D tensor (got {candidates.shape})"
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:858: in rank
return rank_internal(input, name, optimize=True)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:878: in rank_internal
input = ops.convert_to_tensor(input)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:279: in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:304: in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102: in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
self = 0 1 2 3
item_id
1 0.013313 0.0...133 -0.008792 0.017329 0.010425 -0.002457
231 -0.022575 0.008712 0.012764 -0.003492
[100 rows x 4 columns]
dtype = None
def __array__(self, dtype=None):
raise TypeError(
"Implicit conversion to a host NumPy array via __array__ is not "
"allowed, To explicitly construct a GPU matrix, consider using "
".to_cupy()\nTo explicitly construct a host matrix, consider "
"using .to_numpy()."
)
E TypeError: Implicit conversion to a host NumPy array via array is not allowed, To explicitly construct a GPU matrix, consider using .to_cupy()
E To explicitly construct a host matrix, consider using .to_numpy().
/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:473: TypeError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 3.8926 - recall_at_10: 0.1600 - mrr_at_10: 0.0374 - ndcg_at_10: 0.0659 - map_at_10: 0.0374 - precision_at_10: 0.0160 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 898ms/step - loss: 3.8926 - recall_at_10: 0.1600 - mrr_at_10: 0.0374 - ndcg_at_10: 0.0659 - map_at_10: 0.0374 - precision_at_10: 0.0160 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
------------------------------ Captured log call -------------------------------
WARNING absl:save.py:233 Found untraced functions such as item_id_layer_call_fn, item_id_layer_call_and_return_conditional_losses, model_context_layer_call_fn, model_context_layer_call_and_return_conditional_losses while saving (showing 4 of 4). These functions will not be directly callable after loading.
WARNING tensorflow:load.py:167 No training configuration found in save file, so the model was not compiled. Compile it manually.
_______________ test_matrix_factorization_topk_evaluation[False] _______________
music_streaming_data = <merlin.io.dataset.Dataset object at 0x7f8784b4cd00>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_matrix_factorization_topk_evaluation(music_streaming_data: Dataset, run_eagerly):
music_streaming_data.schema = music_streaming_data.schema.select_by_name(["user_id", "item_id"])
model = mm.MatrixFactorizationModelV2(
music_streaming_data.schema, negative_samplers="in-batch", dim=4
)
model.compile(optimizer="adam", run_eagerly=run_eagerly)
_, losses = testing_utils.model_test(model, music_streaming_data, reload_model=False)
# Top-K evaluation
candidates = model.candidate_embeddings(
music_streaming_data, batch_size=10, index=Tags.ITEM_ID
).compute()
topk_model = mm.TopKEncoder(model.query_encoder, candidates=candidates)
tests/unit/tf/models/test_retrieval.py:79:
merlin/models/tf/core/encoder.py:294: in init
topk_output = TopKOutput(to_call=topk_layer, candidates=candidates, k=k, **kwargs)
merlin/models/tf/outputs/topk.py:272: in init
to_call = block_registry.parse(to_call).index(candidates)
merlin/models/tf/outputs/topk.py:133: in index
tf.rank(candidates), 2, f"candidates must be 2-D tensor (got {candidates.shape})"
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:858: in rank
return rank_internal(input, name, optimize=True)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:878: in rank_internal
input = ops.convert_to_tensor(input)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:279: in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:304: in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102: in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
self = 0 1 2 3
item_id
1 0.008653 0.0...136 -0.001883 -0.011358 -0.015552 -0.004689
172 -0.008424 0.010785 0.022289 0.012375
[100 rows x 4 columns]
dtype = None
def __array__(self, dtype=None):
raise TypeError(
"Implicit conversion to a host NumPy array via __array__ is not "
"allowed, To explicitly construct a GPU matrix, consider using "
".to_cupy()\nTo explicitly construct a host matrix, consider "
"using .to_numpy()."
)
E TypeError: Implicit conversion to a host NumPy array via array is not allowed, To explicitly construct a GPU matrix, consider using .to_cupy()
E To explicitly construct a host matrix, consider using .to_numpy().
/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:473: TypeError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 3.8859 - recall_at_10: 0.1400 - mrr_at_10: 0.0533 - ndcg_at_10: 0.0739 - map_at_10: 0.0533 - precision_at_10: 0.0140 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 913ms/step - loss: 3.8859 - recall_at_10: 0.1400 - mrr_at_10: 0.0533 - ndcg_at_10: 0.0739 - map_at_10: 0.0533 - precision_at_10: 0.0140 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
------------------------------ Captured log call -------------------------------
WARNING absl:save.py:233 Found untraced functions such as item_id_layer_call_fn, item_id_layer_call_and_return_conditional_losses, model_context_layer_call_fn, model_context_layer_call_and_return_conditional_losses while saving (showing 4 of 4). These functions will not be directly callable after loading.
WARNING tensorflow:load.py:167 No training configuration found in save file, so the model was not compiled. Compile it manually.
__________________ test_two_tower_model_topk_evaluation[True] __________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f8775a93070>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_two_tower_model_topk_evaluation(ecommerce_data: Dataset, run_eagerly):
dataset = ecommerce_data
schema = dataset.schema
query = mm.Encoder(
schema.select_by_tag(Tags.USER), mm.MLPBlock([4], no_activation_last_layer=True)
)
candidate = mm.Encoder(
schema.select_by_tag(Tags.ITEM), mm.MLPBlock([4], no_activation_last_layer=True)
)
model = mm.TwoTowerModelV2(
query,
candidate,
negative_samplers=["in-batch"],
)
model.compile(optimizer="adam", run_eagerly=run_eagerly)
_ = testing_utils.model_test(model, dataset)
# Top-K evaluation
candidates = model.candidate_embeddings(
ecommerce_data, batch_size=10, index=Tags.ITEM_ID
).compute()
topk_model = mm.TopKEncoder(query, candidates=candidates)
tests/unit/tf/models/test_retrieval.py:298:
merlin/models/tf/core/encoder.py:294: in init
topk_output = TopKOutput(to_call=topk_layer, candidates=candidates, k=k, **kwargs)
merlin/models/tf/outputs/topk.py:272: in init
to_call = block_registry.parse(to_call).index(candidates)
merlin/models/tf/outputs/topk.py:133: in index
tf.rank(candidates), 2, f"candidates must be 2-D tensor (got {candidates.shape})"
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:858: in rank
return rank_internal(input, name, optimize=True)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:878: in rank_internal
input = ops.convert_to_tensor(input)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:279: in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:304: in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102: in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
self = 0 1 2 3
item_id
2 0.010746 0.0...126 -0.061176 0.035026 -0.011782 -0.033973
265 0.088493 0.026366 -0.009573 0.006230
[100 rows x 4 columns]
dtype = None
def __array__(self, dtype=None):
raise TypeError(
"Implicit conversion to a host NumPy array via __array__ is not "
"allowed, To explicitly construct a GPU matrix, consider using "
".to_cupy()\nTo explicitly construct a host matrix, consider "
"using .to_numpy()."
)
E TypeError: Implicit conversion to a host NumPy array via array is not allowed, To explicitly construct a GPU matrix, consider using .to_cupy()
E To explicitly construct a host matrix, consider using .to_numpy().
/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:473: TypeError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 3.8943 - recall_at_10: 0.2000 - mrr_at_10: 0.0796 - ndcg_at_10: 0.1076 - map_at_10: 0.0796 - precision_at_10: 0.0200 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 1s/step - loss: 3.8943 - recall_at_10: 0.2000 - mrr_at_10: 0.0796 - ndcg_at_10: 0.1076 - map_at_10: 0.0796 - precision_at_10: 0.0200 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
------------------------------ Captured log call -------------------------------
WARNING absl:save.py:233 Found untraced functions such as concat_features_1_layer_call_fn, concat_features_1_layer_call_and_return_conditional_losses, model_context_layer_call_fn, model_context_layer_call_and_return_conditional_losses, embeddings_layer_call_fn while saving (showing 5 of 30). These functions will not be directly callable after loading.
WARNING tensorflow:load.py:167 No training configuration found in save file, so the model was not compiled. Compile it manually.
_________________ test_two_tower_model_topk_evaluation[False] __________________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f877456e760>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_two_tower_model_topk_evaluation(ecommerce_data: Dataset, run_eagerly):
dataset = ecommerce_data
schema = dataset.schema
query = mm.Encoder(
schema.select_by_tag(Tags.USER), mm.MLPBlock([4], no_activation_last_layer=True)
)
candidate = mm.Encoder(
schema.select_by_tag(Tags.ITEM), mm.MLPBlock([4], no_activation_last_layer=True)
)
model = mm.TwoTowerModelV2(
query,
candidate,
negative_samplers=["in-batch"],
)
model.compile(optimizer="adam", run_eagerly=run_eagerly)
_ = testing_utils.model_test(model, dataset)
# Top-K evaluation
candidates = model.candidate_embeddings(
ecommerce_data, batch_size=10, index=Tags.ITEM_ID
).compute()
topk_model = mm.TopKEncoder(query, candidates=candidates)
tests/unit/tf/models/test_retrieval.py:298:
merlin/models/tf/core/encoder.py:294: in init
topk_output = TopKOutput(to_call=topk_layer, candidates=candidates, k=k, **kwargs)
merlin/models/tf/outputs/topk.py:272: in init
to_call = block_registry.parse(to_call).index(candidates)
merlin/models/tf/outputs/topk.py:133: in index
tf.rank(candidates), 2, f"candidates must be 2-D tensor (got {candidates.shape})"
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:858: in rank
return rank_internal(input, name, optimize=True)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:878: in rank_internal
input = ops.convert_to_tensor(input)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:279: in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:304: in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102: in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
self = 0 1 2 3
item_id
1 0.004530 -0.0...123 0.000196 -0.032682 0.060176 0.013600
152 -0.013238 -0.003009 -0.045097 0.028733
[100 rows x 4 columns]
dtype = None
def __array__(self, dtype=None):
raise TypeError(
"Implicit conversion to a host NumPy array via __array__ is not "
"allowed, To explicitly construct a GPU matrix, consider using "
".to_cupy()\nTo explicitly construct a host matrix, consider "
"using .to_numpy()."
)
E TypeError: Implicit conversion to a host NumPy array via array is not allowed, To explicitly construct a GPU matrix, consider using .to_cupy()
E To explicitly construct a host matrix, consider using .to_numpy().
/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:473: TypeError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 3.8958 - recall_at_10: 0.1600 - mrr_at_10: 0.0327 - ndcg_at_10: 0.0617 - map_at_10: 0.0327 - precision_at_10: 0.0160 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 1s/step - loss: 3.8958 - recall_at_10: 0.1600 - mrr_at_10: 0.0327 - ndcg_at_10: 0.0617 - map_at_10: 0.0327 - precision_at_10: 0.0160 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
------------------------------ Captured log call -------------------------------
WARNING absl:save.py:233 Found untraced functions such as concat_features_1_layer_call_fn, concat_features_1_layer_call_and_return_conditional_losses, model_context_layer_call_fn, model_context_layer_call_and_return_conditional_losses, embeddings_layer_call_fn while saving (showing 5 of 30). These functions will not be directly callable after loading.
WARNING tensorflow:load.py:167 No training configuration found in save file, so the model was not compiled. Compile it manually.
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 104 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 70 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filelha7aimi.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 26 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/distributed/node.py:177: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 33529 instead
warnings.warn(
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 4 failed, 761 passed, 12 skipped, 1245 warnings in 1372.90s (0:22:52) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins10585445149495457255.sh
Click to view CI Results
GitHub pull request #790 of commit 53d1958e94fb28e046988398ce1641057d278744, no merge conflicts.
Running as SYSTEM
Setting status of 53d1958e94fb28e046988398ce1641057d278744 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1481/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 53d1958e94fb28e046988398ce1641057d278744^{commit} # timeout=10
Checking out Revision 53d1958e94fb28e046988398ce1641057d278744 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 53d1958e94fb28e046988398ce1641057d278744 # timeout=10
Commit message: "try to fix failing gpu tests"
> git rev-list --no-walk 2d5f3079b34686d825d1b45fb9e87924e183c753 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins17061518648229853232.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 777 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 23%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 27%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 28%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 29%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 30%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 35%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 38%]
tests/unit/tf/layers/test_queue.py .............. [ 39%]
tests/unit/tf/losses/test_losses.py ....................... [ 42%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 46%]
tests/unit/tf/models/test_base.py s................... [ 49%]
tests/unit/tf/models/test_benchmark.py .. [ 49%]
tests/unit/tf/models/test_ranking.py .................................. [ 53%]
tests/unit/tf/models/test_retrieval.py ................................. [ 57%]
............................. [ 61%]
tests/unit/tf/outputs/test_base.py ..... [ 62%]
tests/unit/tf/outputs/test_classification.py ...... [ 63%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 65%]
tests/unit/tf/outputs/test_topk.py . [ 65%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 69%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 69%]
tests/unit/tf/transformers/test_block.py .............. [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 80%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 81%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 85%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 90%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 5 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 104 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 22 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 70 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filegi_q210o.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 765 passed, 12 skipped, 1248 warnings in 1374.89s (0:22:54) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins16758523690884683711.sh
Click to view CI Results
GitHub pull request #790 of commit ec3701e0928126ce1b7acf2aebffafe19fc0354c, no merge conflicts.
Running as SYSTEM
Setting status of ec3701e0928126ce1b7acf2aebffafe19fc0354c to PENDING with url https://10.20.13.93:8080/job/merlin_models/1491/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse ec3701e0928126ce1b7acf2aebffafe19fc0354c^{commit} # timeout=10
Checking out Revision ec3701e0928126ce1b7acf2aebffafe19fc0354c (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f ec3701e0928126ce1b7acf2aebffafe19fc0354c # timeout=10
Commit message: "add tests of youtube-dnn"
> git rev-list --no-walk cf47815f124a15af1a9269c643007fb802bcaa5a # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8665169838282976786.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 782 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 34%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 37%]
tests/unit/tf/layers/test_queue.py .............. [ 39%]
tests/unit/tf/losses/test_losses.py ....................... [ 42%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 46%]
tests/unit/tf/models/test_base.py s..................... [ 48%]
tests/unit/tf/models/test_benchmark.py .. [ 49%]
tests/unit/tf/models/test_ranking.py .................................. [ 53%]
tests/unit/tf/models/test_retrieval.py ................................. [ 57%]
................................ [ 61%]
tests/unit/tf/outputs/test_base.py ..... [ 62%]
tests/unit/tf/outputs/test_classification.py ...... [ 63%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 65%]
tests/unit/tf/outputs/test_topk.py . [ 65%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 69%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 69%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 70%]
tests/unit/tf/transformers/test_block.py .............. [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 80%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 81%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileivrssc5b.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 770 passed, 12 skipped, 1266 warnings in 1377.18s (0:22:57) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins7229117064164471311.sh
Click to view CI Results
GitHub pull request #790 of commit e1375fa6fbefc1c8c2febeff7d18aecc7a84457d, no merge conflicts.
Running as SYSTEM
Setting status of e1375fa6fbefc1c8c2febeff7d18aecc7a84457d to PENDING with url https://10.20.13.93:8080/job/merlin_models/1504/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse e1375fa6fbefc1c8c2febeff7d18aecc7a84457d^{commit} # timeout=10
Checking out Revision e1375fa6fbefc1c8c2febeff7d18aecc7a84457d (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f e1375fa6fbefc1c8c2febeff7d18aecc7a84457d # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 2c9a1547a439af393f6d63e22485ed21cb6adfd5 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins12132649144774325459.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 782 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 9%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 18%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 21%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 24%]
tests/unit/tf/core/test_base.py .. [ 24%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 27%]
tests/unit/tf/core/test_index.py ... [ 27%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 28%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 29%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 29%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 29%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 29%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 29%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 29%]
[ 29%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 30%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 34%]
...... [ 35%]
tests/unit/tf/inputs/test_tabular.py .................. [ 37%]
tests/unit/tf/layers/test_queue.py .............. [ 39%]
tests/unit/tf/losses/test_losses.py ....................... [ 42%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 43%]
tests/unit/tf/metrics/test_metrics_topk.py ....................... [ 46%]
tests/unit/tf/models/test_base.py s..................... [ 48%]
tests/unit/tf/models/test_benchmark.py .. [ 49%]
tests/unit/tf/models/test_ranking.py .................................. [ 53%]
tests/unit/tf/models/test_retrieval.py ................................. [ 57%]
................................ [ 61%]
tests/unit/tf/outputs/test_base.py ..... [ 62%]
tests/unit/tf/outputs/test_classification.py ...... [ 63%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 65%]
tests/unit/tf/outputs/test_topk.py . [ 65%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 69%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 69%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 70%]
tests/unit/tf/transformers/test_block.py .............. [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 80%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 81%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py ........ [ 82%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 83%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 84%]
tests/unit/torch/test_dataset.py ......... [ 85%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 86%]
tests/unit/torch/features/test_continuous.py .. [ 86%]
tests/unit/torch/features/test_embedding.py .............. [ 88%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 90%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 24 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 3 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 8 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filesxuxy9xq.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 770 passed, 12 skipped, 1266 warnings in 1352.60s (0:22:32) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins7354302358713172747.sh
Click to view CI Results
GitHub pull request #790 of commit be11791b412cb7b04350cedcedef3aac31f6b914, no merge conflicts.
Running as SYSTEM
Setting status of be11791b412cb7b04350cedcedef3aac31f6b914 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1516/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse be11791b412cb7b04350cedcedef3aac31f6b914^{commit} # timeout=10
Checking out Revision be11791b412cb7b04350cedcedef3aac31f6b914 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f be11791b412cb7b04350cedcedef3aac31f6b914 # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 3ddbebe6ef77b3128fb76231440e823eb9dcb8a0 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins16207187231902685829.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 803 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 22%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 28%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 29%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
...... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 42%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 45%]
tests/unit/tf/models/test_base.py s....................... [ 48%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
...F.F........................FF [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py ........... [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
_______ test_two_tower_model_v2_with_custom_options[bpr-max-True-False] ________
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f1830845f40>
run_eagerly = False, logits_pop_logq_correction = True, loss = 'bpr-max'
@pytest.mark.parametrize("run_eagerly", [True, False])
@pytest.mark.parametrize("logits_pop_logq_correction", [True, False])
@pytest.mark.parametrize("loss", ["categorical_crossentropy", "bpr-max", "binary_crossentropy"])
def test_two_tower_model_v2_with_custom_options(
ecommerce_data: Dataset,
run_eagerly,
logits_pop_logq_correction,
loss,
):
from functools import partial
from tensorflow.keras import regularizers
from merlin.models.tf.outputs.base import DotProduct
from merlin.models.tf.transforms.bias import PopularityLogitsCorrection
from merlin.models.utils import schema_utils
data = ecommerce_data
data.schema = data.schema.select_by_name(["user_categories", "item_id"])
user_schema = data.schema.select_by_tag(Tags.USER)
item_schema = data.schema.select_by_tag(Tags.ITEM)
metrics = [
tf.keras.metrics.AUC(from_logits=True, name="auc"),
mm.RecallAt(5),
mm.RecallAt(10),
mm.MRRAt(10),
mm.NDCGAt(10),
]
post_logits = None
if logits_pop_logq_correction:
cardinalities = schema_utils.categorical_cardinalities(data.schema)
item_id_cardinalities = cardinalities[
data.schema.select_by_tag(Tags.ITEM_ID).column_names[0]
]
items_frequencies = tf.sort(
tf.random.uniform((item_id_cardinalities,), minval=0, maxval=1000, dtype=tf.int32)
)
post_logits = PopularityLogitsCorrection(
items_frequencies,
schema=data.schema,
)
user_inputs = mm.InputBlockV2(
user_schema,
categorical=mm.Embeddings(
user_schema.select_by_tag(Tags.CATEGORICAL),
infer_dim_fn=partial(schema_utils.infer_embedding_dim, multiplier=3.0),
embeddings_regularizer=tf.keras.regularizers.L2(1.0e-5),
),
)
tower_block = mm.MLPBlock(
[2],
activation="relu",
no_activation_last_layer=True,
dropout=0.1,
kernel_regularizer=regularizers.l2(1e-5),
bias_regularizer=regularizers.l2(1e-6),
)
query = mm.Encoder(user_inputs, tower_block)
item_inputs = mm.InputBlockV2(
item_schema,
categorical=mm.Embeddings(
item_schema.select_by_tag(Tags.CATEGORICAL),
infer_dim_fn=partial(schema_utils.infer_embedding_dim, multiplier=3.0),
embeddings_regularizer=tf.keras.regularizers.L2(1.0e-5),
),
)
candidate = mm.Encoder(item_inputs, tower_block.copy())
output = mm.ContrastiveOutput(
DotProduct(),
logits_temperature=0.1,
post=post_logits,
negative_samplers="in-batch",
schema=data.schema.select_by_tag(Tags.ITEM_ID),
)
model = mm.TwoTowerModelV2(
query,
candidate,
outputs=output,
)
model.compile(optimizer="adam", run_eagerly=run_eagerly, loss=loss, metrics=metrics)
losses = model.fit(data, batch_size=50, epochs=1, steps_per_epoch=1)
tests/unit/tf/models/test_retrieval.py:480:
merlin/models/tf/models/base.py:831: in fit
out = super().fit(**fit_kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1409: in fit
tmp_logs = self.train_function(iterator)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:915: in call
result = self.call(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:963: in call
self.initialize(args, kwds, add_initializers_to=initializers)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:785: in initialize
self.stateful_fn.get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2480: in get_concrete_function_internal_garbage_collected
graph_function, _ = self.maybe_define_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2711: in maybe_define_function
graph_function = self.create_graph_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2627: in create_graph_function
func_graph_module.func_graph_from_py_func(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1141: in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:677: in wrapped_fn
out = weak_wrapped_fn().wrapped(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1127: in autograph_handler
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1116: in autograph_handler
return autograph.converted_call(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_file2qk1ejax.py:15: in tf__train_function
retval = ag.converted_call(ag.ld(step_function), (ag.ld(self), ag.ld(iterator)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1040: in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:1312: in run
return self.extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2888: in call_for_each_replica
return self.call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:3689: in call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:458: in call_unconverted
return f(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1030: in run_step
outputs = model.train_step(data)
merlin/models/tf/models/base.py:653: in train_step
loss = self.compute_loss(x, outputs.targets, outputs.predictions, outputs.sample_weight)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:948: in compute_loss
return self.compiled_loss(
/usr/local/lib/python3.8/dist-packages/keras/engine/compile_utils.py:201: in call
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
merlin/models/tf/losses/pairwise.py:56: in call
loss = super().call(y_true, y_pred, sample_weight)
/usr/local/lib/python3.8/dist-packages/keras/losses.py:139: in call
losses = call_fn(y_true, y_pred)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_fileaspphs_i.py:11: in tf__call
(positives_scores, negatives_scores, valid_rows_with_positive_mask) = ag.converted_call(ag.ld(self).separate_positives_negatives_scores, (ag.ld(y_true), ag.ld(y_pred)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:441: in converted_call
result = converted_f(*effective_args)
/tmp/autograph_generated_filexu7tbtc3.py:15: in tf___separate_positives_negatives_scores
y_pred_valid_rows = ag.converted_call(ag.ld(tf).boolean_mask, (ag.ld(y_pred), ag.ld(valid_rows_with_positive_mask)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:331: in converted_call
return _call_unconverted(f, args, kwargs, options, False)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in _call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:1978: in boolean_mask_v2
return boolean_mask(tensor, mask, name, axis)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
tensor = <tf.Tensor 'retrieval_model_v2/contrastive_output/Squeeze_3:0' shape= dtype=float32>
mask = <tf.Tensor 'BPRmaxLoss/Cast:0' shape= dtype=bool>
name = 'boolean_mask', axis = None
@tf_export(v1=["boolean_mask"])
@dispatch.add_dispatch_support
def boolean_mask(tensor, mask, name="boolean_mask", axis=None):
"""Apply boolean mask to tensor.
Numpy equivalent is `tensor[mask]`.
In general, `0 < dim(mask) = K <= dim(tensor)`, and `mask`'s shape must match
the first K dimensions of `tensor`'s shape. We then have:
`boolean_mask(tensor, mask)[i, j1,...,jd] = tensor[i1,...,iK,j1,...,jd]`
where `(i1,...,iK)` is the ith `True` entry of `mask` (row-major order).
The `axis` could be used with `mask` to indicate the axis to mask from.
In that case, `axis + dim(mask) <= dim(tensor)` and `mask`'s shape must match
the first `axis + dim(mask)` dimensions of `tensor`'s shape.
See also: `tf.ragged.boolean_mask`, which can be applied to both dense and
ragged tensors, and can be used if you need to preserve the masked dimensions
of `tensor` (rather than flattening them, as `tf.boolean_mask` does).
Examples:
```python
# 1-D example
tensor = [0, 1, 2, 3]
mask = np.array([True, False, True, False])
tf.boolean_mask(tensor, mask) # [0, 2]
# 2-D example
tensor = [[1, 2], [3, 4], [5, 6]]
mask = np.array([True, False, True])
tf.boolean_mask(tensor, mask) # [[1, 2], [5, 6]]
```
Args:
tensor: N-D Tensor.
mask: K-D boolean Tensor, K <= N and K must be known statically.
name: A name for this operation (optional).
axis: A 0-D int Tensor representing the axis in `tensor` to mask from. By
default, axis is 0 which will mask from the first dimension. Otherwise K +
axis <= N.
Returns:
(N-K+1)-dimensional tensor populated by entries in `tensor` corresponding
to `True` values in `mask`.
Raises:
ValueError: If shapes do not conform.
"""
def _apply_mask_1d(reshaped_tensor, mask, axis=None):
"""Mask tensor along dimension 0 with a 1-D mask."""
indices = squeeze(where_v2(mask), axis=[1])
return gather(reshaped_tensor, indices, axis=axis)
with ops.name_scope(name, values=[tensor, mask]):
tensor = ops.convert_to_tensor(tensor, name="tensor")
mask = ops.convert_to_tensor(mask, name="mask")
shape_mask = mask.get_shape()
ndims_mask = shape_mask.ndims
shape_tensor = tensor.get_shape()
if ndims_mask == 0:
raise ValueError("mask cannot be scalar.")
if ndims_mask is None:
raise ValueError(
"Number of mask dimensions must be specified, even if some dimensions"
" are None. E.g. shape=[None] is ok, but shape=None is not.")
E ValueError: in user code:
E
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function *
E return step_function(self, iterator)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/losses/pairwise.py", line 75, in call *
E (
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/losses/pairwise.py", line 152, in _separate_positives_negatives_scores *
E y_pred_valid_rows = tf.boolean_mask(y_pred, valid_rows_with_positive_mask)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 1978, in boolean_mask_v2
E return boolean_mask(tensor, mask, name, axis)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 1891, in boolean_mask
E raise ValueError(
E
E ValueError: Number of mask dimensions must be specified, even if some dimensions are None. E.g. shape=[None] is ok, but shape=None is not.
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:1891: ValueError
_______ test_two_tower_model_v2_with_custom_options[bpr-max-False-False] _______
ecommerce_data = <merlin.io.dataset.Dataset object at 0x7f186c36e310>
run_eagerly = False, logits_pop_logq_correction = False, loss = 'bpr-max'
@pytest.mark.parametrize("run_eagerly", [True, False])
@pytest.mark.parametrize("logits_pop_logq_correction", [True, False])
@pytest.mark.parametrize("loss", ["categorical_crossentropy", "bpr-max", "binary_crossentropy"])
def test_two_tower_model_v2_with_custom_options(
ecommerce_data: Dataset,
run_eagerly,
logits_pop_logq_correction,
loss,
):
from functools import partial
from tensorflow.keras import regularizers
from merlin.models.tf.outputs.base import DotProduct
from merlin.models.tf.transforms.bias import PopularityLogitsCorrection
from merlin.models.utils import schema_utils
data = ecommerce_data
data.schema = data.schema.select_by_name(["user_categories", "item_id"])
user_schema = data.schema.select_by_tag(Tags.USER)
item_schema = data.schema.select_by_tag(Tags.ITEM)
metrics = [
tf.keras.metrics.AUC(from_logits=True, name="auc"),
mm.RecallAt(5),
mm.RecallAt(10),
mm.MRRAt(10),
mm.NDCGAt(10),
]
post_logits = None
if logits_pop_logq_correction:
cardinalities = schema_utils.categorical_cardinalities(data.schema)
item_id_cardinalities = cardinalities[
data.schema.select_by_tag(Tags.ITEM_ID).column_names[0]
]
items_frequencies = tf.sort(
tf.random.uniform((item_id_cardinalities,), minval=0, maxval=1000, dtype=tf.int32)
)
post_logits = PopularityLogitsCorrection(
items_frequencies,
schema=data.schema,
)
user_inputs = mm.InputBlockV2(
user_schema,
categorical=mm.Embeddings(
user_schema.select_by_tag(Tags.CATEGORICAL),
infer_dim_fn=partial(schema_utils.infer_embedding_dim, multiplier=3.0),
embeddings_regularizer=tf.keras.regularizers.L2(1.0e-5),
),
)
tower_block = mm.MLPBlock(
[2],
activation="relu",
no_activation_last_layer=True,
dropout=0.1,
kernel_regularizer=regularizers.l2(1e-5),
bias_regularizer=regularizers.l2(1e-6),
)
query = mm.Encoder(user_inputs, tower_block)
item_inputs = mm.InputBlockV2(
item_schema,
categorical=mm.Embeddings(
item_schema.select_by_tag(Tags.CATEGORICAL),
infer_dim_fn=partial(schema_utils.infer_embedding_dim, multiplier=3.0),
embeddings_regularizer=tf.keras.regularizers.L2(1.0e-5),
),
)
candidate = mm.Encoder(item_inputs, tower_block.copy())
output = mm.ContrastiveOutput(
DotProduct(),
logits_temperature=0.1,
post=post_logits,
negative_samplers="in-batch",
schema=data.schema.select_by_tag(Tags.ITEM_ID),
)
model = mm.TwoTowerModelV2(
query,
candidate,
outputs=output,
)
model.compile(optimizer="adam", run_eagerly=run_eagerly, loss=loss, metrics=metrics)
losses = model.fit(data, batch_size=50, epochs=1, steps_per_epoch=1)
tests/unit/tf/models/test_retrieval.py:480:
merlin/models/tf/models/base.py:831: in fit
out = super().fit(**fit_kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1409: in fit
tmp_logs = self.train_function(iterator)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:915: in call
result = self.call(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:963: in call
self.initialize(args, kwds, add_initializers_to=initializers)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:785: in initialize
self.stateful_fn.get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2480: in get_concrete_function_internal_garbage_collected
graph_function, _ = self.maybe_define_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2711: in maybe_define_function
graph_function = self.create_graph_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2627: in create_graph_function
func_graph_module.func_graph_from_py_func(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1141: in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:677: in wrapped_fn
out = weak_wrapped_fn().wrapped(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1127: in autograph_handler
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1116: in autograph_handler
return autograph.converted_call(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_file2qk1ejax.py:15: in tf__train_function
retval = ag.converted_call(ag.ld(step_function), (ag.ld(self), ag.ld(iterator)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1040: in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:1312: in run
return self.extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2888: in call_for_each_replica
return self.call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:3689: in call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:458: in call_unconverted
return f(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1030: in run_step
outputs = model.train_step(data)
merlin/models/tf/models/base.py:653: in train_step
loss = self.compute_loss(x, outputs.targets, outputs.predictions, outputs.sample_weight)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:948: in compute_loss
return self.compiled_loss(
/usr/local/lib/python3.8/dist-packages/keras/engine/compile_utils.py:201: in call
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
merlin/models/tf/losses/pairwise.py:56: in call
loss = super().call(y_true, y_pred, sample_weight)
/usr/local/lib/python3.8/dist-packages/keras/losses.py:139: in call
losses = call_fn(y_true, y_pred)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_fileaspphs_i.py:11: in tf__call
(positives_scores, negatives_scores, valid_rows_with_positive_mask) = ag.converted_call(ag.ld(self).separate_positives_negatives_scores, (ag.ld(y_true), ag.ld(y_pred)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:441: in converted_call
result = converted_f(*effective_args)
/tmp/autograph_generated_filexu7tbtc3.py:15: in tf___separate_positives_negatives_scores
y_pred_valid_rows = ag.converted_call(ag.ld(tf).boolean_mask, (ag.ld(y_pred), ag.ld(valid_rows_with_positive_mask)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:331: in converted_call
return _call_unconverted(f, args, kwargs, options, False)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in _call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:1978: in boolean_mask_v2
return boolean_mask(tensor, mask, name, axis)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
tensor = <tf.Tensor 'retrieval_model_v2/contrastive_output/Squeeze_3:0' shape= dtype=float32>
mask = <tf.Tensor 'BPRmaxLoss/Cast:0' shape= dtype=bool>
name = 'boolean_mask', axis = None
@tf_export(v1=["boolean_mask"])
@dispatch.add_dispatch_support
def boolean_mask(tensor, mask, name="boolean_mask", axis=None):
"""Apply boolean mask to tensor.
Numpy equivalent is `tensor[mask]`.
In general, `0 < dim(mask) = K <= dim(tensor)`, and `mask`'s shape must match
the first K dimensions of `tensor`'s shape. We then have:
`boolean_mask(tensor, mask)[i, j1,...,jd] = tensor[i1,...,iK,j1,...,jd]`
where `(i1,...,iK)` is the ith `True` entry of `mask` (row-major order).
The `axis` could be used with `mask` to indicate the axis to mask from.
In that case, `axis + dim(mask) <= dim(tensor)` and `mask`'s shape must match
the first `axis + dim(mask)` dimensions of `tensor`'s shape.
See also: `tf.ragged.boolean_mask`, which can be applied to both dense and
ragged tensors, and can be used if you need to preserve the masked dimensions
of `tensor` (rather than flattening them, as `tf.boolean_mask` does).
Examples:
```python
# 1-D example
tensor = [0, 1, 2, 3]
mask = np.array([True, False, True, False])
tf.boolean_mask(tensor, mask) # [0, 2]
# 2-D example
tensor = [[1, 2], [3, 4], [5, 6]]
mask = np.array([True, False, True])
tf.boolean_mask(tensor, mask) # [[1, 2], [5, 6]]
```
Args:
tensor: N-D Tensor.
mask: K-D boolean Tensor, K <= N and K must be known statically.
name: A name for this operation (optional).
axis: A 0-D int Tensor representing the axis in `tensor` to mask from. By
default, axis is 0 which will mask from the first dimension. Otherwise K +
axis <= N.
Returns:
(N-K+1)-dimensional tensor populated by entries in `tensor` corresponding
to `True` values in `mask`.
Raises:
ValueError: If shapes do not conform.
"""
def _apply_mask_1d(reshaped_tensor, mask, axis=None):
"""Mask tensor along dimension 0 with a 1-D mask."""
indices = squeeze(where_v2(mask), axis=[1])
return gather(reshaped_tensor, indices, axis=axis)
with ops.name_scope(name, values=[tensor, mask]):
tensor = ops.convert_to_tensor(tensor, name="tensor")
mask = ops.convert_to_tensor(mask, name="mask")
shape_mask = mask.get_shape()
ndims_mask = shape_mask.ndims
shape_tensor = tensor.get_shape()
if ndims_mask == 0:
raise ValueError("mask cannot be scalar.")
if ndims_mask is None:
raise ValueError(
"Number of mask dimensions must be specified, even if some dimensions"
" are None. E.g. shape=[None] is ok, but shape=None is not.")
E ValueError: in user code:
E
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function *
E return step_function(self, iterator)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/losses/pairwise.py", line 75, in call *
E (
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/losses/pairwise.py", line 152, in _separate_positives_negatives_scores *
E y_pred_valid_rows = tf.boolean_mask(y_pred, valid_rows_with_positive_mask)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 1978, in boolean_mask_v2
E return boolean_mask(tensor, mask, name, axis)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 1891, in boolean_mask
E raise ValueError(
E
E ValueError: Number of mask dimensions must be specified, even if some dimensions are None. E.g. shape=[None] is ok, but shape=None is not.
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:1891: ValueError
____________________ test_youtube_dnn_topk_evaluation[True] ____________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f1821fb0fd0>
run_eagerly = True
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_youtube_dnn_topk_evaluation(sequence_testing_data: Dataset, run_eagerly):
to_remove = (
sequence_testing_data.schema.select_by_tag(Tags.SEQUENCE)
.select_by_tag(Tags.CONTINUOUS)
.column_names
)
sequence_testing_data.schema = sequence_testing_data.schema.excluding_by_name(to_remove)
seq_schema = sequence_testing_data.schema.select_by_tag(Tags.SEQUENCE)
target = sequence_testing_data.schema.select_by_tag(Tags.ITEM_ID).column_names[0]
predict_next = mm.SequencePredictLast(schema=seq_schema, target=target)
model = mm.YoutubeDNNRetrievalModelV2(
schema=sequence_testing_data.schema, top_block=mm.MLPBlock([32]), num_sampled=1000
)
dataloader = mm.Loader(sequence_testing_data, batch_size=50, transform=predict_next)
model, _ = testing_utils.model_test(model, dataloader, reload_model=False)
# Top-K evaluation
topk_model = model.to_top_k_encoder()
topk_model.compile(run_eagerly=run_eagerly)
metrics = topk_model.evaluate(dataloader, return_dict=True)
tests/unit/tf/models/test_retrieval.py:859:
merlin/models/tf/models/base.py:876: in evaluate
out = super().evaluate(
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1756: in evaluate
tmp_logs = self.test_function(iterator)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1557: in test_function
return step_function(self, iterator)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1546: in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:1312: in run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2888: in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:3689: in _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:595: in wrapper
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1535: in run_step
outputs = model.test_step(data)
merlin/models/tf/models/base.py:679: in test_step
outputs = self.call_train_test(x, y, sample_weight=sample_weight, testing=True)
merlin/models/tf/models/base.py:515: in call_train_test
forward = self(
merlin/models/tf/core/encoder.py:166: in call
return super().call(inputs, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:490: in call
return super().call(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:96: in error_handler
raise e
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
merlin/models/tf/core/encoder.py:143: in call
return combinators.call_sequentially(
merlin/models/tf/core/combinators.py:819: in call_sequentially
outputs = call_layer(layer, outputs, **kwargs)
merlin/models/tf/utils/tf_utils.py:433: in call_layer
return layer(inputs, *args, **filtered_kwargs)
merlin/models/tf/outputs/base.py:136: in call
outputs = super(ModelOutput, self).call(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:96: in error_handler
raise e
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
merlin/models/tf/outputs/base.py:114: in call
return tf_utils.call_layer(
merlin/models/tf/utils/tf_utils.py:433: in call_layer
return layer(inputs, *args, **filtered_kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:146: in error_handler
raise new_e.with_traceback(e.traceback) from None
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
merlin/models/tf/outputs/topk.py:203: in call
targets = tf.cast(tf.squeeze(targets), tf.int32)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:4639: in squeeze_v2
return squeeze(input, axis, name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py:561: in new_func
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:4587: in squeeze
return gen_array_ops.squeeze(input, axis, name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py:10363: in squeeze
return squeeze_eager_fallback(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py:10398: in squeeze_eager_fallback
_attr_T, (input,) = _execute.args_to_matching_eager([input], ctx, [])
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py:264: in args_to_matching_eager
tensor = ops.convert_to_tensor(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:279: in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:304: in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
value = {'item_id_seq': <tf.Tensor: shape=(50,), dtype=int64, numpy=
array([ 2, 461, 13, 66, 12, 7, 21, 8, 9, 29, ... 16, 26, 170, 62, 32, 4, 37, 5, 40, 22, 26,
17, 23, 16, 27, 24, 37, 27, 15, 52, 16, 23])>}
ctx = <tensorflow.python.eager.context.Context object at 0x7f1a4e3f2fd0>
dtype = None
def convert_to_eager_tensor(value, ctx, dtype=None):
"""Converts the given `value` to an `EagerTensor`.
Note that this function could return cached copies of created constants for
performance reasons.
Args:
value: value to convert to EagerTensor.
ctx: value of context.context().
dtype: optional desired dtype of the converted EagerTensor.
Returns:
EagerTensor created from value.
Raises:
TypeError: if `dtype` is not compatible with the type of t.
"""
if isinstance(value, ops.EagerTensor):
if dtype is not None and value.dtype != dtype:
raise TypeError(f"Expected tensor {value} with dtype {dtype!r}, but got "
f"dtype {value.dtype!r}.")
return value
if dtype is not None:
try:
dtype = dtype.as_datatype_enum
except AttributeError:
dtype = dtypes.as_dtype(dtype).as_datatype_enum
ctx.ensure_initialized()
return ops.EagerTensor(value, ctx.device_name, dtype)
E ValueError: Exception encountered when calling layer "brute_force" (type BruteForce).
E
E Attempt to convert a value ({'item_id_seq': <tf.Tensor: shape=(50,), dtype=int64, numpy=
E array([ 2, 461, 13, 66, 12, 7, 21, 8, 9, 29, 4, 12, 52,
E 59, 40, 8, 15, 16, 22, 8, 15, 116, 22, 62, 4, 36,
E 30, 31, 16, 26, 170, 62, 32, 4, 37, 5, 40, 22, 26,
E 17, 23, 16, 27, 24, 37, 27, 15, 52, 16, 23])>}) with an unsupported type (<class 'dict'>) to a Tensor.
E
E Call arguments received by layer "brute_force" (type BruteForce):
E • inputs=tf.Tensor(shape=(50, 32), dtype=float32)
E • targets={'item_id_seq': 'tf.Tensor(shape=(50,), dtype=int64)'}
E • testing=True
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102: ValueError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 6.9070 - recall_at_10: 0.0000e+00 - mrr_at_10: 0.0000e+00 - ndcg_at_10: 0.0000e+00 - map_at_10: 0.0000e+00 - precision_at_10: 0.0000e+00 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 957ms/step - loss: 6.9070 - recall_at_10: 0.0000e+00 - mrr_at_10: 0.0000e+00 - ndcg_at_10: 0.0000e+00 - map_at_10: 0.0000e+00 - precision_at_10: 0.0000e+00 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:Gradients do not exist for variables ['retrieval_model_v2/output_layer_bias:0'] when minimizing the loss. If you're using model.compile(), did you forget to provide a lossargument?
------------------------------ Captured log call -------------------------------
WARNING tensorflow:utils.py:76 Gradients do not exist for variables ['retrieval_model_v2/output_layer_bias:0'] when minimizing the loss. If you're using model.compile(), did you forget to provide a lossargument?
___________________ test_youtube_dnn_topk_evaluation[False] ____________________
sequence_testing_data = <merlin.io.dataset.Dataset object at 0x7f18205d4fd0>
run_eagerly = False
@pytest.mark.parametrize("run_eagerly", [True, False])
def test_youtube_dnn_topk_evaluation(sequence_testing_data: Dataset, run_eagerly):
to_remove = (
sequence_testing_data.schema.select_by_tag(Tags.SEQUENCE)
.select_by_tag(Tags.CONTINUOUS)
.column_names
)
sequence_testing_data.schema = sequence_testing_data.schema.excluding_by_name(to_remove)
seq_schema = sequence_testing_data.schema.select_by_tag(Tags.SEQUENCE)
target = sequence_testing_data.schema.select_by_tag(Tags.ITEM_ID).column_names[0]
predict_next = mm.SequencePredictLast(schema=seq_schema, target=target)
model = mm.YoutubeDNNRetrievalModelV2(
schema=sequence_testing_data.schema, top_block=mm.MLPBlock([32]), num_sampled=1000
)
dataloader = mm.Loader(sequence_testing_data, batch_size=50, transform=predict_next)
model, _ = testing_utils.model_test(model, dataloader, reload_model=False)
# Top-K evaluation
topk_model = model.to_top_k_encoder()
topk_model.compile(run_eagerly=run_eagerly)
metrics = topk_model.evaluate(dataloader, return_dict=True)
tests/unit/tf/models/test_retrieval.py:859:
merlin/models/tf/models/base.py:876: in evaluate
out = super().evaluate(
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1756: in evaluate
tmp_logs = self.test_function(iterator)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:915: in call
result = self.call(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:963: in call
self.initialize(args, kwds, add_initializers_to=initializers)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:785: in initialize
self.stateful_fn.get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2480: in get_concrete_function_internal_garbage_collected
graph_function, _ = self.maybe_define_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2711: in maybe_define_function
graph_function = self.create_graph_function(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:2627: in create_graph_function
func_graph_module.func_graph_from_py_func(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1141: in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py:677: in wrapped_fn
out = weak_wrapped_fn().wrapped(*args, **kwds)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1127: in autograph_handler
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py:1116: in autograph_handler
return autograph.converted_call(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_file9we65ozv.py:15: in tf__test_function
retval = ag.converted_call(ag.ld(step_function), (ag.ld(self), ag.ld(iterator)), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1546: in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:1312: in run
return self.extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2888: in call_for_each_replica
return self.call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:3689: in call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:458: in call_unconverted
return f(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:1535: in run_step
outputs = model.test_step(data)
merlin/models/tf/models/base.py:679: in test_step
outputs = self.call_train_test(x, y, sample_weight=sample_weight, testing=True)
merlin/models/tf/models/base.py:515: in call_train_test
forward = self(
merlin/models/tf/core/encoder.py:166: in call
return super().call(inputs, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py:490: in call
return super().call(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:146: in error_handler
raise new_e.with_traceback(e.traceback) from None
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_file6nm79smx.py:12: in tf__call
retval = ag.converted_call(ag.ld(combinators).call_sequentially, (ag.converted_call(ag.ld(list), (ag.ld(self).to_call,), None, fscope),), dict(inputs=ag.ld(inputs), features=ag__.ld(inputs), targets=ag__.ld(targets), training=ag__.ld(training), testing=ag__.ld(testing)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filed3n_lrfp.py:25: in tf__call_sequentially
ag.for_stmt(ag__.ld(layers), None, loop_body, get_state, set_state, ('outputs',), {'iterate_names': 'layer'})
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py:449: in for_stmt
py_for_stmt(iter, extra_test, body, None, None)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py:498: in py_for_stmt
body(target)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py:464: in protected_body
original_body(protected_iter)
/tmp/autograph_generated_filed3n_lrfp.py:23: in loop_body
outputs = ag.converted_call(ag_.ld(call_layer), (ag__.ld(layer), ag__.ld(outputs)), dict(**ag__.ld(kwargs)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filety1wsrps.py:50: in tf__call_layer
retval = ag_.converted_call(ag__.ld(layer), ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(filtered_kwargs)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filel5di3d_t.py:27: in tf____call
outputs = ag__.converted_call(ag__.converted_call(ag__.ld(super), (ag__.ld(ModelOutput), ag__.ld(self)), None, fscope).call, ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(kwargs)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:331: in converted_call
return call_unconverted(f, args, kwargs, options, False)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:458: in call_unconverted
return f(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:146: in error_handler
raise new_e.with_traceback(e.traceback) from None
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filefm6ho632.py:12: in tf__call
retval = ag.converted_call(ag_.ld(tf_utils).call_layer, (ag__.ld(self).to_call, ag__.ld(inputs)), dict(training=ag__.ld(training), testing=ag__.ld(testing), **ag__.ld(kwargs)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filety1wsrps.py:50: in tf__call_layer
retval = ag_.converted_call(ag__.ld(layer), ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(filtered_kwargs)), fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:377: in converted_call
return call_unconverted(f, args, kwargs, options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:458: in call_unconverted
return f(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:60: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py:1014: in call
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:146: in error_handler
raise new_e.with_traceback(e.traceback) from None
/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:92: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:692: in wrapper
raise e.ag_error_metadata.to_exception(e)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:689: in wrapper
return converted_call(f, args, kwargs, options=options)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:439: in converted_call
result = converted_f(*effective_args, **kwargs)
/tmp/autograph_generated_filear01qoy8.py:56: in tf__call
ag.if_stmt(ag.ld(testing), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval_', 'targets'), 2)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1341: in if_stmt
py_if_stmt(cond, body, orelse)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1394: in py_if_stmt
return body() if cond else orelse()
/tmp/autograph_generated_filear01qoy8.py:38: in if_body_1
targets = ag.converted_call(ag.ld(tf).cast, (ag__.converted_call(ag__.ld(tf).squeeze, (ag__.ld(targets),), None, fscope), ag__.ld(tf).int32), None, fscope)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:331: in converted_call
return _call_unconverted(f, args, kwargs, options, False)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:459: in _call_unconverted
return f(*args)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:4639: in squeeze_v2
return squeeze(input, axis, name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:141: in error_handler
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: in op_dispatch_handler
return dispatch_target(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py:561: in new_func
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py:4587: in squeeze
return gen_array_ops.squeeze(input, axis, name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py:10375: in squeeze
_, _, _op, _outputs = _op_def_library._apply_op_helper(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py:779: in _apply_op_helper
_ExtractInputsAndAttrs(op_type_name, op_def, allowed_list_attr_map,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py:560: in _ExtractInputsAndAttrs
raise err
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py:552: in _ExtractInputsAndAttrs
values = ops.convert_to_tensor(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py:183: in wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:1640: in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:343: in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:267: in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:284: in _constant_impl
tensor_util.make_tensor_proto(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py:453: in make_tensor_proto
_AssertCompatible(values, dtype)
values = {'item_id_seq': <tf.Tensor 'IteratorGetNext:5' shape=(None,) dtype=int64>}
dtype = None
def _AssertCompatible(values, dtype):
if dtype is None:
fn = _check_not_tensor
else:
try:
fn = _TF_TO_IS_OK[dtype]
except KeyError:
# There isn't a specific fn, so we try to do the best possible.
if dtype.is_integer:
fn = _check_int
elif dtype.is_floating:
fn = _check_float
elif dtype.is_complex:
fn = _check_complex
elif dtype.is_quantized:
fn = _check_quantized
else:
fn = _check_not_tensor
try:
fn(values)
except ValueError as e:
[mismatch] = e.args
if dtype is None:
raise TypeError("Expected any non-tensor type, but got a tensor instead.")
E TypeError: in user code:
E
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1557, in test_function *
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1546, in step_function **
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py", line 1312, in run
E return self.extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py", line 2888, in call_for_each_replica
E return self.call_for_each_replica(fn, args, kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py", line 3689, in call_for_each_replica
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1535, in run_step **
E outputs = model.test_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 679, in test_step
E outputs = self.call_train_test(x, y, sample_weight=sample_weight, testing=True)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 515, in call_train_test
E forward = self(
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/encoder.py", line 166, in call
E return super().call(inputs, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 490, in call
E return super().call(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_file6nm79smx.py", line 12, in tf__call **
E retval = ag.converted_call(ag.ld(combinators).call_sequentially, (ag__.converted_call(ag__.ld(list), (ag__.ld(self).to_call,), None, fscope),), dict(inputs=ag__.ld(inputs), features=ag__.ld(inputs), targets=ag__.ld(targets), training=ag__.ld(training), testing=ag__.ld(testing)), fscope)
E File "/tmp/autograph_generated_filed3n_lrfp.py", line 25, in tf__call_sequentially **
E ag.for_stmt(ag__.ld(layers), None, loop_body, get_state, set_state, ('outputs',), {'iterate_names': 'layer'})
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 449, in for_stmt
E py_for_stmt(iter, extra_test, body, None, None)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 498, in py_for_stmt
E body(target)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 464, in protected_body
E original_body(protected_iter)
E File "/tmp/autograph_generated_filed3n_lrfp.py", line 23, in loop_body
E outputs = ag.converted_call(ag_.ld(call_layer), (ag__.ld(layer), ag__.ld(outputs)), dict(**ag__.ld(kwargs)), fscope)
E File "/tmp/autograph_generated_filety1wsrps.py", line 50, in tf__call_layer **
E retval = ag_.converted_call(ag__.ld(layer), ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(filtered_kwargs)), fscope)
E File "/tmp/autograph_generated_filel5di3d_t.py", line 27, in tf____call **
E outputs = ag__.converted_call(ag__.converted_call(ag__.ld(super), (ag__.ld(ModelOutput), ag__.ld(self)), None, fscope).call, ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(kwargs)), fscope)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_filefm6ho632.py", line 12, in tf__call **
E retval = ag_.converted_call(ag__.ld(tf_utils).call_layer, (ag__.ld(self).to_call, ag__.ld(inputs)), dict(training=ag__.ld(training), testing=ag__.ld(testing), **ag__.ld(kwargs)), fscope)
E File "/tmp/autograph_generated_filety1wsrps.py", line 50, in tf__call_layer **
E retval = ag_.converted_call(ag__.ld(layer), ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(filtered_kwargs)), fscope)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_filear01qoy8.py", line 56, in tf__call **
E ag.if_stmt(ag__.ld(testing), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval_', 'targets'), 2)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
E py_if_stmt(cond, body, orelse)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in py_if_stmt
E return body() if cond else orelse()
E File "/tmp/autograph_generated_filear01qoy8.py", line 38, in if_body_1
E targets = ag.converted_call(ag.ld(tf).cast, (ag__.converted_call(ag__.ld(tf).squeeze, (ag__.ld(targets),), None, fscope), ag__.ld(tf).int32), None, fscope)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4639, in squeeze_v2
E return squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4587, in squeeze
E return gen_array_ops.squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 10375, in squeeze
E , , op, outputs = op_def_library.apply_op_helper(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 779, in apply_op_helper
E ExtractInputsAndAttrs(op_type_name, op_def, allowed_list_attr_map,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 560, in ExtractInputsAndAttrs
E raise err
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 552, in ExtractInputsAndAttrs
E values = ops.convert_to_tensor(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py", line 183, in wrapped
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1640, in convert_to_tensor
E ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 343, in constant_tensor_conversion_function
E return constant(v, dtype=dtype, name=name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 267, in constant
E return constant_impl(value, dtype, shape, name, verify_shape=False,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 284, in constant_impl
E tensor_util.make_tensor_proto(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 453, in make_tensor_proto
E AssertCompatible(values, dtype)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 330, in AssertCompatible
E raise TypeError("Expected any non-tensor type, but got a tensor instead.")
E
E TypeError: Exception encountered when calling layer "top_k_encoder_1" (type TopKEncoder).
E
E in user code:
E
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/encoder.py", line 144, in call *
E list(self.to_call),
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/combinators.py", line 819, in call_sequentially *
E outputs = call_layer(layer, outputs, **kwargs)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py", line 433, in call_layer *
E return layer(inputs, *args, **filtered_kwargs)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/outputs/base.py", line 136, in call *
E outputs = super(ModelOutput, self).call(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_filefm6ho632.py", line 12, in tf__call **
E retval = ag.converted_call(ag.ld(tf_utils).call_layer, (ag.ld(self).to_call, ag.ld(inputs)), dict(training=ag.ld(training), testing=ag.ld(testing), **ag.ld(kwargs)), fscope)
E File "/tmp/autograph_generated_filety1wsrps.py", line 50, in tf__call_layer **
E retval = ag.converted_call(ag_.ld(layer), ((ag__.ld(inputs),) + tuple(ag__.ld(args))), dict(**ag__.ld(filtered_kwargs)), fscope)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_filear01qoy8.py", line 56, in tf__call **
E ag.if_stmt(ag__.ld(testing), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval_', 'targets'), 2)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
E py_if_stmt(cond, body, orelse)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in py_if_stmt
E return body() if cond else orelse()
E File "/tmp/autograph_generated_filear01qoy8.py", line 38, in if_body_1
E targets = ag.converted_call(ag.ld(tf).cast, (ag__.converted_call(ag__.ld(tf).squeeze, (ag__.ld(targets),), None, fscope), ag__.ld(tf).int32), None, fscope)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4639, in squeeze_v2
E return squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4587, in squeeze
E return gen_array_ops.squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 10375, in squeeze
E _, _, _op, _outputs = op_def_library.apply_op_helper(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 779, in apply_op_helper
E ExtractInputsAndAttrs(op_type_name, op_def, allowed_list_attr_map,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 560, in ExtractInputsAndAttrs
E raise err
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 552, in ExtractInputsAndAttrs
E values = ops.convert_to_tensor(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py", line 183, in wrapped
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1640, in convert_to_tensor
E ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 343, in constant_tensor_conversion_function
E return constant(v, dtype=dtype, name=name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 267, in constant
E return constant_impl(value, dtype, shape, name, verify_shape=False,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 284, in constant_impl
E tensor_util.make_tensor_proto(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 453, in make_tensor_proto
E AssertCompatible(values, dtype)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 330, in AssertCompatible
E raise TypeError("Expected any non-tensor type, but got a tensor instead.")
E
E TypeError: Exception encountered when calling layer "top_k_output" (type TopKOutput).
E
E in user code:
E
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/outputs/base.py", line 115, in call *
E self.to_call, inputs, training=training, testing=testing, **kwargs
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py", line 433, in call_layer *
E return layer(inputs, *args, **filtered_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 60, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in call
E outputs = call_fn(inputs, *args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 146, in error_handler
E raise new_e.with_traceback(e.traceback) from None
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
E return fn(*args, **kwargs)
E File "/tmp/autograph_generated_filear01qoy8.py", line 56, in tf__call **
E ag.if_stmt(ag.ld(testing), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval', 'targets'), 2)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
E py_if_stmt(cond, body, orelse)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in py_if_stmt
E return body() if cond else orelse()
E File "/tmp/autograph_generated_filear01qoy8.py", line 38, in if_body_1
E targets = ag.converted_call(ag.ld(tf).cast, (ag.converted_call(ag.ld(tf).squeeze, (ag.ld(targets),), None, fscope), ag.ld(tf).int32), None, fscope)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4639, in squeeze_v2
E return squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4587, in squeeze
E return gen_array_ops.squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 10375, in squeeze
E _, _, _op, _outputs = _op_def_library._apply_op_helper(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 779, in _apply_op_helper
E _ExtractInputsAndAttrs(op_type_name, op_def, allowed_list_attr_map,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 560, in _ExtractInputsAndAttrs
E raise err
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 552, in _ExtractInputsAndAttrs
E values = ops.convert_to_tensor(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py", line 183, in wrapped
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1640, in convert_to_tensor
E ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 343, in _constant_tensor_conversion_function
E return constant(v, dtype=dtype, name=name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 267, in constant
E return _constant_impl(value, dtype, shape, name, verify_shape=False,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
E tensor_util.make_tensor_proto(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 453, in make_tensor_proto
E _AssertCompatible(values, dtype)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 330, in _AssertCompatible
E raise TypeError("Expected any non-tensor type, but got a tensor instead.")
E
E TypeError: Exception encountered when calling layer "brute_force" (type BruteForce).
E
E in user code:
E
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/outputs/topk.py", line 203, in call *
E targets = tf.cast(tf.squeeze(targets), tf.int32)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler **
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4639, in squeeze_v2
E return squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 141, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py", line 1082, in op_dispatch_handler
E return dispatch_target(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/array_ops.py", line 4587, in squeeze
E return gen_array_ops.squeeze(input, axis, name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 10375, in squeeze
E _, _, _op, _outputs = _op_def_library._apply_op_helper(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 779, in _apply_op_helper
E _ExtractInputsAndAttrs(op_type_name, op_def, allowed_list_attr_map,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 560, in _ExtractInputsAndAttrs
E raise err
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 552, in _ExtractInputsAndAttrs
E values = ops.convert_to_tensor(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/trace.py", line 183, in wrapped
E return func(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1640, in convert_to_tensor
E ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 343, in _constant_tensor_conversion_function
E return constant(v, dtype=dtype, name=name)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 267, in constant
E return _constant_impl(value, dtype, shape, name, verify_shape=False,
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
E tensor_util.make_tensor_proto(
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 453, in make_tensor_proto
E _AssertCompatible(values, dtype)
E File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py", line 330, in _AssertCompatible
E raise TypeError("Expected any non-tensor type, but got a tensor instead.")
E
E TypeError: Expected any non-tensor type, but got a tensor instead.
E
E
E Call arguments received by layer "brute_force" (type BruteForce):
E • inputs=tf.Tensor(shape=(None, 32), dtype=float32)
E • targets={'item_id_seq': 'tf.Tensor(shape=(None,), dtype=int64)'}
E • testing=True
E
E
E Call arguments received by layer "top_k_output" (type TopKOutput):
E • inputs=tf.Tensor(shape=(None, 32), dtype=float32)
E • training=False
E • testing=True
E • kwargs={'features': {'item_id_seq': 'tf.RaggedTensor(values=Tensor("RaggedFromVariant_1/RaggedTensorFromVariant:1", shape=(None,), dtype=int64), row_splits=Tensor("RaggedFromVariant_1/RaggedTensorFromVariant:0", shape=(None,), dtype=int32))', 'categories': 'tf.RaggedTensor(values=Tensor("RaggedFromVariant/RaggedTensorFromVariant:1", shape=(None,), dtype=int64), row_splits=Tensor("RaggedFromVariant/RaggedTensorFromVariant:0", shape=(None,), dtype=int32))', 'test_user_id': 'tf.Tensor(shape=(None, None), dtype=int64)', 'user_country': 'tf.Tensor(shape=(None, None), dtype=int64)', 'user_age': 'tf.Tensor(shape=(None, None), dtype=float32)'}, 'targets': {'item_id_seq': 'tf.Tensor(shape=(None,), dtype=int64)'}}
E
E
E Call arguments received by layer "top_k_encoder_1" (type TopKEncoder):
E • inputs={'item_id_seq': 'tf.RaggedTensor(values=Tensor("RaggedFromVariant_1/RaggedTensorFromVariant:1", shape=(None,), dtype=int64), row_splits=Tensor("RaggedFromVariant_1/RaggedTensorFromVariant:0", shape=(None,), dtype=int32))', 'categories': 'tf.RaggedTensor(values=Tensor("RaggedFromVariant/RaggedTensorFromVariant:1", shape=(None,), dtype=int64), row_splits=Tensor("RaggedFromVariant/RaggedTensorFromVariant:0", shape=(None,), dtype=int32))', 'test_user_id': 'tf.Tensor(shape=(None, None), dtype=int64)', 'user_country': 'tf.Tensor(shape=(None, None), dtype=int64)', 'user_age': 'tf.Tensor(shape=(None, None), dtype=float32)'}
E • training=False
E • testing=True
E • targets={'item_id_seq': 'tf.Tensor(shape=(None,), dtype=int64)'}
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py:330: TypeError
----------------------------- Captured stdout call -----------------------------
1/1 [==============================] - ETA: 0s - loss: 6.9075 - recall_at_10: 0.0000e+00 - mrr_at_10: 0.0000e+00 - ndcg_at_10: 0.0000e+00 - map_at_10: 0.0000e+00 - precision_at_10: 0.0000e+00 - regularization_loss: 0.0000e+00
1/1 [==============================] - 1s 949ms/step - loss: 6.9075 - recall_at_10: 0.0000e+00 - mrr_at_10: 0.0000e+00 - ndcg_at_10: 0.0000e+00 - map_at_10: 0.0000e+00 - precision_at_10: 0.0000e+00 - regularization_loss: 0.0000e+00
----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:Gradients do not exist for variables ['retrieval_model_v2/output_layer_bias:0'] when minimizing the loss. If you're using model.compile(), did you forget to provide a lossargument?
------------------------------ Captured log call -------------------------------
WARNING tensorflow:utils.py:76 Gradients do not exist for variables ['retrieval_model_v2/output_layer_bias:0'] when minimizing the loss. If you're using model.compile(), did you forget to provide a lossargument?
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 15 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileng7n_xzk.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 4 failed, 787 passed, 12 skipped, 1312 warnings in 1631.33s (0:27:11) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins16925676776844816438.sh
Click to view CI Results
GitHub pull request #790 of commit 713972ae13f7a6588506c0cf10ffa3cf458ca4bd, no merge conflicts.
Running as SYSTEM
Setting status of 713972ae13f7a6588506c0cf10ffa3cf458ca4bd to PENDING with url https://10.20.13.93:8080/job/merlin_models/1522/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 713972ae13f7a6588506c0cf10ffa3cf458ca4bd^{commit} # timeout=10
Checking out Revision 713972ae13f7a6588506c0cf10ffa3cf458ca4bd (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 713972ae13f7a6588506c0cf10ffa3cf458ca4bd # timeout=10
Commit message: "fix failing top-k tests"
> git rev-list --no-walk 635634ad2e0b0348bcac2adfd493c832f58e9535 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins14096963521624941696.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 805 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
...... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filenffc5ii8.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 793 passed, 12 skipped, 1316 warnings in 1640.89s (0:27:20) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins12211145519547995016.sh
Click to view CI Results
GitHub pull request #790 of commit 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed, no merge conflicts.
Running as SYSTEM
Setting status of 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed to PENDING with url https://10.20.13.93:8080/job/merlin_models/1524/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed^{commit} # timeout=10
Checking out Revision 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6b1025abcca0875c05a61fbc9f7f87d655f1c7ed # timeout=10
Commit message: "add cut-off k parameter to retrievam models"
> git rev-list --no-walk d24372378b20916b00decfe5dc7ae02e8869b683 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins6297622052316829492.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 805 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
...... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:943: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileqcev274f.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 793 passed, 12 skipped, 1316 warnings in 1648.03s (0:27:28) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins6059351119452169172.sh
Click to view CI Results
GitHub pull request #790 of commit 32cebd01fd7ea9db47f2ed38ce860c74dd7f2906, no merge conflicts.
Running as SYSTEM
Setting status of 32cebd01fd7ea9db47f2ed38ce860c74dd7f2906 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1526/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 32cebd01fd7ea9db47f2ed38ce860c74dd7f2906^{commit} # timeout=10
Checking out Revision 32cebd01fd7ea9db47f2ed38ce860c74dd7f2906 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 32cebd01fd7ea9db47f2ed38ce860c74dd7f2906 # timeout=10
Commit message: "fix failing tests"
> git rev-list --no-walk 0707b8f2cf53ebbd1469b55a2a3bdf5c65cc2d34 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins14841762696049563797.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 805 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 29%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
...... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py ............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 67%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 80%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 19 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 17 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:958: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileal4ad8xk.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:465: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:570: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 793 passed, 12 skipped, 1316 warnings in 1634.50s (0:27:14) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins18392024672974183181.sh
Check out this pull request on 
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
![review-notebook-app[bot] avatar](https://avatars.githubusercontent.com/in/15474?v=4 "Published by a pub.dev verified publisher"){kind=small}
Click to view CI Results
GitHub pull request #790 of commit c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e, no merge conflicts.
Running as SYSTEM
Setting status of c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e to PENDING with url https://10.20.13.93:8080/job/merlin_models/1553/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e^{commit} # timeout=10
Checking out Revision c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e # timeout=10
Commit message: "remove stale code"
> git rev-list --no-walk 6ebbb78330f12a3e5bc47283faccee5d6fda506f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins18315517616564509028.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 807 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
....... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 42%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filezvr9dgek.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:467: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:572: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 795 passed, 12 skipped, 1322 warnings in 1659.89s (0:27:39) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins9922864880153795470.sh
Click to view CI Results
GitHub pull request #790 of commit 2c2062dcb9c8ec545bf784415cf3acafca86d14d, no merge conflicts.
Running as SYSTEM
Setting status of 2c2062dcb9c8ec545bf784415cf3acafca86d14d to PENDING with url https://10.20.13.93:8080/job/merlin_models/1554/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 2c2062dcb9c8ec545bf784415cf3acafca86d14d^{commit} # timeout=10
Checking out Revision 2c2062dcb9c8ec545bf784415cf3acafca86d14d (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 2c2062dcb9c8ec545bf784415cf3acafca86d14d # timeout=10
Commit message: "remove stale code"
> git rev-list --no-walk c7bbe6dbdcb4cc098725cdf8ba89b9f3f6dba38e # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8785921034040049003.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 807 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 20%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 20%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 22%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 22%]
tests/unit/tf/core/test_aggregation.py ......... [ 23%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 26%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 27%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 28%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 28%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 28%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 28%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ..... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
....... [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 42%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 48%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
................................ [ 60%]
tests/unit/tf/outputs/test_base.py ..... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 61%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 63%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 71%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 75%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 83%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 86%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 107 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 73 warnings
tests/unit/tf/outputs/test_base.py: 5 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 2 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file5jxivzid.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:467: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:572: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 795 passed, 12 skipped, 1322 warnings in 1659.07s (0:27:39) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins709936259297131140.sh
Click to view CI Results
GitHub pull request #790 of commit 457ffc5b8c0b89a2d5fbce98b28b88967d4c984f, no merge conflicts.
Running as SYSTEM
Setting status of 457ffc5b8c0b89a2d5fbce98b28b88967d4c984f to PENDING with url https://10.20.13.93:8080/job/merlin_models/1582/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 457ffc5b8c0b89a2d5fbce98b28b88967d4c984f^{commit} # timeout=10
Checking out Revision 457ffc5b8c0b89a2d5fbce98b28b88967d4c984f (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 457ffc5b8c0b89a2d5fbce98b28b88967d4c984f # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 3359497e92ed1c7173e4ae3f8e8e442b95f1a55f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8239773651128567337.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 819 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 26%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 28%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 41%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 52%]
tests/unit/tf/models/test_retrieval.py ................................. [ 56%]
........................................ [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 64%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 65%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 67%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 69%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py ...... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 115 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 81 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file5tea00fm.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:467: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:572: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 807 passed, 12 skipped, 1346 warnings in 1660.23s (0:27:40) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins15257443878320252151.sh
Click to view CI Results
GitHub pull request #790 of commit 74249c9c773f0592fc0a09ab1f2f8daf89d5b4ec, no merge conflicts.
Running as SYSTEM
Setting status of 74249c9c773f0592fc0a09ab1f2f8daf89d5b4ec to PENDING with url https://10.20.13.93:8080/job/merlin_models/1593/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 74249c9c773f0592fc0a09ab1f2f8daf89d5b4ec^{commit} # timeout=10
Checking out Revision 74249c9c773f0592fc0a09ab1f2f8daf89d5b4ec (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 74249c9c773f0592fc0a09ab1f2f8daf89d5b4ec # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk aacef203f028b8f058a12042a135e5caf6c05226 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins16594831668046636151.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version =2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10; python_version =2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0; python_version =1.4.0; python_version jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-4.0.0
collected 823 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
........................................ [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 115 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 81 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileg90hu2fc.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2[False]
tests/unit/tf/models/test_retrieval.py::test_two_tower_model_v2_l2_reg
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[True]
tests/unit/tf/models/test_retrieval.py::test_two_tower_retrieval_model_v2_with_topk_metrics_aggregator[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:467: UserWarning: By default, the TwoTower model is trained using constrastive learning.When negative_samplers is not specified, in-batch negative sampling is used.
warnings.warn(
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_retrieval_v2[False]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/retrieval.py:572: UserWarning: By default, the YoutubeDNN model is trained using popularity-bases sampled softmax.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 811 passed, 12 skipped, 1346 warnings in 1644.06s (0:27:24) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins17668472463315271573.sh
Click to view CI Results
GitHub pull request #790 of commit 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1, no merge conflicts.
Running as SYSTEM
Setting status of 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1615/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1^{commit} # timeout=10
Checking out Revision 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1 # timeout=10
Commit message: "update integration test with the merged fix"
> git rev-list --no-walk c21e69de57116371fe6161dcd1e4eaaa6892e32d # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4135838985176223404.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 823 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
........................................ [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 115 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 81 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileq9tsogjy.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 811 passed, 12 skipped, 1336 warnings in 1670.88s (0:27:50) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins5879220674264487835.sh
Click to view CI Results
GitHub pull request #790 of commit 9d5a3e98785ed8e5a31f6c89075ebf23ae14092a, no merge conflicts.
Running as SYSTEM
Setting status of 9d5a3e98785ed8e5a31f6c89075ebf23ae14092a to PENDING with url https://10.20.13.93:8080/job/merlin_models/1616/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 9d5a3e98785ed8e5a31f6c89075ebf23ae14092a^{commit} # timeout=10
Checking out Revision 9d5a3e98785ed8e5a31f6c89075ebf23ae14092a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9d5a3e98785ed8e5a31f6c89075ebf23ae14092a # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 7e44acf291f8433146d64dc0d8c97d7ce7bfd2b1 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4245628701990199401.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 823 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
........................................ [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 115 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 81 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filehxuczyur.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/distributed/node.py:177: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 46093 instead
warnings.warn(
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 811 passed, 12 skipped, 1337 warnings in 1704.48s (0:28:24) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins4885158019184571631.sh
Click to view CI Results
GitHub pull request #790 of commit 11b0bed40362185712b121e37922d695930d8998, no merge conflicts.
Running as SYSTEM
Setting status of 11b0bed40362185712b121e37922d695930d8998 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1618/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 11b0bed40362185712b121e37922d695930d8998^{commit} # timeout=10
Checking out Revision 11b0bed40362185712b121e37922d695930d8998 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 11b0bed40362185712b121e37922d695930d8998 # timeout=10
Commit message: "add SequencePredictRandom to youtube-dnn test"
> git rev-list --no-walk 40f868e85034406ae9862cd334bce0f87107138f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins15731724187766072167.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 825 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 20%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 28%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py .................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filem4ju_0qc.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 813 passed, 12 skipped, 1344 warnings in 1690.79s (0:28:10) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins16295506314421980391.sh
Click to view CI Results
GitHub pull request #790 of commit 5e70f405cd339a0d39866cb51dacf6993b034a5e, no merge conflicts.
Running as SYSTEM
Setting status of 5e70f405cd339a0d39866cb51dacf6993b034a5e to PENDING with url https://10.20.13.93:8080/job/merlin_models/1650/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 5e70f405cd339a0d39866cb51dacf6993b034a5e^{commit} # timeout=10
Checking out Revision 5e70f405cd339a0d39866cb51dacf6993b034a5e (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5e70f405cd339a0d39866cb51dacf6993b034a5e # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 0dfdd5235cf9008f448987564d00229724ac9675 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins247924920640258628.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: tox in /usr/local/lib/python3.8/dist-packages (3.26.0)
Requirement already satisfied: six>=1.14.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from tox) (1.15.0)
Requirement already satisfied: pluggy>=0.12.0 in /usr/local/lib/python3.8/dist-packages (from tox) (1.0.0)
Requirement already satisfied: py>=1.4.17 in /usr/local/lib/python3.8/dist-packages (from tox) (1.11.0)
Requirement already satisfied: virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 in /usr/local/lib/python3.8/dist-packages (from tox) (20.16.5)
Requirement already satisfied: packaging>=14 in /usr/local/lib/python3.8/dist-packages (from tox) (21.3)
Requirement already satisfied: tomli>=2.0.1 in /usr/local/lib/python3.8/dist-packages (from tox) (2.0.1)
Requirement already satisfied: filelock>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from tox) (3.8.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=14->tox) (3.0.9)
Requirement already satisfied: platformdirs=2.4 in /usr/local/lib/python3.8/dist-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (2.5.2)
Requirement already satisfied: distlib=0.3.5 in /usr/local/lib/python3.8/dist-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (0.3.6)
ERROR: tox config file (either pyproject.toml, tox.ini, setup.cfg) not found
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins13032079822582410581.sh
Click to view CI Results
GitHub pull request #790 of commit 5e70f405cd339a0d39866cb51dacf6993b034a5e, no merge conflicts.
Running as SYSTEM
Setting status of 5e70f405cd339a0d39866cb51dacf6993b034a5e to PENDING with url https://10.20.13.93:8080/job/merlin_models/1657/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 5e70f405cd339a0d39866cb51dacf6993b034a5e^{commit} # timeout=10
Checking out Revision 5e70f405cd339a0d39866cb51dacf6993b034a5e (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5e70f405cd339a0d39866cb51dacf6993b034a5e # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 3255bfbfd65a42c2abdaf1c6fe719e98e829e69b # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3726017894182847854.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 826 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 23%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py F [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py F [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 38%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ........................ [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 84%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
_______________________ test_example_02_nvt_integration ________________________
self = <testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>
cell = [50], kwargs = {}, cell_indexes = [50], executed_cells = [], idx = 50
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:
args = (<testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>, {'id': '53b1b18e', 'cell_type': 'code', 'metadata'...\x1b[0m: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory']}]}, 50)
kwargs = {}
def wrapped(*args, **kwargs):
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85:
coro = <coroutine object NotebookClient.async_execute_cell at 0x7f030378fc40>
def just_run(coro: Awaitable) -> Any:
"""Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop is None:
had_running_loop = False
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
else:
had_running_loop = True
if had_running_loop:
# if there is a running loop, we patch using nest_asyncio
# to have reentrant event loops
check_ipython()
import nest_asyncio
nest_asyncio.apply()
check_patch_tornado()
return loop.run_until_complete(coro)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60:
self = <_UnixSelectorEventLoop running=False closed=False debug=False>
future = <Task finished name='Task-74' coro=<NotebookClient.async_execute_cell() done, defined at /usr/local/lib/python3.8/dist...RuntimeError: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory\n')>
def run_until_complete(self, future):
"""Run until the Future is done.
If the argument is a coroutine, it is wrapped in a Task.
WARNING: It would be disastrous to call run_until_complete()
with the same coroutine twice -- it would wrap it in two
different Tasks and that can't be good.
Return the Future's result, or raise its exception.
"""
self._check_closed()
self._check_running()
new_task = not futures.isfuture(future)
future = tasks.ensure_future(future, loop=self)
if new_task:
# An exception is raised if the future didn't complete, so there
# is no need to log the "destroy pending task" message
future._log_destroy_pending = False
future.add_done_callback(_run_until_complete_cb)
try:
self.run_forever()
except:
if new_task and future.done() and not future.cancelled():
# The coroutine raised a BaseException. Consume the exception
# to not log a warning, the caller doesn't have access to the
# local task.
future.exception()
raise
finally:
future.remove_done_callback(_run_until_complete_cb)
if not future.done():
raise RuntimeError('Event loop stopped before Future completed.')
return future.result()
/usr/lib/python3.8/asyncio/base_events.py:616:
self = <testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>
cell = {'id': '53b1b18e', 'cell_type': 'code', 'metadata': {'execution': {'iopub.status.busy': '2022-10-31T15:40:36.889285Z',...Error\x1b[0m: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory']}]}
cell_index = 50, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
exec_reply = await self.task_poll_for_reply
except asyncio.CancelledError:
# can only be cancelled by task_poll_kernel_alive when the kernel is dead
task_poll_output_msg.cancel()
raise DeadKernelError("Kernel died")
except Exception as e:
# Best effort to cancel request if it hasn't been resolved
try:
# Check if the task_poll_output is doing the raising for us
if not isinstance(e, CellControlSignal):
task_poll_output_msg.cancel()
finally:
raise
if execution_count:
cell['execution_count'] = execution_count
await run_hook(
self.on_cell_executed, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
await self._check_raise_for_error(cell, cell_index, exec_reply)
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025:
self = <testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>
cell = {'id': '53b1b18e', 'cell_type': 'code', 'metadata': {'execution': {'iopub.status.busy': '2022-10-31T15:40:36.889285Z',...Error\x1b[0m: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory']}]}
cell_index = 50
exec_reply = {'buffers': [], 'content': {'ename': 'RuntimeError', 'engine_info': {'engine_id': -1, 'engine_uuid': '39bfdfe5-5f30-44...e, 'engine': '39bfdfe5-5f30-4451-84f5-4eec0bee4ec6', 'started': '2022-10-31T15:40:36.889855Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E
E import os
E os.environ["INPUT_DATA_DIR"] = "/tmp/pytest-of-jenkins/pytest-15/test_example_02_nvt_integratio0"
E from unittest.mock import patch
E from merlin.datasets.synthetic import generate_data
E mock_train, mock_valid = generate_data(
E input="movielens-1m",
E num_rows=1000,
E set_sizes=(0.8, 0.2)
E )
E p1 = patch(
E "merlin.datasets.entertainment.get_movielens",
E return_value=[mock_train, mock_valid]
E )
E p1.start()
E p2 = patch(
E "merlin.core.utils.download_file",
E return_value=[]
E )
E p2.start()
E import numpy as np
E import pandas
E from pathlib import Path
E from merlin.datasets.synthetic import generate_data
E mock_data = generate_data(
E input="movielens-1m-raw-ratings",
E num_rows=1000
E )
E mock_data = mock_data.to_ddf().compute()
E if not isinstance(mock_data, pandas.core.frame.DataFrame):
E mock_data = mock_data.to_pandas()
E input_path = os.environ.get(
E "INPUT_DATA_DIR",
E "/tmp/pytest-of-jenkins/pytest-15/test_example_02_nvt_integratio0"
E )
E path = Path(input_path) / "ml-1m"
E path.mkdir(parents=True, exist_ok=True)
E np.savetxt(
E str(path / "ratings.dat"),
E mock_data.values,
E delimiter='::',
E fmt='%s',
E encoding='utf-8'
E )
E
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mRuntimeError[0m Traceback (most recent call last)
E Cell [0;32mIn [1], line 5[0m
E [1;32m 3[0m [38;5;28;01mfrom[39;00m [38;5;21;01munittest[39;00m[38;5;21;01m.[39;00m[38;5;21;01mmock[39;00m [38;5;28;01mimport[39;00m patch
E [1;32m 4[0m [38;5;28;01mfrom[39;00m [38;5;21;01mmerlin[39;00m[38;5;21;01m.[39;00m[38;5;21;01mdatasets[39;00m[38;5;21;01m.[39;00m[38;5;21;01msynthetic[39;00m [38;5;28;01mimport[39;00m generate_data
E [0;32m----> 5[0m mock_train, mock_valid [38;5;241m=[39m [43mgenerate_data[49m[43m([49m
E [1;32m 6[0m [43m [49m[38;5;28;43minput[39;49m[38;5;241;43m=[39;49m[38;5;124;43m"[39;49m[38;5;124;43mmovielens-1m[39;49m[38;5;124;43m"[39;49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mnum_rows[49m[38;5;241;43m=[39;49m[38;5;241;43m1000[39;49m[43m,[49m
E [1;32m 8[0m [43m [49m[43mset_sizes[49m[38;5;241;43m=[39;49m[43m([49m[38;5;241;43m0.8[39;49m[43m,[49m[43m [49m[38;5;241;43m0.2[39;49m[43m)[49m
E [1;32m 9[0m [43m)[49m
E [1;32m 10[0m p1 [38;5;241m=[39m patch(
E [1;32m 11[0m [38;5;124m"[39m[38;5;124mmerlin.datasets.entertainment.get_movielens[39m[38;5;124m"[39m,
E [1;32m 12[0m return_value[38;5;241m=[39m[mock_train, mock_valid]
E [1;32m 13[0m )
E [1;32m 14[0m p1[38;5;241m.[39mstart()
E
E File [0;32m~/workspace/merlin_models/models/merlin/datasets/synthetic.py:131[0m, in [0;36mgenerate_data[0;34m(input, num_rows, set_sizes, min_session_length, max_session_length, device)[0m
E [1;32m 128[0m start_i [38;5;241m=[39m end_i
E [1;32m 129[0m output_datasets[38;5;241m.[39mappend(set_df)
E [0;32m--> 131[0m [38;5;28;01mreturn[39;00m [38;5;28mtuple[39m([merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(d, schema[38;5;241m=[39mschema) [38;5;28;01mfor[39;00m d [38;5;129;01min[39;00m output_datasets])
E [1;32m 133[0m [38;5;28;01mreturn[39;00m merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(df, schema[38;5;241m=[39mschema)
E
E File [0;32m~/workspace/merlin_models/models/merlin/datasets/synthetic.py:131[0m, in [0;36m[0;34m(.0)[0m
E [1;32m 128[0m start_i [38;5;241m=[39m end_i
E [1;32m 129[0m output_datasets[38;5;241m.[39mappend(set_df)
E [0;32m--> 131[0m [38;5;28;01mreturn[39;00m [38;5;28mtuple[39m([[43mmerlin[49m[38;5;241;43m.[39;49m[43mio[49m[38;5;241;43m.[39;49m[43mDataset[49m[43m([49m[43md[49m[43m,[49m[43m [49m[43mschema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m)[49m [38;5;28;01mfor[39;00m d [38;5;129;01min[39;00m output_datasets])
E [1;32m 133[0m [38;5;28;01mreturn[39;00m merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(df, schema[38;5;241m=[39mschema)
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/merlin/io/dataset.py:262[0m, in [0;36mDataset.__init__[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)[0m
E [1;32m 256[0m npartitions [38;5;241m=[39m npartitions [38;5;129;01mor[39;00m [38;5;241m1[39m
E [1;32m 257[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(path_or_source, dask[38;5;241m.[39mdataframe[38;5;241m.[39mDataFrame) [38;5;129;01mor[39;00m is_dataframe_object(
E [1;32m 258[0m path_or_source
E [1;32m 259[0m ):
E [1;32m 260[0m [38;5;66;03m# User is passing in a <dask.dataframe|cudf|pd>.DataFrame[39;00m
E [1;32m 261[0m [38;5;66;03m# Use DataFrameDatasetEngine[39;00m
E [0;32m--> 262[0m _path_or_source [38;5;241m=[39m [43mconvert_data[49m[43m([49m
E [1;32m 263[0m [43m [49m[43mpath_or_source[49m[43m,[49m[43m [49m[43mcpu[49m[38;5;241;43m=[39;49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mcpu[49m[43m,[49m[43m [49m[43mto_collection[49m[38;5;241;43m=[39;49m[38;5;28;43;01mTrue[39;49;00m[43m,[49m[43m [49m[43mnpartitions[49m[38;5;241;43m=[39;49m[43mnpartitions[49m
E [1;32m 264[0m [43m [49m[43m)[49m
E [1;32m 265[0m [38;5;66;03m# Check if this is a collection that has now moved between host <-> device[39;00m
E [1;32m 266[0m moved_collection [38;5;241m=[39m [38;5;28misinstance[39m(path_or_source, dask[38;5;241m.[39mdataframe[38;5;241m.[39mDataFrame) [38;5;129;01mand[39;00m (
E [1;32m 267[0m [38;5;129;01mnot[39;00m [38;5;28misinstance[39m(_path_or_source[38;5;241m.[39m_meta, [38;5;28mtype[39m(path_or_source[38;5;241m.[39m_meta))
E [1;32m 268[0m )
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/merlin/core/dispatch.py:567[0m, in [0;36mconvert_data[0;34m(x, cpu, to_collection, npartitions)[0m
E [1;32m 565[0m _x [38;5;241m=[39m cudf[38;5;241m.[39mDataFrame[38;5;241m.[39mfrom_arrow(x)
E [1;32m 566[0m [38;5;28;01melif[39;00m [38;5;28misinstance[39m(x, pd[38;5;241m.[39mDataFrame):
E [0;32m--> 567[0m _x [38;5;241m=[39m [43mcudf[49m[38;5;241;43m.[39;49m[43mDataFrame[49m[38;5;241;43m.[39;49m[43mfrom_pandas[49m[43m([49m[43mx[49m[43m)[49m
E [1;32m 568[0m [38;5;66;03m# Output a collection if to_collection=True[39;00m
E [1;32m 569[0m [38;5;28;01mreturn[39;00m (
E [1;32m 570[0m dask_cudf[38;5;241m.[39mfrom_cudf(_x, sort[38;5;241m=[39m[38;5;28;01mFalse[39;00m, npartitions[38;5;241m=[39mnpartitions)
E [1;32m 571[0m [38;5;28;01mif[39;00m to_collection
E [1;32m 572[0m [38;5;28;01melse[39;00m _x
E [1;32m 573[0m )
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/nvtx/nvtx.py:101[0m, in [0;36mannotate.call..inner[0;34m(args, **kwargs)[0m
E [1;32m 98[0m [38;5;129m@wraps[39m(func)
E [1;32m 99[0m [38;5;28;01mdef[39;00m [38;5;21minner[39m([38;5;241m[39margs, [38;5;241m[39m[38;5;241m[39mkwargs):
E [1;32m 100[0m libnvtx_push_range([38;5;28mself[39m[38;5;241m.[39mattributes, [38;5;28mself[39m[38;5;241m.[39mdomain[38;5;241m.[39mhandle)
E [0;32m--> 101[0m result [38;5;241m=[39m [43mfunc[49m[43m([49m[38;5;241;43m[39;49m[43margs[49m[43m,[49m[43m [49m[38;5;241;43m[39;49m[38;5;241;43m*[39;49m[43mkwargs[49m[43m)[49m
E [1;32m 102[0m libnvtx_pop_range([38;5;28mself[39m[38;5;241m.[39mdomain[38;5;241m.[39mhandle)
E [1;32m 103[0m [38;5;28;01mreturn[39;00m result
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:4547[0m, in [0;36mDataFrame.from_pandas[0;34m(cls, dataframe, nan_as_null)[0m
E [1;32m 4543[0m [38;5;28;01mfor[39;00m col_name, col_value [38;5;129;01min[39;00m dataframe[38;5;241m.[39mitems():
E [1;32m 4544[0m [38;5;66;03m# necessary because multi-index can return multiple[39;00m
E [1;32m 4545[0m [38;5;66;03m# columns for a single key[39;00m
E [1;32m 4546[0m [38;5;28;01mif[39;00m [38;5;28mlen[39m(col_value[38;5;241m.[39mshape) [38;5;241m==[39m [38;5;241m1[39m:
E [0;32m-> 4547[0m df[col_name] [38;5;241m=[39m [43mcolumn[49m[38;5;241;43m.[39;49m[43mas_column[49m[43m([49m
E [1;32m 4548[0m [43m [49m[43mcol_value[49m[38;5;241;43m.[39;49m[43marray[49m[43m,[49m[43m [49m[43mnan_as_null[49m[38;5;241;43m=[39;49m[43mnan_as_null[49m
E [1;32m 4549[0m [43m [49m[43m)[49m
E [1;32m 4550[0m [38;5;28;01melse[39;00m:
E [1;32m 4551[0m vals [38;5;241m=[39m col_value[38;5;241m.[39mvalues[38;5;241m.[39mT
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:1966[0m, in [0;36mas_column[0;34m(arbitrary, nan_as_null, dtype, length)[0m
E [1;32m 1964[0m data [38;5;241m=[39m as_column(pa[38;5;241m.[39mArray[38;5;241m.[39mfrom_pandas(arbitrary), dtype[38;5;241m=[39marb_dtype)
E [1;32m 1965[0m [38;5;28;01melse[39;00m:
E [0;32m-> 1966[0m data [38;5;241m=[39m [43mas_column[49m[43m([49m
E [1;32m 1967[0m [43m [49m[43mpa[49m[38;5;241;43m.[39;49m[43marray[49m[43m([49m
E [1;32m 1968[0m [43m [49m[43marbitrary[49m[43m,[49m
E [1;32m 1969[0m [43m [49m[43mfrom_pandas[49m[38;5;241;43m=[39;49m[38;5;28;43;01mTrue[39;49;00m[43m [49m[38;5;28;43;01mif[39;49;00m[43m [49m[43mnan_as_null[49m[43m [49m[38;5;129;43;01mis[39;49;00m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m [49m[38;5;28;43;01melse[39;49;00m[43m [49m[43mnan_as_null[49m[43m,[49m
E [1;32m 1970[0m [43m [49m[43m)[49m[43m,[49m
E [1;32m 1971[0m [43m [49m[43mnan_as_null[49m[38;5;241;43m=[39;49m[43mnan_as_null[49m[43m,[49m
E [1;32m 1972[0m [43m [49m[43m)[49m
E [1;32m 1973[0m [38;5;28;01mif[39;00m dtype [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m:
E [1;32m 1974[0m data [38;5;241m=[39m data[38;5;241m.[39mastype(dtype)
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:1760[0m, in [0;36mas_column[0;34m(arbitrary, nan_as_null, dtype, length)[0m
E [1;32m 1754[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(arbitrary, pa[38;5;241m.[39mlib[38;5;241m.[39mHalfFloatArray):
E [1;32m 1755[0m [38;5;28;01mraise[39;00m [38;5;167;01mNotImplementedError[39;00m(
E [1;32m 1756[0m [38;5;124m"[39m[38;5;124mType casting from float16 to float32 is not [39m[38;5;124m"[39m
E [1;32m 1757[0m [38;5;124m"[39m[38;5;124myet supported in pyarrow, see: [39m[38;5;124m"[39m
E [1;32m 1758[0m [38;5;124m"[39m[38;5;124mhttps://issues.apache.org/jira/browse/ARROW-3802[39m[38;5;124m"[39m
E [1;32m 1759[0m )
E [0;32m-> 1760[0m col [38;5;241m=[39m [43mColumnBase[49m[38;5;241;43m.[39;49m[43mfrom_arrow[49m[43m([49m[43marbitrary[49m[43m)[49m
E [1;32m 1762[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(arbitrary, pa[38;5;241m.[39mNullArray):
E [1;32m 1763[0m new_dtype [38;5;241m=[39m cudf[38;5;241m.[39mdtype(arbitrary[38;5;241m.[39mtype[38;5;241m.[39mto_pandas_dtype())
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:297[0m, in [0;36mColumnBase.from_arrow[0;34m(cls, array)[0m
E [1;32m 292[0m [38;5;28;01melif[39;00m [38;5;28misinstance[39m(
E [1;32m 293[0m array[38;5;241m.[39mtype, pd[38;5;241m.[39mcore[38;5;241m.[39marrays[38;5;241m.[39m_arrow_utils[38;5;241m.[39mArrowIntervalType
E [1;32m 294[0m ):
E [1;32m 295[0m [38;5;28;01mreturn[39;00m cudf[38;5;241m.[39mcore[38;5;241m.[39mcolumn[38;5;241m.[39mIntervalColumn[38;5;241m.[39mfrom_arrow(array)
E [0;32m--> 297[0m result [38;5;241m=[39m [43mlibcudf[49m[38;5;241;43m.[39;49m[43minterop[49m[38;5;241;43m.[39;49m[43mfrom_arrow[49m[43m([49m[43mdata[49m[43m)[49m[[38;5;241m0[39m]
E [1;32m 299[0m [38;5;28;01mreturn[39;00m result[38;5;241m.[39m_with_type_metadata(cudf_dtype_from_pa_type(array[38;5;241m.[39mtype))
E
E File [0;32mcudf/_lib/interop.pyx:150[0m, in [0;36mcudf._lib.interop.from_arrow[0;34m()[0m
E
E [0;31mRuntimeError[0m: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory
E RuntimeError: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>
tmpdir = local('/tmp/pytest-of-jenkins/pytest-15/test_example_02_nvt_integratio0')
@testbook(REPO_ROOT / "examples/02-Merlin-Models-and-NVTabular-integration.ipynb", execute=False)
def test_example_02_nvt_integration(tb, tmpdir):
tb.inject(
f"""
import os
os.environ["INPUT_DATA_DIR"] = "{tmpdir}"
from unittest.mock import patch
from merlin.datasets.synthetic import generate_data
mock_train, mock_valid = generate_data(
input="movielens-1m",
num_rows=1000,
set_sizes=(0.8, 0.2)
)
p1 = patch(
"merlin.datasets.entertainment.get_movielens",
return_value=[mock_train, mock_valid]
)
p1.start()
p2 = patch(
"merlin.core.utils.download_file",
return_value=[]
)
p2.start()
import numpy as np
import pandas
from pathlib import Path
from merlin.datasets.synthetic import generate_data
mock_data = generate_data(
input="movielens-1m-raw-ratings",
num_rows=1000
)
mock_data = mock_data.to_ddf().compute()
if not isinstance(mock_data, pandas.core.frame.DataFrame):
mock_data = mock_data.to_pandas()
input_path = os.environ.get(
"INPUT_DATA_DIR",
"{tmpdir}"
)
path = Path(input_path) / "ml-1m"
path.mkdir(parents=True, exist_ok=True)
np.savetxt(
str(path / "ratings.dat"),
mock_data.values,
delimiter='::',
fmt='%s',
encoding='utf-8'
)
"""
)
tests/unit/tf/examples/test_02_dataschema.py:8:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:237: in inject
cell = TestbookNode(self.execute_cell(inject_idx)) if run else TestbookNode(code_cell)
self = <testbook.client.TestbookNotebookClient object at 0x7f033f4e2af0>
cell = [50], kwargs = {}, cell_indexes = [50], executed_cells = [], idx = 50
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
"""
Executes a cell or list of cells
"""
if isinstance(cell, slice):
start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
if cell.step is not None:
raise TestbookError('testbook does not support step argument')
cell = range(start, stop + 1)
elif isinstance(cell, str) or isinstance(cell, int):
cell = [cell]
cell_indexes = cell
if all(isinstance(x, str) for x in cell):
cell_indexes = [self._cell_index(tag) for tag in cell]
executed_cells = []
for idx in cell_indexes:
try:
cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
except CellExecutionError as ce:
raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))
E testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:
E ------------------
E
E import os
E os.environ["INPUT_DATA_DIR"] = "/tmp/pytest-of-jenkins/pytest-15/test_example_02_nvt_integratio0"
E from unittest.mock import patch
E from merlin.datasets.synthetic import generate_data
E mock_train, mock_valid = generate_data(
E input="movielens-1m",
E num_rows=1000,
E set_sizes=(0.8, 0.2)
E )
E p1 = patch(
E "merlin.datasets.entertainment.get_movielens",
E return_value=[mock_train, mock_valid]
E )
E p1.start()
E p2 = patch(
E "merlin.core.utils.download_file",
E return_value=[]
E )
E p2.start()
E import numpy as np
E import pandas
E from pathlib import Path
E from merlin.datasets.synthetic import generate_data
E mock_data = generate_data(
E input="movielens-1m-raw-ratings",
E num_rows=1000
E )
E mock_data = mock_data.to_ddf().compute()
E if not isinstance(mock_data, pandas.core.frame.DataFrame):
E mock_data = mock_data.to_pandas()
E input_path = os.environ.get(
E "INPUT_DATA_DIR",
E "/tmp/pytest-of-jenkins/pytest-15/test_example_02_nvt_integratio0"
E )
E path = Path(input_path) / "ml-1m"
E path.mkdir(parents=True, exist_ok=True)
E np.savetxt(
E str(path / "ratings.dat"),
E mock_data.values,
E delimiter='::',
E fmt='%s',
E encoding='utf-8'
E )
E
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mRuntimeError[0m Traceback (most recent call last)
E Cell [0;32mIn [1], line 5[0m
E [1;32m 3[0m [38;5;28;01mfrom[39;00m [38;5;21;01munittest[39;00m[38;5;21;01m.[39;00m[38;5;21;01mmock[39;00m [38;5;28;01mimport[39;00m patch
E [1;32m 4[0m [38;5;28;01mfrom[39;00m [38;5;21;01mmerlin[39;00m[38;5;21;01m.[39;00m[38;5;21;01mdatasets[39;00m[38;5;21;01m.[39;00m[38;5;21;01msynthetic[39;00m [38;5;28;01mimport[39;00m generate_data
E [0;32m----> 5[0m mock_train, mock_valid [38;5;241m=[39m [43mgenerate_data[49m[43m([49m
E [1;32m 6[0m [43m [49m[38;5;28;43minput[39;49m[38;5;241;43m=[39;49m[38;5;124;43m"[39;49m[38;5;124;43mmovielens-1m[39;49m[38;5;124;43m"[39;49m[43m,[49m
E [1;32m 7[0m [43m [49m[43mnum_rows[49m[38;5;241;43m=[39;49m[38;5;241;43m1000[39;49m[43m,[49m
E [1;32m 8[0m [43m [49m[43mset_sizes[49m[38;5;241;43m=[39;49m[43m([49m[38;5;241;43m0.8[39;49m[43m,[49m[43m [49m[38;5;241;43m0.2[39;49m[43m)[49m
E [1;32m 9[0m [43m)[49m
E [1;32m 10[0m p1 [38;5;241m=[39m patch(
E [1;32m 11[0m [38;5;124m"[39m[38;5;124mmerlin.datasets.entertainment.get_movielens[39m[38;5;124m"[39m,
E [1;32m 12[0m return_value[38;5;241m=[39m[mock_train, mock_valid]
E [1;32m 13[0m )
E [1;32m 14[0m p1[38;5;241m.[39mstart()
E
E File [0;32m~/workspace/merlin_models/models/merlin/datasets/synthetic.py:131[0m, in [0;36mgenerate_data[0;34m(input, num_rows, set_sizes, min_session_length, max_session_length, device)[0m
E [1;32m 128[0m start_i [38;5;241m=[39m end_i
E [1;32m 129[0m output_datasets[38;5;241m.[39mappend(set_df)
E [0;32m--> 131[0m [38;5;28;01mreturn[39;00m [38;5;28mtuple[39m([merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(d, schema[38;5;241m=[39mschema) [38;5;28;01mfor[39;00m d [38;5;129;01min[39;00m output_datasets])
E [1;32m 133[0m [38;5;28;01mreturn[39;00m merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(df, schema[38;5;241m=[39mschema)
E
E File [0;32m~/workspace/merlin_models/models/merlin/datasets/synthetic.py:131[0m, in [0;36m[0;34m(.0)[0m
E [1;32m 128[0m start_i [38;5;241m=[39m end_i
E [1;32m 129[0m output_datasets[38;5;241m.[39mappend(set_df)
E [0;32m--> 131[0m [38;5;28;01mreturn[39;00m [38;5;28mtuple[39m([[43mmerlin[49m[38;5;241;43m.[39;49m[43mio[49m[38;5;241;43m.[39;49m[43mDataset[49m[43m([49m[43md[49m[43m,[49m[43m [49m[43mschema[49m[38;5;241;43m=[39;49m[43mschema[49m[43m)[49m [38;5;28;01mfor[39;00m d [38;5;129;01min[39;00m output_datasets])
E [1;32m 133[0m [38;5;28;01mreturn[39;00m merlin[38;5;241m.[39mio[38;5;241m.[39mDataset(df, schema[38;5;241m=[39mschema)
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/merlin/io/dataset.py:262[0m, in [0;36mDataset.__init__[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)[0m
E [1;32m 256[0m npartitions [38;5;241m=[39m npartitions [38;5;129;01mor[39;00m [38;5;241m1[39m
E [1;32m 257[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(path_or_source, dask[38;5;241m.[39mdataframe[38;5;241m.[39mDataFrame) [38;5;129;01mor[39;00m is_dataframe_object(
E [1;32m 258[0m path_or_source
E [1;32m 259[0m ):
E [1;32m 260[0m [38;5;66;03m# User is passing in a <dask.dataframe|cudf|pd>.DataFrame[39;00m
E [1;32m 261[0m [38;5;66;03m# Use DataFrameDatasetEngine[39;00m
E [0;32m--> 262[0m _path_or_source [38;5;241m=[39m [43mconvert_data[49m[43m([49m
E [1;32m 263[0m [43m [49m[43mpath_or_source[49m[43m,[49m[43m [49m[43mcpu[49m[38;5;241;43m=[39;49m[38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43mcpu[49m[43m,[49m[43m [49m[43mto_collection[49m[38;5;241;43m=[39;49m[38;5;28;43;01mTrue[39;49;00m[43m,[49m[43m [49m[43mnpartitions[49m[38;5;241;43m=[39;49m[43mnpartitions[49m
E [1;32m 264[0m [43m [49m[43m)[49m
E [1;32m 265[0m [38;5;66;03m# Check if this is a collection that has now moved between host <-> device[39;00m
E [1;32m 266[0m moved_collection [38;5;241m=[39m [38;5;28misinstance[39m(path_or_source, dask[38;5;241m.[39mdataframe[38;5;241m.[39mDataFrame) [38;5;129;01mand[39;00m (
E [1;32m 267[0m [38;5;129;01mnot[39;00m [38;5;28misinstance[39m(_path_or_source[38;5;241m.[39m_meta, [38;5;28mtype[39m(path_or_source[38;5;241m.[39m_meta))
E [1;32m 268[0m )
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/merlin/core/dispatch.py:567[0m, in [0;36mconvert_data[0;34m(x, cpu, to_collection, npartitions)[0m
E [1;32m 565[0m _x [38;5;241m=[39m cudf[38;5;241m.[39mDataFrame[38;5;241m.[39mfrom_arrow(x)
E [1;32m 566[0m [38;5;28;01melif[39;00m [38;5;28misinstance[39m(x, pd[38;5;241m.[39mDataFrame):
E [0;32m--> 567[0m _x [38;5;241m=[39m [43mcudf[49m[38;5;241;43m.[39;49m[43mDataFrame[49m[38;5;241;43m.[39;49m[43mfrom_pandas[49m[43m([49m[43mx[49m[43m)[49m
E [1;32m 568[0m [38;5;66;03m# Output a collection if to_collection=True[39;00m
E [1;32m 569[0m [38;5;28;01mreturn[39;00m (
E [1;32m 570[0m dask_cudf[38;5;241m.[39mfrom_cudf(_x, sort[38;5;241m=[39m[38;5;28;01mFalse[39;00m, npartitions[38;5;241m=[39mnpartitions)
E [1;32m 571[0m [38;5;28;01mif[39;00m to_collection
E [1;32m 572[0m [38;5;28;01melse[39;00m _x
E [1;32m 573[0m )
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/nvtx/nvtx.py:101[0m, in [0;36mannotate.call..inner[0;34m(args, **kwargs)[0m
E [1;32m 98[0m [38;5;129m@wraps[39m(func)
E [1;32m 99[0m [38;5;28;01mdef[39;00m [38;5;21minner[39m([38;5;241m[39margs, [38;5;241m[39m[38;5;241m[39mkwargs):
E [1;32m 100[0m libnvtx_push_range([38;5;28mself[39m[38;5;241m.[39mattributes, [38;5;28mself[39m[38;5;241m.[39mdomain[38;5;241m.[39mhandle)
E [0;32m--> 101[0m result [38;5;241m=[39m [43mfunc[49m[43m([49m[38;5;241;43m[39;49m[43margs[49m[43m,[49m[43m [49m[38;5;241;43m[39;49m[38;5;241;43m*[39;49m[43mkwargs[49m[43m)[49m
E [1;32m 102[0m libnvtx_pop_range([38;5;28mself[39m[38;5;241m.[39mdomain[38;5;241m.[39mhandle)
E [1;32m 103[0m [38;5;28;01mreturn[39;00m result
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:4547[0m, in [0;36mDataFrame.from_pandas[0;34m(cls, dataframe, nan_as_null)[0m
E [1;32m 4543[0m [38;5;28;01mfor[39;00m col_name, col_value [38;5;129;01min[39;00m dataframe[38;5;241m.[39mitems():
E [1;32m 4544[0m [38;5;66;03m# necessary because multi-index can return multiple[39;00m
E [1;32m 4545[0m [38;5;66;03m# columns for a single key[39;00m
E [1;32m 4546[0m [38;5;28;01mif[39;00m [38;5;28mlen[39m(col_value[38;5;241m.[39mshape) [38;5;241m==[39m [38;5;241m1[39m:
E [0;32m-> 4547[0m df[col_name] [38;5;241m=[39m [43mcolumn[49m[38;5;241;43m.[39;49m[43mas_column[49m[43m([49m
E [1;32m 4548[0m [43m [49m[43mcol_value[49m[38;5;241;43m.[39;49m[43marray[49m[43m,[49m[43m [49m[43mnan_as_null[49m[38;5;241;43m=[39;49m[43mnan_as_null[49m
E [1;32m 4549[0m [43m [49m[43m)[49m
E [1;32m 4550[0m [38;5;28;01melse[39;00m:
E [1;32m 4551[0m vals [38;5;241m=[39m col_value[38;5;241m.[39mvalues[38;5;241m.[39mT
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:1966[0m, in [0;36mas_column[0;34m(arbitrary, nan_as_null, dtype, length)[0m
E [1;32m 1964[0m data [38;5;241m=[39m as_column(pa[38;5;241m.[39mArray[38;5;241m.[39mfrom_pandas(arbitrary), dtype[38;5;241m=[39marb_dtype)
E [1;32m 1965[0m [38;5;28;01melse[39;00m:
E [0;32m-> 1966[0m data [38;5;241m=[39m [43mas_column[49m[43m([49m
E [1;32m 1967[0m [43m [49m[43mpa[49m[38;5;241;43m.[39;49m[43marray[49m[43m([49m
E [1;32m 1968[0m [43m [49m[43marbitrary[49m[43m,[49m
E [1;32m 1969[0m [43m [49m[43mfrom_pandas[49m[38;5;241;43m=[39;49m[38;5;28;43;01mTrue[39;49;00m[43m [49m[38;5;28;43;01mif[39;49;00m[43m [49m[43mnan_as_null[49m[43m [49m[38;5;129;43;01mis[39;49;00m[43m [49m[38;5;28;43;01mNone[39;49;00m[43m [49m[38;5;28;43;01melse[39;49;00m[43m [49m[43mnan_as_null[49m[43m,[49m
E [1;32m 1970[0m [43m [49m[43m)[49m[43m,[49m
E [1;32m 1971[0m [43m [49m[43mnan_as_null[49m[38;5;241;43m=[39;49m[43mnan_as_null[49m[43m,[49m
E [1;32m 1972[0m [43m [49m[43m)[49m
E [1;32m 1973[0m [38;5;28;01mif[39;00m dtype [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m:
E [1;32m 1974[0m data [38;5;241m=[39m data[38;5;241m.[39mastype(dtype)
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:1760[0m, in [0;36mas_column[0;34m(arbitrary, nan_as_null, dtype, length)[0m
E [1;32m 1754[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(arbitrary, pa[38;5;241m.[39mlib[38;5;241m.[39mHalfFloatArray):
E [1;32m 1755[0m [38;5;28;01mraise[39;00m [38;5;167;01mNotImplementedError[39;00m(
E [1;32m 1756[0m [38;5;124m"[39m[38;5;124mType casting from float16 to float32 is not [39m[38;5;124m"[39m
E [1;32m 1757[0m [38;5;124m"[39m[38;5;124myet supported in pyarrow, see: [39m[38;5;124m"[39m
E [1;32m 1758[0m [38;5;124m"[39m[38;5;124mhttps://issues.apache.org/jira/browse/ARROW-3802[39m[38;5;124m"[39m
E [1;32m 1759[0m )
E [0;32m-> 1760[0m col [38;5;241m=[39m [43mColumnBase[49m[38;5;241;43m.[39;49m[43mfrom_arrow[49m[43m([49m[43marbitrary[49m[43m)[49m
E [1;32m 1762[0m [38;5;28;01mif[39;00m [38;5;28misinstance[39m(arbitrary, pa[38;5;241m.[39mNullArray):
E [1;32m 1763[0m new_dtype [38;5;241m=[39m cudf[38;5;241m.[39mdtype(arbitrary[38;5;241m.[39mtype[38;5;241m.[39mto_pandas_dtype())
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/cudf/core/column/column.py:297[0m, in [0;36mColumnBase.from_arrow[0;34m(cls, array)[0m
E [1;32m 292[0m [38;5;28;01melif[39;00m [38;5;28misinstance[39m(
E [1;32m 293[0m array[38;5;241m.[39mtype, pd[38;5;241m.[39mcore[38;5;241m.[39marrays[38;5;241m.[39m_arrow_utils[38;5;241m.[39mArrowIntervalType
E [1;32m 294[0m ):
E [1;32m 295[0m [38;5;28;01mreturn[39;00m cudf[38;5;241m.[39mcore[38;5;241m.[39mcolumn[38;5;241m.[39mIntervalColumn[38;5;241m.[39mfrom_arrow(array)
E [0;32m--> 297[0m result [38;5;241m=[39m [43mlibcudf[49m[38;5;241;43m.[39;49m[43minterop[49m[38;5;241;43m.[39;49m[43mfrom_arrow[49m[43m([49m[43mdata[49m[43m)[49m[[38;5;241m0[39m]
E [1;32m 299[0m [38;5;28;01mreturn[39;00m result[38;5;241m.[39m_with_type_metadata(cudf_dtype_from_pa_type(array[38;5;241m.[39mtype))
E
E File [0;32mcudf/_lib/interop.pyx:150[0m, in [0;36mcudf._lib.interop.from_arrow[0;34m()[0m
E
E [0;31mRuntimeError[0m: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory
E RuntimeError: CUDA error encountered at: ../src/bitmask/null_mask.cu:93: 2 cudaErrorMemoryAllocation out of memory
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError
_________________ test_usecase_accelerate_training_by_lazyadam _________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f033ecd0d30>
@testbook(
REPO_ROOT / p,
timeout=180,
execute=False,
)
def test_usecase_accelerate_training_by_lazyadam(tb):
tb.inject(
"""
import os
os.environ["NUM_ROWS"] = "1000"
"""
)
tb.execute()
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py:22:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f033ecd0d30>
cell = {'cell_type': 'code', 'execution_count': 7, 'id': '0500ad25-29e0-40c8-85bc-6e3864107c6a', 'metadata': {'execution': {'...e_train_function_3859]']}], 'source': 'model1.compile(optimizer="adam")\nmodel1.fit(train, batch_size=1024, epochs=1)'}
cell_index = 12
exec_reply = {'buffers': [], 'content': {'ename': 'ResourceExhaustedError', 'engine_info': {'engine_id': -1, 'engine_uuid': '7a1e3b...e, 'engine': '7a1e3b1f-a99f-494c-b016-de57a5e15c5e', 'started': '2022-10-31T15:43:17.090162Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model1.compile(optimizer="adam")
E model1.fit(train, batch_size=1024, epochs=1)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mResourceExhaustedError[0m Traceback (most recent call last)
E Cell [0;32mIn [7], line 2[0m
E [1;32m 1[0m model1[38;5;241m.[39mcompile(optimizer[38;5;241m=[39m[38;5;124m"[39m[38;5;124madam[39m[38;5;124m"[39m)
E [0;32m----> 2[0m [43mmodel1[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[43mtrain[49m[43m,[49m[43m [49m[43mbatch_size[49m[38;5;241;43m=[39;49m[38;5;241;43m1024[39;49m[43m,[49m[43m [49m[43mepochs[49m[38;5;241;43m=[39;49m[38;5;241;43m1[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:839[0m, in [0;36mBaseModel.fit[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, train_metrics_steps, pre, **kwargs)[0m
E [1;32m 836[0m [38;5;28mself[39m[38;5;241m.[39m_reset_compile_cache()
E [1;32m 837[0m [38;5;28mself[39m[38;5;241m.[39mtrain_pre [38;5;241m=[39m pre
E [0;32m--> 839[0m out [38;5;241m=[39m [38;5;28;43msuper[39;49m[43m([49m[43m)[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[38;5;241;43m[39;49m[38;5;241;43m[39;49m[43mfit_kwargs[49m[43m)[49m
E [1;32m 841[0m [38;5;28;01mif[39;00m pre:
E [1;32m 842[0m [38;5;28;01mdel[39;00m [38;5;28mself[39m[38;5;241m.[39mtrain_pre
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:67[0m, in [0;36mfilter_traceback..error_handler[0;34m(*args, **kwargs)[0m
E [1;32m 65[0m [38;5;28;01mexcept[39;00m [38;5;167;01mException[39;00m [38;5;28;01mas[39;00m e: [38;5;66;03m# pylint: disable=broad-except[39;00m
E [1;32m 66[0m filtered_tb [38;5;241m=[39m process_traceback_frames(e[38;5;241m.[39m__traceback_)
E [0;32m---> 67[0m [38;5;28;01mraise[39;00m e[38;5;241m.[39mwith_traceback(filtered_tb) [38;5;28;01mfrom[39;00m [38;5;28mNone[39m
E [1;32m 68[0m [38;5;28;01mfinally[39;00m:
E [1;32m 69[0m [38;5;28;01mdel[39;00m filtered_tb
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py:54[0m, in [0;36mquick_execute[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)[0m
E [1;32m 52[0m [38;5;28;01mtry[39;00m:
E [1;32m 53[0m ctx[38;5;241m.[39mensure_initialized()
E [0;32m---> 54[0m tensors [38;5;241m=[39m pywrap_tfe[38;5;241m.[39mTFE_Py_Execute(ctx[38;5;241m.[39m_handle, device_name, op_name,
E [1;32m 55[0m inputs, attrs, num_outputs)
E [1;32m 56[0m [38;5;28;01mexcept[39;00m core[38;5;241m.[39m_NotOkStatusException [38;5;28;01mas[39;00m e:
E [1;32m 57[0m [38;5;28;01mif[39;00m name [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m:
E
E [0;31mResourceExhaustedError[0m: Graph execution error:
E
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_7091/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_7091/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
E ResourceExhaustedError: Graph execution error:
E
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_7091/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_7091/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-31 15:43:13.505472: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-31 15:43:15.695956: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-31 15:43:15.696113: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-31 15:43:15.696790: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-31 15:43:15.696845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13851 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
2022-10-31 15:43:27.273649: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 342425600/17069309952
2022-10-31 15:43:27.273701: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4217113500
MaxInUse: 4217113500
NumAllocs: 245
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 15:43:27.273721: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 15:43:27.273729: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 15:43:27.273736: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 15:43:27.273742: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 15:43:27.273748: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 15:43:27.273754: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8
2022-10-31 15:43:27.273761: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 7
2022-10-31 15:43:27.273766: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 15:43:27.273772: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 15:43:27.273778: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4
2022-10-31 15:43:27.273784: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 4
2022-10-31 15:43:27.273790: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 15:43:27.273796: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 15:43:27.273803: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 15:43:27.273809: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 15:43:27.273815: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 15:43:27.273821: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 15:43:27.273827: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 15:43:27.273833: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 4
2022-10-31 15:43:27.273839: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 15:43:27.273867: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 15:43:27.273875: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 15:43:27.273881: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 4
2022-10-31 15:43:27.273887: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 3
2022-10-31 15:43:27.273893: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 15:43:27.273918: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 15:43:27.285532: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 174653440/17069309952
2022-10-31 15:43:27.285577: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4398355676
MaxInUse: 4398355676
NumAllocs: 252
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 15:43:27.285596: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 15:43:27.285604: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 15:43:27.285612: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 15:43:27.285618: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 15:43:27.285625: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 15:43:27.285631: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8
2022-10-31 15:43:27.285638: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 7
2022-10-31 15:43:27.285644: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 15:43:27.285651: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 15:43:27.285657: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 15:43:27.285664: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 4
2022-10-31 15:43:27.285670: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 15:43:27.285677: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5
2022-10-31 15:43:27.285683: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 15:43:27.285690: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 15:43:27.285697: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 15:43:27.285703: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 15:43:27.285710: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 15:43:27.285716: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 15:43:27.285723: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 15:43:27.285729: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 5
2022-10-31 15:43:27.285736: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 15:43:27.285742: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 4
2022-10-31 15:43:27.285749: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 15:43:27.285781: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 15:43:27.285795: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 15:43:27.294632: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 135407776 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 107544576/17069309952
2022-10-31 15:43:27.294667: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4454699516
MaxInUse: 4454699516
NumAllocs: 259
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 15:43:27.294687: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 15:43:27.294697: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 15:43:27.294705: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 15:43:27.294713: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 15:43:27.294721: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 15:43:27.294729: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 15:43:27.294737: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8
2022-10-31 15:43:27.294744: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 15:43:27.294752: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 15:43:27.294760: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 15:43:27.294768: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 15:43:27.294776: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 15:43:27.294783: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5
2022-10-31 15:43:27.294791: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 15:43:27.294799: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 15:43:27.294807: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 147200, 1
2022-10-31 15:43:27.294815: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 15:43:27.294823: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 15:43:27.294831: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 15:43:27.294838: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 15:43:27.294846: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 5
2022-10-31 15:43:27.294854: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 5
2022-10-31 15:43:27.294862: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 5
2022-10-31 15:43:27.294870: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 4
2022-10-31 15:43:27.294878: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 15:43:27.294885: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 15:43:27.294899: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file8i0ssop5.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:294: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 2 failed, 812 passed, 12 skipped, 1344 warnings in 1713.27s (0:28:33) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins11866987217456358787.sh
Click to view CI Results
GitHub pull request #790 of commit 16d949f6293bf6b8789246b7714cce272cec1b5c, no merge conflicts.
Running as SYSTEM
Setting status of 16d949f6293bf6b8789246b7714cce272cec1b5c to PENDING with url https://10.20.13.93:8080/job/merlin_models/1658/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 16d949f6293bf6b8789246b7714cce272cec1b5c^{commit} # timeout=10
Checking out Revision 16d949f6293bf6b8789246b7714cce272cec1b5c (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 16d949f6293bf6b8789246b7714cce272cec1b5c # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 5e70f405cd339a0d39866cb51dacf6993b034a5e # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8204682001507294745.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 827 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 22%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 37%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ......................... [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 85%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file0x6o7qox.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 815 passed, 12 skipped, 1344 warnings in 1699.62s (0:28:19) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins1109554367126418022.sh
Click to view CI Results
GitHub pull request #790 of commit 9cd56ba8157467d084eac1ff7a49818ccbc52182, no merge conflicts.
Running as SYSTEM
Setting status of 9cd56ba8157467d084eac1ff7a49818ccbc52182 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1660/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 9cd56ba8157467d084eac1ff7a49818ccbc52182^{commit} # timeout=10
Checking out Revision 9cd56ba8157467d084eac1ff7a49818ccbc52182 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9cd56ba8157467d084eac1ff7a49818ccbc52182 # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 802b7c1465123680571bac6a07c51a992913df19 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1057344186700732849.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 827 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 22%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py F [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 37%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ......................... [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 85%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
_________________ test_usecase_accelerate_training_by_lazyadam _________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f3e5a96a250>
@testbook(
REPO_ROOT / p,
timeout=180,
execute=False,
)
def test_usecase_accelerate_training_by_lazyadam(tb):
tb.inject(
"""
import os
os.environ["NUM_ROWS"] = "1000"
"""
)
tb.execute()
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py:22:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f3e5a96a250>
cell = {'cell_type': 'code', 'execution_count': 7, 'id': '0500ad25-29e0-40c8-85bc-6e3864107c6a', 'metadata': {'execution': {'...e_train_function_3859]']}], 'source': 'model1.compile(optimizer="adam")\nmodel1.fit(train, batch_size=1024, epochs=1)'}
cell_index = 12
exec_reply = {'buffers': [], 'content': {'ename': 'ResourceExhaustedError', 'engine_info': {'engine_id': -1, 'engine_uuid': '8c3472...e, 'engine': '8c347270-8a37-4e34-a406-78f6fde93aa9', 'started': '2022-10-31T17:12:48.975226Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model1.compile(optimizer="adam")
E model1.fit(train, batch_size=1024, epochs=1)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mResourceExhaustedError[0m Traceback (most recent call last)
E Cell [0;32mIn [7], line 2[0m
E [1;32m 1[0m model1[38;5;241m.[39mcompile(optimizer[38;5;241m=[39m[38;5;124m"[39m[38;5;124madam[39m[38;5;124m"[39m)
E [0;32m----> 2[0m [43mmodel1[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[43mtrain[49m[43m,[49m[43m [49m[43mbatch_size[49m[38;5;241;43m=[39;49m[38;5;241;43m1024[39;49m[43m,[49m[43m [49m[43mepochs[49m[38;5;241;43m=[39;49m[38;5;241;43m1[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:839[0m, in [0;36mBaseModel.fit[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, train_metrics_steps, pre, **kwargs)[0m
E [1;32m 836[0m [38;5;28mself[39m[38;5;241m.[39m_reset_compile_cache()
E [1;32m 837[0m [38;5;28mself[39m[38;5;241m.[39mtrain_pre [38;5;241m=[39m pre
E [0;32m--> 839[0m out [38;5;241m=[39m [38;5;28;43msuper[39;49m[43m([49m[43m)[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[38;5;241;43m[39;49m[38;5;241;43m[39;49m[43mfit_kwargs[49m[43m)[49m
E [1;32m 841[0m [38;5;28;01mif[39;00m pre:
E [1;32m 842[0m [38;5;28;01mdel[39;00m [38;5;28mself[39m[38;5;241m.[39mtrain_pre
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:67[0m, in [0;36mfilter_traceback..error_handler[0;34m(*args, **kwargs)[0m
E [1;32m 65[0m [38;5;28;01mexcept[39;00m [38;5;167;01mException[39;00m [38;5;28;01mas[39;00m e: [38;5;66;03m# pylint: disable=broad-except[39;00m
E [1;32m 66[0m filtered_tb [38;5;241m=[39m process_traceback_frames(e[38;5;241m.[39m__traceback_)
E [0;32m---> 67[0m [38;5;28;01mraise[39;00m e[38;5;241m.[39mwith_traceback(filtered_tb) [38;5;28;01mfrom[39;00m [38;5;28mNone[39m
E [1;32m 68[0m [38;5;28;01mfinally[39;00m:
E [1;32m 69[0m [38;5;28;01mdel[39;00m filtered_tb
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py:54[0m, in [0;36mquick_execute[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)[0m
E [1;32m 52[0m [38;5;28;01mtry[39;00m:
E [1;32m 53[0m ctx[38;5;241m.[39mensure_initialized()
E [0;32m---> 54[0m tensors [38;5;241m=[39m pywrap_tfe[38;5;241m.[39mTFE_Py_Execute(ctx[38;5;241m.[39m_handle, device_name, op_name,
E [1;32m 55[0m inputs, attrs, num_outputs)
E [1;32m 56[0m [38;5;28;01mexcept[39;00m core[38;5;241m.[39m_NotOkStatusException [38;5;28;01mas[39;00m e:
E [1;32m 57[0m [38;5;28;01mif[39;00m name [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m:
E
E [0;31mResourceExhaustedError[0m: Graph execution error:
E
E Detected at node 'Adam/Adam/update_12/mul_4' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_23431/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse
E v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],
E Node: 'Adam/Adam/update_12/mul_4'
E Detected at node 'Adam/Adam/update_12/mul_4' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_23431/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse
E v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],
E Node: 'Adam/Adam/update_12/mul_4'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_12/mul_4}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_12/mul_4}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
E ResourceExhaustedError: Graph execution error:
E
E Detected at node 'Adam/Adam/update_12/mul_4' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_23431/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse
E v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],
E Node: 'Adam/Adam/update_12/mul_4'
E Detected at node 'Adam/Adam/update_12/mul_4' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_23431/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse
E v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],
E Node: 'Adam/Adam/update_12/mul_4'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_12/mul_4}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_12/mul_4}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-31 17:12:45.419949: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-31 17:12:47.594064: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-31 17:12:47.594223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-31 17:12:47.595017: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-31 17:12:47.595070: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13875 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
2022-10-31 17:12:58.982498: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 18678720 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.982560: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4292654100
MaxInUse: 4292654100
NumAllocs: 243
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.982584: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.982595: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.982605: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.982614: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 7
2022-10-31 17:12:58.982623: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.982631: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.982641: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8
2022-10-31 17:12:58.982649: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 4
2022-10-31 17:12:58.982658: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.982667: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4
2022-10-31 17:12:58.982676: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 17:12:58.982685: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.982695: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.982704: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.982713: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.982722: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 17:12:58.982731: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.982740: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 17:12:58.982749: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 4
2022-10-31 17:12:58.982758: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.982795: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.982806: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.982815: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.982824: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.982833: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.982863: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.983935: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.983960: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4292654588
MaxInUse: 4292654588
NumAllocs: 245
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.983980: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.983990: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.983999: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.984008: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.984017: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.984038: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.984048: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8
2022-10-31 17:12:58.984057: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 4
2022-10-31 17:12:58.984066: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.984074: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4
2022-10-31 17:12:58.984083: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 4
2022-10-31 17:12:58.984092: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.984100: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.984109: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.984118: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.984126: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 17:12:58.984141: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.984150: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 17:12:58.984158: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 4
2022-10-31 17:12:58.984167: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.984175: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.984184: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.984193: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.984202: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.984227: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.984241: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.985000: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 135407776 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.985022: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4293238940
MaxInUse: 4293238940
NumAllocs: 246
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.985039: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.985048: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.985057: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.985066: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.985075: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.985084: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.985092: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8
2022-10-31 17:12:58.985101: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 4
2022-10-31 17:12:58.985110: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.985119: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4
2022-10-31 17:12:58.985127: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 4
2022-10-31 17:12:58.985136: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.985145: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.985154: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.985162: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.985171: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.985180: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.985188: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 17:12:58.985197: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 4
2022-10-31 17:12:58.985206: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.985215: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.985223: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.985232: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.985241: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.985250: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.985261: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.986064: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 18678720 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.986098: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304991004
MaxInUse: 4304991004
NumAllocs: 253
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.986116: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.986126: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.986135: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.986143: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.986152: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.986160: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.986169: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.986177: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.986186: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.986195: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.986203: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.986212: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.986220: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.986229: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.986238: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.986246: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.986255: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.986264: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.986273: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.986281: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.986290: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.986298: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.986307: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.986316: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.986324: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.986336: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.987075: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 39970560 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.987095: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304991004
MaxInUse: 4304991004
NumAllocs: 253
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.987121: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.987132: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.987141: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.987149: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.987158: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.987167: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.987176: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.987185: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.987194: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.987203: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.987211: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.987220: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.987229: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.987237: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.987246: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.987255: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.987263: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.987272: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.987281: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.987289: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.987298: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.987307: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.987315: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.987324: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.987333: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.987344: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.988076: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 56589504 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.988098: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304991004
MaxInUse: 4304991004
NumAllocs: 253
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.988122: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.988143: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.988153: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 17:12:58.988162: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.988170: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.988179: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 6
2022-10-31 17:12:58.988188: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.988196: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.988205: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.988213: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.988222: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.988230: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.988239: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 17:12:58.988248: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.988256: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.988265: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.988274: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 17:12:58.988283: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.988291: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.988300: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.988309: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.988317: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.988326: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.988335: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.988343: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.988355: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.989142: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.989162: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4305818304
MaxInUse: 4305818304
NumAllocs: 258
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.989178: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.989187: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.989197: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 17:12:58.989205: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.989214: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.989234: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8
2022-10-31 17:12:58.989244: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.989253: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.989262: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.989271: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.989280: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.989288: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.989297: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5
2022-10-31 17:12:58.989306: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.989314: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.989323: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.989332: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 17:12:58.989341: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.989349: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.989358: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.989367: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.989376: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.989385: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.989393: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.989402: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.989414: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.990170: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 15401440 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.990191: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4305818432
MaxInUse: 4305818432
NumAllocs: 259
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.990206: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.990215: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.990224: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 17:12:58.990233: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.990242: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.990250: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 17:12:58.990259: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.990268: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.990276: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.990296: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.990307: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.990315: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.990324: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5
2022-10-31 17:12:58.990333: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.990341: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.990350: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.990359: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 17:12:58.990368: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.990377: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.990385: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.990394: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.990402: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.990411: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.990419: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.990428: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.990439: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 17:12:58.991193: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 56589504 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 5832704/17069309952
2022-10-31 17:12:58.991213: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4305818560
MaxInUse: 4305818560
NumAllocs: 260
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 17:12:58.991228: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 17:12:58.991238: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 17:12:58.991246: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 17:12:58.991255: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 17:12:58.991264: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 17:12:58.991273: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 10
2022-10-31 17:12:58.991281: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10
2022-10-31 17:12:58.991290: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5
2022-10-31 17:12:58.991299: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 17:12:58.991307: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 17:12:58.991316: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5
2022-10-31 17:12:58.991325: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 17:12:58.991334: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5
2022-10-31 17:12:58.991353: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 17:12:58.991364: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 17:12:58.991372: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 17:12:58.991381: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 17:12:58.991390: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 17:12:58.991399: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 17:12:58.991407: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4
2022-10-31 17:12:58.991416: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3
2022-10-31 17:12:58.991425: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 17:12:58.991433: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 17:12:58.991442: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 17:12:58.991451: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 17:12:58.991462: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_fileq1bii9cf.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 1 failed, 814 passed, 12 skipped, 1344 warnings in 1708.03s (0:28:28) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins5658968340034700549.sh
Click to view CI Results
GitHub pull request #790 of commit 9cd56ba8157467d084eac1ff7a49818ccbc52182, no merge conflicts.
Running as SYSTEM
Setting status of 9cd56ba8157467d084eac1ff7a49818ccbc52182 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1665/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 9cd56ba8157467d084eac1ff7a49818ccbc52182^{commit} # timeout=10
Checking out Revision 9cd56ba8157467d084eac1ff7a49818ccbc52182 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9cd56ba8157467d084eac1ff7a49818ccbc52182 # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 47a3cd881ab441ac1e89a8e61c2025e41a758215 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7157314346137095933.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 827 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 22%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py F [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 37%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ......................... [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 85%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=================================== FAILURES ===================================
_________________ test_usecase_accelerate_training_by_lazyadam _________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f830a3f3f70>
@testbook(
REPO_ROOT / p,
timeout=180,
execute=False,
)
def test_usecase_accelerate_training_by_lazyadam(tb):
tb.inject(
"""
import os
os.environ["NUM_ROWS"] = "1000"
"""
)
tb.execute()
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py:22:
/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute
super().execute_cell(cell, index)
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:85: in wrapped
return just_run(coro(*args, **kwargs))
/usr/local/lib/python3.8/dist-packages/nbclient/util.py:60: in just_run
return loop.run_until_complete(coro)
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:1025: in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
self = <testbook.client.TestbookNotebookClient object at 0x7f830a3f3f70>
cell = {'cell_type': 'code', 'execution_count': 7, 'id': '0500ad25-29e0-40c8-85bc-6e3864107c6a', 'metadata': {'execution': {'...e_train_function_3859]']}], 'source': 'model1.compile(optimizer="adam")\nmodel1.fit(train, batch_size=1024, epochs=1)'}
cell_index = 12
exec_reply = {'buffers': [], 'content': {'ename': 'ResourceExhaustedError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'e0567d...e, 'engine': 'e0567dce-7bac-44d1-88f9-9b0a961a5ef4', 'started': '2022-10-31T19:25:44.422830Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:
if exec_reply is None:
return None
exec_reply_content = exec_reply['content']
if exec_reply_content['status'] != 'error':
return None
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors
or exec_reply_content.get('ename') in self.allow_error_names
or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(
self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
)
if not cell_allows_errors:
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E ------------------
E model1.compile(optimizer="adam")
E model1.fit(train, batch_size=1024, epochs=1)
E ------------------
E
E [0;31m---------------------------------------------------------------------------[0m
E [0;31mResourceExhaustedError[0m Traceback (most recent call last)
E Cell [0;32mIn [7], line 2[0m
E [1;32m 1[0m model1[38;5;241m.[39mcompile(optimizer[38;5;241m=[39m[38;5;124m"[39m[38;5;124madam[39m[38;5;124m"[39m)
E [0;32m----> 2[0m [43mmodel1[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[43mtrain[49m[43m,[49m[43m [49m[43mbatch_size[49m[38;5;241;43m=[39;49m[38;5;241;43m1024[39;49m[43m,[49m[43m [49m[43mepochs[49m[38;5;241;43m=[39;49m[38;5;241;43m1[39;49m[43m)[49m
E
E File [0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:839[0m, in [0;36mBaseModel.fit[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, train_metrics_steps, pre, **kwargs)[0m
E [1;32m 836[0m [38;5;28mself[39m[38;5;241m.[39m_reset_compile_cache()
E [1;32m 837[0m [38;5;28mself[39m[38;5;241m.[39mtrain_pre [38;5;241m=[39m pre
E [0;32m--> 839[0m out [38;5;241m=[39m [38;5;28;43msuper[39;49m[43m([49m[43m)[49m[38;5;241;43m.[39;49m[43mfit[49m[43m([49m[38;5;241;43m[39;49m[38;5;241;43m[39;49m[43mfit_kwargs[49m[43m)[49m
E [1;32m 841[0m [38;5;28;01mif[39;00m pre:
E [1;32m 842[0m [38;5;28;01mdel[39;00m [38;5;28mself[39m[38;5;241m.[39mtrain_pre
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py:67[0m, in [0;36mfilter_traceback..error_handler[0;34m(*args, **kwargs)[0m
E [1;32m 65[0m [38;5;28;01mexcept[39;00m [38;5;167;01mException[39;00m [38;5;28;01mas[39;00m e: [38;5;66;03m# pylint: disable=broad-except[39;00m
E [1;32m 66[0m filtered_tb [38;5;241m=[39m process_traceback_frames(e[38;5;241m.[39m__traceback_)
E [0;32m---> 67[0m [38;5;28;01mraise[39;00m e[38;5;241m.[39mwith_traceback(filtered_tb) [38;5;28;01mfrom[39;00m [38;5;28mNone[39m
E [1;32m 68[0m [38;5;28;01mfinally[39;00m:
E [1;32m 69[0m [38;5;28;01mdel[39;00m filtered_tb
E
E File [0;32m/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py:54[0m, in [0;36mquick_execute[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)[0m
E [1;32m 52[0m [38;5;28;01mtry[39;00m:
E [1;32m 53[0m ctx[38;5;241m.[39mensure_initialized()
E [0;32m---> 54[0m tensors [38;5;241m=[39m pywrap_tfe[38;5;241m.[39mTFE_Py_Execute(ctx[38;5;241m.[39m_handle, device_name, op_name,
E [1;32m 55[0m inputs, attrs, num_outputs)
E [1;32m 56[0m [38;5;28;01mexcept[39;00m core[38;5;241m.[39m_NotOkStatusException [38;5;28;01mas[39;00m e:
E [1;32m 57[0m [38;5;28;01mif[39;00m name [38;5;129;01mis[39;00m [38;5;129;01mnot[39;00m [38;5;28;01mNone[39;00m:
E
E [0;31mResourceExhaustedError[0m: Graph execution error:
E
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_21766/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_21766/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
E ResourceExhaustedError: Graph execution error:
E
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_21766/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E Detected at node 'Adam/Adam/update_17/mul_1' defined at (most recent call last):
E File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
E return _run_code(code, main_globals, None,
E File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
E exec(code, run_globals)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in
E app.launch_new_instance()
E File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance
E app.start()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start
E self.io_loop.start()
E File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 215, in start
E self.asyncio_loop.run_forever()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
E self._run_once()
E File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
E handle._run()
E File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
E self._context.run(self._callback, *self._args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
E await self.process_one()
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
E await dispatch(*args)
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
E await result
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
E reply_content = await reply_content
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
E res = shell.run_cell(
E File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
E return super().run_cell(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell
E result = self._run_cell(
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell
E return runner(coro)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner
E coro.send(None)
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async
E has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes
E if await self.run_code(code, result, async=asy):
E File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code
E exec(code_obj, self.user_global_ns, self.user_ns)
E File "/tmp/ipykernel_21766/3741080137.py", line 2, in
E model1.fit(train, batch_size=1024, epochs=1)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 839, in fit
E out = super().fit(**fit_kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
E return fn(*args, **kwargs)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1409, in fit
E tmp_logs = self.train_function(iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1051, in train_function
E return step_function(self, iterator)
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1040, in step_function
E outputs = model.distribute_strategy.run(run_step, args=(data,))
E File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1030, in run_step
E outputs = model.train_step(data)
E File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 666, in train_step
E self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize
E return self.apply_gradients(grads_and_vars, name=name)
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients
E return tf.internal.distribute.interim.maybe_merge_call(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply
E update_op = distribution.extended.update(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var
E return self._resource_apply_sparse_duplicate_indices(
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices
E return self._resource_apply_sparse(summed_grad, handle, unique_indices,
E File "/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/adam.py", line 206, in _resource_apply_sparse
E m_t = tf.compat.v1.assign(m, m * coefficients['beta_1_t'],
E Node: 'Adam/Adam/update_17/mul_1'
E 2 root error(s) found.
E (0) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E (1) RESOURCE_EXHAUSTED: failed to allocate memory
E [[{{node Adam/Adam/update_17/mul_1}}]]
E Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
E
E 0 successful operations.
E 0 derived errors ignored. [Op:__inference_train_function_3859]
/usr/local/lib/python3.8/dist-packages/nbclient/client.py:919: CellExecutionError
----------------------------- Captured stderr call -----------------------------
2022-10-31 19:25:40.928993: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-31 19:25:43.027747: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0
2022-10-31 19:25:43.027887: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory: -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0
2022-10-31 19:25:43.028950: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1
2022-10-31 19:25:43.029009: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13875 MB memory: -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0
2022-10-31 19:25:54.415684: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 150536192/17069309952
2022-10-31 19:25:54.415739: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4159935420
MaxInUse: 4159935420
NumAllocs: 238
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.415757: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.415766: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.415773: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 19:25:54.415779: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.415785: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.415791: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 7
2022-10-31 19:25:54.415797: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 6
2022-10-31 19:25:54.415803: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.415809: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.415815: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.415821: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.415827: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.415833: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.415839: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.415845: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.415851: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 3
2022-10-31 19:25:54.415857: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 4
2022-10-31 19:25:54.415863: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.415870: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 4
2022-10-31 19:25:54.415876: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.415906: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.415914: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.415920: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.415926: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 3
2022-10-31 19:25:54.415932: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.415959: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.427521: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 18678720 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.427562: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177596
MaxInUse: 4304177596
NumAllocs: 243
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.427580: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.427589: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.427596: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 19:25:54.427603: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.427610: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.427616: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8
2022-10-31 19:25:54.427623: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 6
2022-10-31 19:25:54.427630: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.427636: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.427643: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.427649: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.427656: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.427663: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.427670: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.427676: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.427683: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.427690: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.427696: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.427703: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.427710: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.427716: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.427723: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.427730: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.427736: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.427764: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.427778: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.428388: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.428414: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177596
MaxInUse: 4304177596
NumAllocs: 243
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.428437: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.428451: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.428460: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 19:25:54.428467: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.428474: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.428480: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8
2022-10-31 19:25:54.428487: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 6
2022-10-31 19:25:54.428493: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.428501: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.428513: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.428523: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.428530: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.428536: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.428543: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.428549: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.428556: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.428563: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.428569: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.428576: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.428587: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.428599: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.428610: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.428622: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.428633: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.428640: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.428651: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.429375: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 39970560 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.429417: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177724
MaxInUse: 4304177724
NumAllocs: 244
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.429433: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.429441: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.429448: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 19:25:54.429455: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.429461: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.429468: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 19:25:54.429474: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 6
2022-10-31 19:25:54.429481: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.429488: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.429494: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.429501: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.429508: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.429514: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.429521: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.429528: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.429534: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.429541: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.429547: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.429554: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.429561: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.429567: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.429574: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.429581: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.429587: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.429594: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.429604: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.430159: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 15401440 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.430177: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177724
MaxInUse: 4304177724
NumAllocs: 244
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.430200: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.430209: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.430216: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33
2022-10-31 19:25:54.430222: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.430229: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.430236: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 19:25:54.430242: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 6
2022-10-31 19:25:54.430249: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.430256: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.430262: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.430269: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.430276: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.430282: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.430289: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.430296: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.430302: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.430309: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.430316: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.430322: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.430329: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.430336: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.430343: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.430349: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.430356: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.430363: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.430373: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.430928: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 135407776 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.430948: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177888
MaxInUse: 4304177888
NumAllocs: 246
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.430963: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.430982: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.430991: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 19:25:54.430999: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.431006: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.431013: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 19:25:54.431020: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 7
2022-10-31 19:25:54.431027: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.431034: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.431041: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.431048: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.431055: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.431062: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.431069: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.431076: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.431083: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.431090: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.431097: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.431104: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.431111: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.431118: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.431126: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.431132: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.431139: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.431147: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.431157: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.431684: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 56589504 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.431702: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4304177888
MaxInUse: 4304177888
NumAllocs: 246
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.431715: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.431723: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.431731: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 19:25:54.431738: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.431745: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.431763: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 19:25:54.431771: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 7
2022-10-31 19:25:54.431778: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.431785: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.431792: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 3
2022-10-31 19:25:54.431799: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.431806: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.431813: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.431820: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.431827: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.431834: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4
2022-10-31 19:25:54.431842: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.431849: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4
2022-10-31 19:25:54.431856: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.431863: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.431870: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.431878: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.431885: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.431892: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.431899: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.431909: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.432473: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 56589504 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.432493: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4309087264
MaxInUse: 4309087264
NumAllocs: 249
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.432507: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.432515: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.432523: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 19:25:54.432529: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.432537: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.432544: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9
2022-10-31 19:25:54.432551: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 7
2022-10-31 19:25:54.432558: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3
2022-10-31 19:25:54.432565: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.432582: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4
2022-10-31 19:25:54.432591: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.432598: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.432605: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 3
2022-10-31 19:25:54.432612: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.432619: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.432626: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 19:25:54.432633: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.432640: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 19:25:54.432647: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.432665: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.432672: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.432679: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.432686: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.432693: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.432699: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.432709: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
2022-10-31 19:25:54.433323: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 15401440 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)
Reported by CUDA: Free memory/Total memory: 16318464/17069309952
2022-10-31 19:25:54.433342: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit: 1706033152
InUse: 4309091200
MaxInUse: 4309091200
NumAllocs: 254
MaxAllocSize: 1083564064
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2022-10-31 19:25:54.433357: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;
2022-10-31 19:25:54.433365: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4
2022-10-31 19:25:54.433372: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34
2022-10-31 19:25:54.433379: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8
2022-10-31 19:25:54.433385: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2
2022-10-31 19:25:54.433392: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 10
2022-10-31 19:25:54.433399: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8
2022-10-31 19:25:54.433406: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 4
2022-10-31 19:25:54.433413: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 7
2022-10-31 19:25:54.433419: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5
2022-10-31 19:25:54.433426: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 3
2022-10-31 19:25:54.433433: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1
2022-10-31 19:25:54.433440: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4
2022-10-31 19:25:54.433458: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3
2022-10-31 19:25:54.433466: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3
2022-10-31 19:25:54.433473: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5
2022-10-31 19:25:54.433480: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5
2022-10-31 19:25:54.433486: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5
2022-10-31 19:25:54.433493: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5
2022-10-31 19:25:54.433500: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3
2022-10-31 19:25:54.433507: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 4
2022-10-31 19:25:54.433514: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 4
2022-10-31 19:25:54.433521: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 3
2022-10-31 19:25:54.433528: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 4
2022-10-31 19:25:54.433534: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3
2022-10-31 19:25:54.433544: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown
h.close()
File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close
self.stream.close()
File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close
self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_file9ow08nmf.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/distributed/node.py:177: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 46549 instead
warnings.warn(
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
==== 1 failed, 814 passed, 12 skipped, 1345 warnings in 1718.50s (0:28:38) =====
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins8930170084743811908.sh
Click to view CI Results
GitHub pull request #790 of commit 9cd56ba8157467d084eac1ff7a49818ccbc52182, no merge conflicts.
Running as SYSTEM
Setting status of 9cd56ba8157467d084eac1ff7a49818ccbc52182 to PENDING with url https://10.20.13.93:8080/job/merlin_models/1667/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
> git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/790/*:refs/remotes/origin/pr/790/* # timeout=10
> git rev-parse 9cd56ba8157467d084eac1ff7a49818ccbc52182^{commit} # timeout=10
Checking out Revision 9cd56ba8157467d084eac1ff7a49818ccbc52182 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9cd56ba8157467d084eac1ff7a49818ccbc52182 # timeout=10
Commit message: "Merge branch 'main' into tf/update-retrieval"
> git rev-list --no-walk 69938bf2a34d351af1c8fb2a35388439ef858ac2 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins17517111180834351603.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 827 items
tests/unit/config/test_schema.py .... [ 0%]
tests/unit/datasets/test_advertising.py .s [ 0%]
tests/unit/datasets/test_ecommerce.py ..sss [ 1%]
tests/unit/datasets/test_entertainment.py ....sss. [ 2%]
tests/unit/datasets/test_social.py . [ 2%]
tests/unit/datasets/test_synthetic.py ...... [ 3%]
tests/unit/implicit/test_implicit.py . [ 3%]
tests/unit/lightfm/test_lightfm.py . [ 3%]
tests/unit/tf/test_core.py ...... [ 4%]
tests/unit/tf/test_loader.py ................ [ 6%]
tests/unit/tf/test_public_api.py . [ 6%]
tests/unit/tf/blocks/test_cross.py ........... [ 7%]
tests/unit/tf/blocks/test_dlrm.py .......... [ 8%]
tests/unit/tf/blocks/test_interactions.py ... [ 9%]
tests/unit/tf/blocks/test_mlp.py ................................. [ 13%]
tests/unit/tf/blocks/test_optimizer.py s................................ [ 17%]
..................... [ 19%]
tests/unit/tf/blocks/retrieval/test_base.py . [ 19%]
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py .. [ 19%]
tests/unit/tf/blocks/retrieval/test_two_tower.py ............ [ 21%]
tests/unit/tf/blocks/sampling/test_cross_batch.py . [ 21%]
tests/unit/tf/blocks/sampling/test_in_batch.py . [ 21%]
tests/unit/tf/core/test_aggregation.py ......... [ 22%]
tests/unit/tf/core/test_base.py .. [ 22%]
tests/unit/tf/core/test_combinators.py s.................... [ 25%]
tests/unit/tf/core/test_encoder.py .. [ 25%]
tests/unit/tf/core/test_index.py ... [ 26%]
tests/unit/tf/core/test_prediction.py .. [ 26%]
tests/unit/tf/core/test_tabular.py ...... [ 27%]
tests/unit/tf/examples/test_01_getting_started.py . [ 27%]
tests/unit/tf/examples/test_02_dataschema.py . [ 27%]
tests/unit/tf/examples/test_03_exploring_different_models.py . [ 27%]
tests/unit/tf/examples/test_04_export_ranking_models.py . [ 27%]
tests/unit/tf/examples/test_05_export_retrieval_model.py . [ 27%]
tests/unit/tf/examples/test_06_advanced_own_architecture.py . [ 27%]
tests/unit/tf/examples/test_07_train_traditional_models.py . [ 27%]
tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 28%]
[ 28%]
tests/unit/tf/examples/test_usecase_ecommerce_session_based.py . [ 28%]
tests/unit/tf/examples/test_usecase_pretrained_embeddings.py . [ 28%]
tests/unit/tf/inputs/test_continuous.py ....... [ 29%]
tests/unit/tf/inputs/test_embedding.py ................................. [ 33%]
........ [ 34%]
tests/unit/tf/inputs/test_tabular.py .................. [ 36%]
tests/unit/tf/layers/test_queue.py .............. [ 37%]
tests/unit/tf/losses/test_losses.py ....................... [ 40%]
tests/unit/tf/metrics/test_metrics_popularity.py ..... [ 41%]
tests/unit/tf/metrics/test_metrics_topk.py ......................... [ 44%]
tests/unit/tf/models/test_base.py s....................... [ 47%]
tests/unit/tf/models/test_benchmark.py .. [ 47%]
tests/unit/tf/models/test_ranking.py .................................. [ 51%]
tests/unit/tf/models/test_retrieval.py ................................. [ 55%]
.......................................... [ 60%]
tests/unit/tf/outputs/test_base.py ...... [ 61%]
tests/unit/tf/outputs/test_classification.py ...... [ 62%]
tests/unit/tf/outputs/test_contrastive.py .............. [ 63%]
tests/unit/tf/outputs/test_regression.py .. [ 64%]
tests/unit/tf/outputs/test_sampling.py .... [ 64%]
tests/unit/tf/outputs/test_topk.py . [ 64%]
tests/unit/tf/prediction_tasks/test_classification.py .. [ 64%]
tests/unit/tf/prediction_tasks/test_multi_task.py ................ [ 66%]
tests/unit/tf/prediction_tasks/test_next_item.py ..... [ 67%]
tests/unit/tf/prediction_tasks/test_regression.py ..... [ 68%]
tests/unit/tf/prediction_tasks/test_retrieval.py . [ 68%]
tests/unit/tf/prediction_tasks/test_sampling.py ...... [ 68%]
tests/unit/tf/transformers/test_block.py ..................... [ 71%]
tests/unit/tf/transformers/test_transforms.py .......... [ 72%]
tests/unit/tf/transforms/test_bias.py .. [ 72%]
tests/unit/tf/transforms/test_features.py s............................. [ 76%]
....................s...... [ 79%]
tests/unit/tf/transforms/test_negative_sampling.py ......... [ 80%]
tests/unit/tf/transforms/test_noise.py ..... [ 81%]
tests/unit/tf/transforms/test_sequence.py .................... [ 83%]
tests/unit/tf/transforms/test_tensor.py ... [ 84%]
tests/unit/tf/utils/test_batch.py .... [ 84%]
tests/unit/tf/utils/test_dataset.py .. [ 85%]
tests/unit/tf/utils/test_tf_utils.py ..... [ 85%]
tests/unit/torch/test_dataset.py ......... [ 86%]
tests/unit/torch/test_public_api.py . [ 86%]
tests/unit/torch/block/test_base.py .... [ 87%]
tests/unit/torch/block/test_mlp.py . [ 87%]
tests/unit/torch/features/test_continuous.py .. [ 87%]
tests/unit/torch/features/test_embedding.py .............. [ 89%]
tests/unit/torch/features/test_tabular.py .... [ 89%]
tests/unit/torch/model/test_head.py ............ [ 91%]
tests/unit/torch/model/test_model.py .. [ 91%]
tests/unit/torch/tabular/test_aggregation.py ........ [ 92%]
tests/unit/torch/tabular/test_tabular.py ... [ 92%]
tests/unit/torch/tabular/test_transformations.py ....... [ 93%]
tests/unit/utils/test_schema_utils.py ................................ [ 97%]
tests/unit/xgb/test_xgboost.py .................... [100%]
=============================== warnings summary ===============================
../../../../../usr/lib/python3/dist-packages/requests/init.py:89
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': pil_image.HAMMING,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': pil_image.BOX,
../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41
/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': pil_image.LANCZOS,
tests/unit/datasets/test_advertising.py: 1 warning
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 6 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 6 warnings
tests/unit/tf/core/test_index.py: 8 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 38 warnings
tests/unit/tf/models/test_retrieval.py: 119 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning
tests/unit/tf/transformers/test_block.py: 15 warnings
tests/unit/tf/transforms/test_bias.py: 2 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_noise.py: 1 warning
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 9 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 3 warnings
tests/unit/xgb/test_xgboost.py: 18 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_ecommerce.py: 2 warnings
tests/unit/datasets/test_entertainment.py: 4 warnings
tests/unit/datasets/test_social.py: 1 warning
tests/unit/datasets/test_synthetic.py: 5 warnings
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_core.py: 6 warnings
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/test_cross.py: 5 warnings
tests/unit/tf/blocks/test_dlrm.py: 9 warnings
tests/unit/tf/blocks/test_interactions.py: 2 warnings
tests/unit/tf/blocks/test_mlp.py: 26 warnings
tests/unit/tf/blocks/test_optimizer.py: 30 warnings
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings
tests/unit/tf/core/test_aggregation.py: 6 warnings
tests/unit/tf/core/test_base.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 7 warnings
tests/unit/tf/core/test_index.py: 3 warnings
tests/unit/tf/core/test_prediction.py: 2 warnings
tests/unit/tf/inputs/test_continuous.py: 6 warnings
tests/unit/tf/inputs/test_embedding.py: 20 warnings
tests/unit/tf/inputs/test_tabular.py: 18 warnings
tests/unit/tf/models/test_base.py: 26 warnings
tests/unit/tf/models/test_benchmark.py: 2 warnings
tests/unit/tf/models/test_ranking.py: 36 warnings
tests/unit/tf/models/test_retrieval.py: 85 warnings
tests/unit/tf/outputs/test_base.py: 6 warnings
tests/unit/tf/outputs/test_classification.py: 6 warnings
tests/unit/tf/outputs/test_contrastive.py: 19 warnings
tests/unit/tf/outputs/test_regression.py: 2 warnings
tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings
tests/unit/tf/transformers/test_block.py: 9 warnings
tests/unit/tf/transforms/test_features.py: 10 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings
tests/unit/tf/transforms/test_sequence.py: 15 warnings
tests/unit/tf/utils/test_batch.py: 7 warnings
tests/unit/tf/utils/test_dataset.py: 2 warnings
tests/unit/torch/block/test_base.py: 4 warnings
tests/unit/torch/block/test_mlp.py: 1 warning
tests/unit/torch/features/test_continuous.py: 1 warning
tests/unit/torch/features/test_embedding.py: 4 warnings
tests/unit/torch/features/test_tabular.py: 4 warnings
tests/unit/torch/model/test_head.py: 12 warnings
tests/unit/torch/model/test_model.py: 2 warnings
tests/unit/torch/tabular/test_aggregation.py: 6 warnings
tests/unit/torch/tabular/test_transformations.py: 2 warnings
tests/unit/xgb/test_xgboost.py: 17 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/datasets/test_entertainment.py: 1 warning
tests/unit/implicit/test_implicit.py: 1 warning
tests/unit/lightfm/test_lightfm.py: 1 warning
tests/unit/tf/test_loader.py: 1 warning
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings
tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings
tests/unit/tf/core/test_combinators.py: 11 warnings
tests/unit/tf/core/test_encoder.py: 2 warnings
tests/unit/tf/core/test_prediction.py: 1 warning
tests/unit/tf/inputs/test_continuous.py: 4 warnings
tests/unit/tf/inputs/test_embedding.py: 9 warnings
tests/unit/tf/inputs/test_tabular.py: 8 warnings
tests/unit/tf/models/test_ranking.py: 20 warnings
tests/unit/tf/models/test_retrieval.py: 10 warnings
tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings
tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings
tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings
tests/unit/xgb/test_xgboost.py: 12 warnings
/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].
warnings.warn(
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export
tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:960: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning
tests/unit/tf/core/test_index.py: 4 warnings
tests/unit/tf/models/test_retrieval.py: 54 warnings
tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings
tests/unit/tf/utils/test_batch.py: 2 warnings
/tmp/autograph_generated_filek76efvsz.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:614: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
elif isinstance(self.feature_names, collections.Sequence):
tests/unit/tf/core/test_encoder.py: 1 warning
tests/unit/tf/core/test_index.py: 5 warnings
tests/unit/tf/models/test_retrieval.py: 30 warnings
tests/unit/tf/utils/test_batch.py: 4 warnings
tests/unit/tf/utils/test_dataset.py: 1 warning
/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.
warnings.warn(
tests/unit/tf/models/test_base.py::test_model_pre_post[True]
tests/unit/tf/models/test_base.py::test_model_pre_post[False]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]
tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
return dispatch_target(*args, **kwargs)
tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]
tests/unit/tf/models/test_base.py::test_freeze_sequential_block
tests/unit/tf/models/test_base.py::test_freeze_unfreeze
tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.
super(SGD, self).init(name, **kwargs)
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_base.py::test_retrieval_model_query
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]
tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack
tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))
tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]
tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag
return py_builtins.overload_of(f)(*args)
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]
tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]
/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
warnings.warn(
tests/unit/torch/block/test_mlp.py::test_mlp_block
/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:201.)
return {key: torch.tensor(value) for key, value in data.items()}
tests/unit/xgb/test_xgboost.py::test_without_dask_client
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]
tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]
tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple
tests/unit/xgb/test_xgboost.py::TestEvals::test_default
tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid
tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data
/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:335: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].
warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")
tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective
/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
self.make_current()
tests/unit/xgb/test_xgboost.py: 14 warnings
/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited
client.wait_for_workers(n_workers)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/unit/xgb/test_xgboost.py: 11 warnings
/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
mask = pd.Series(mask)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP
SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR
SKIPPED [5] ../../../../../usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/test_util.py:2746: Not a test.
========= 815 passed, 12 skipped, 1344 warnings in 1702.53s (0:28:22) ==========
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[merlin_models] $ /bin/bash /tmp/jenkins3933472541945255394.sh