rasa
rasa copied to clipboard
RasaHQ
Fix data validator ignore_warnings logic
Proposed changes:
- Fixes the logic for the
ignore_warningsparameter insidevalidator.py - Explicitly specifies
ignore_warnings=Falseintest_validator.py - Fixes failing tests
- Adds a test
test_verify_utterances_does_not_error_when_no_utterance_template_providedto make sure that utterances specified without a template are not considered a validation error
Status (please check what you already did):
- [x] added some tests for the functionality
- [ ] updated the documentation
- [ ] updated the changelog (please check changelog for instructions)
- [x] reformat files using
black(please check Readme for instructions)
This PR resolves 11843
Thanks a lot for that bug fix and adding a test for it 🔥
My pleasure. I fixed one linter issue but I see some other validation errors in CI which I am not aware of...
I've just approved the CI runs. Once they go green the PR should be automatically merged. Thanks for your contribution 🙏🏻
Could not update branch. Most likely this is due to a merge conflict. Please update the branch manually and fix any issues. Error: HttpError: user doesn't have permission to update head repository
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
it seems that the test-nlu-predictors now consistently fail on Ubuntu, can you look into it?
@m-vdb
it seems that the
test-nlu-predictorsnow consistently fail on Ubuntu, can you look into it?
This is strange, if I run the tests locally they pass:
$ cat /etc/issue.net
Ubuntu 20.04.5 LTS
$ make test-nlu-predictors
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
rm -rf build/
rm -rf .mypy_cache/
rm -rf dist/
rm -rf docs/build
rm -rf docs/.docusaurus
# TF_CPP_MIN_LOG_LEVEL=2 sets C code log level for tensorflow to error suppressing lower log events
OMP_NUM_THREADS=1 TF_CPP_MIN_LOG_LEVEL=2 poetry run pytest tests -n 1 --dist loadscope -m "category_nlu_predictors and (not flaky)" --cov rasa --ignore tests/integration_tests/
================================================= test session starts ==================================================
platform linux -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/dimitri/Projects/idtlsm/rasa, configfile: pytest.ini
plugins: anyio-3.6.2, timeout-2.1.0, cov-4.0.0, asyncio-0.20.3, xdist-3.1.0, typeguard-2.13.3, sanic-1.9.1
timeout: 60.0s
timeout method: signal
timeout func_only: True
asyncio: mode=strict
gw0 [174]
................................................................................................................ [ 64%]
.............................................................. [100%]
=================================================== warnings summary ===================================================
rasa/engine/caching.py:152
rasa/engine/caching.py:152
/home/dimitri/Projects/idtlsm/rasa/rasa/engine/caching.py:152: MovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
Base: DeclarativeMeta = declarative_base()
.venv/lib/python3.10/site-packages/sanic_cors/extension.py:39
.venv/lib/python3.10/site-packages/sanic_cors/extension.py:39
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/sanic_cors/extension.py:39: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
SANIC_VERSION = LooseVersion(sanic_version)
.venv/lib/python3.10/site-packages/sanic_cors/extension.py:40
.venv/lib/python3.10/site-packages/sanic_cors/extension.py:40
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/sanic_cors/extension.py:40: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
SANIC_21_9_0 = LooseVersion("21.9.0")
.venv/lib/python3.10/site-packages/future/standard_library/__init__.py:65
.venv/lib/python3.10/site-packages/future/standard_library/__init__.py:65
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/future/standard_library/__init__.py:65: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
import imp
.venv/lib/python3.10/site-packages/xdist/plugin.py:227
.venv/lib/python3.10/site-packages/xdist/plugin.py:227
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/xdist/plugin.py:227: DeprecationWarning: The --rsyncdir command line argument and rsyncdirs config variable are deprecated.
The rsync feature will be removed in pytest-xdist 4.0.
config.issue_config_time_warning(warning, 2)
.venv/lib/python3.10/site-packages/pytest_sanic/utils.py:168
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/pytest_sanic/utils.py:168: PytestCollectionWarning: cannot collect test class 'TestClient' because it has a __init__ constructor (from: tests/core/test_agent.py)
class TestClient:
tests/core/test_evaluation.py:352
/home/dimitri/Projects/idtlsm/rasa/tests/core/test_evaluation.py:352: PytestUnknownMarkWarning: Unknown pytest.mark.skip_on_ci - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
@pytest.mark.skip_on_ci
tests/core/test_processor.py:1484
/home/dimitri/Projects/idtlsm/rasa/tests/core/test_processor.py:1484: PytestUnknownMarkWarning: Unknown pytest.mark.skip_on_ci - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
@pytest.mark.skip_on_ci
tests/core/test_test.py:110
/home/dimitri/Projects/idtlsm/rasa/tests/core/test_test.py:110: PytestUnknownMarkWarning: Unknown pytest.mark.skip_on_ci - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
@pytest.mark.skip_on_ci
<frozen importlib._bootstrap>:283: 3 warnings
tests/nlu/classifiers/test_diet_classifier.py: 46 warnings
tests/nlu/classifiers/test_sklearn_classifier.py: 3 warnings
tests/nlu/extractors/test_crf_entity_extractor.py: 7 warnings
tests/nlu/extractors/test_extractor.py: 5 warnings
tests/nlu/selectors/test_selectors.py: 43 warnings
<frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead
rasa/shared/core/slot_mappings.py:221
/home/dimitri/Projects/idtlsm/rasa/rasa/shared/core/slot_mappings.py:221: UserWarning: Slot auto-fill has been removed in 3.0 and replaced with a new explicit mechanism to set slots. Please refer to https://rasa.com/docs/rasa/domain#slots to learn more.
rasa.shared.utils.io.raise_warning(
tests/core/test_tracker_stores.py:733
/home/dimitri/Projects/idtlsm/rasa/tests/core/test_tracker_stores.py:733: SADeprecationWarning: Calling URL() directly is deprecated and will be disabled in a future release. The public constructor for URL is now the URL.create() method.
(URL(PGDialect.name), True),
tests/core/test_tracker_stores.py:734
/home/dimitri/Projects/idtlsm/rasa/tests/core/test_tracker_stores.py:734: SADeprecationWarning: Calling URL() directly is deprecated and will be disabled in a future release. The public constructor for URL is now the URL.create() method.
(URL(SQLiteDialect.name), False),
tests/engine/test_caching.py:30
/home/dimitri/Projects/idtlsm/rasa/tests/engine/test_caching.py:30: PytestCollectionWarning: cannot collect test class 'TestCacheableOutput' because it has a __init__ constructor (from: tests/engine/test_caching.py)
@dataclasses.dataclass
rasa/core/tracker_store.py:998
rasa/core/tracker_store.py:998
/home/dimitri/Projects/idtlsm/rasa/rasa/core/tracker_store.py:998: SADeprecationWarning: Calling URL() directly is deprecated and will be disabled in a future release. The public constructor for URL is now the URL.create() method.
return sa.engine.url.URL(
tests/utils/tensorflow/test_data_generator.py:70
/home/dimitri/Projects/idtlsm/rasa/tests/utils/tensorflow/test_data_generator.py:70: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(
tests/utils/tensorflow/test_data_generator.py:86
/home/dimitri/Projects/idtlsm/rasa/tests/utils/tensorflow/test_data_generator.py:86: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(
tests/utils/tensorflow/test_data_generator.py:93
/home/dimitri/Projects/idtlsm/rasa/tests/utils/tensorflow/test_data_generator.py:93: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(
tests/utils/tensorflow/test_data_generator.py:84
/home/dimitri/Projects/idtlsm/rasa/tests/utils/tensorflow/test_data_generator.py:84: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(
tests/utils/tensorflow/test_data_generator.py:149
/home/dimitri/Projects/idtlsm/rasa/tests/utils/tensorflow/test_data_generator.py:149: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(
tests/nlu/classifiers/test_diet_classifier.py::test_compute_default_label_features
/home/dimitri/Projects/idtlsm/rasa/rasa/utils/io.py:69: DeprecationWarning: There is no current event loop
"Enabling coroutine debugging. Loop id {}.".format(id(asyncio.get_event_loop()))
tests/nlu/classifiers/test_diet_classifier.py: 30 warnings
tests/nlu/selectors/test_selectors.py: 18 warnings
/home/dimitri/Projects/idtlsm/rasa/rasa/utils/train_utils.py:528: UserWarning: constrain_similarities is set to `False`. It is recommended to set it to `True` when using cross-entropy loss.
rasa.shared.utils.io.raise_warning(
tests/nlu/classifiers/test_diet_classifier.py: 15 warnings
tests/nlu/selectors/test_selectors.py: 14 warnings
/home/dimitri/Projects/idtlsm/rasa/rasa/shared/utils/io.py:98: UserWarning: 'evaluate_every_number_of_epochs=20' is greater than 'epochs=1'. No evaluation will occur.
warnings.warn(message, category=category, **kwargs)
tests/nlu/classifiers/test_diet_classifier.py: 27 warnings
tests/nlu/selectors/test_selectors.py: 10 warnings
/home/dimitri/Projects/idtlsm/rasa/rasa/utils/tensorflow/model_data.py:774: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.concatenate(np.array(f)),
tests/nlu/classifiers/test_diet_classifier.py::test_train_persist_load_with_different_settings
tests/nlu/classifiers/test_diet_classifier.py::test_margin_loss_is_not_normalized
tests/nlu/selectors/test_selectors.py::test_margin_loss_is_not_normalized[classifier_params0]
/home/dimitri/Projects/idtlsm/rasa/rasa/shared/utils/io.py:98: UserWarning: Overriding defaults by setting model_confidence to auto as loss_type is set to margin in the configuration. This means that model's confidences will be computed as cosine similarities. Users are encouraged to shift to cross entropy loss by setting `loss_type=cross_entropy`.
warnings.warn(message, category=category, **kwargs)
tests/nlu/classifiers/test_diet_classifier.py::test_train_persist_load_with_only_entity_recognition
tests/nlu/classifiers/test_diet_classifier.py::test_train_persist_load_with_composite_entities[classifier_params0]
tests/nlu/classifiers/test_diet_classifier.py::test_train_persist_load_with_composite_entities[classifier_params1]
/home/dimitri/Projects/idtlsm/rasa/rasa/utils/tensorflow/model_data_utils.py:400: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array([v[0] for v in values]), number_of_dimensions=3
tests/nlu/classifiers/test_keyword_classifier.py: 16 warnings
tests/nlu/extractors/test_crf_entity_extractor.py: 2 warnings
/home/dimitri/Projects/idtlsm/rasa/rasa/shared/nlu/training_data/formats/rasa.py:58: FutureWarning: NLU data in Rasa JSON format is deprecated and will be removed in Rasa Open Source 4.0.0. Please convert your JSON NLU data to the Rasa YAML format.
rasa.shared.utils.io.raise_deprecation_warning(
tests/nlu/classifiers/test_keyword_classifier.py::test_valid_data
/home/dimitri/Projects/idtlsm/rasa/.venv/lib/python3.10/site-packages/_pytest/python.py:195: PytestRemovedIn8Warning: Passing None has been deprecated.
See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
result = testfunction(**testargs)
tests/nlu/selectors/test_selectors.py::test_train_selector[config_params1]
tests/nlu/selectors/test_selectors.py::test_transformer_size_gets_corrected
/home/dimitri/Projects/idtlsm/rasa/rasa/nlu/selectors/response_selector.py:377: UserWarning: You have enabled a transformer inside ResponseSelector by setting a positive value for `number_of_transformer_layers`, but you left the hidden layer sizes at their default value: `hidden_layers_sizes={'text': [256, 128], 'label': [256, 128]}`. We recommend to disable the hidden layers when using a transformer, by specifying `hidden_layers_sizes={'text': [], 'label': []}`.
rasa.shared.utils.io.raise_warning(
tests/nlu/selectors/test_selectors.py::test_transformer_size_gets_corrected
/home/dimitri/Projects/idtlsm/rasa/rasa/nlu/selectors/response_selector.py:397: UserWarning: `transformer_size` is set to `None` for ResponseSelector, but a positive size is required when using `number_of_transformer_layers > 0`. ResponseSelector will proceed, using `transformer_size=256`. Alternatively, specify a different value in the component's config.
rasa.shared.utils.io.raise_warning(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.10.8-final-0 -----------
Name Stmts Miss Cover
--------------------------------------------------------------------------------------------
rasa/__init__.py 5 0 100%
rasa/__main__.py 57 36 37%
rasa/api.py 26 20 23%
rasa/cli/__init__.py 2 0 100%
rasa/cli/arguments/__init__.py 0 0 100%
rasa/cli/arguments/data.py 21 13 38%
rasa/cli/arguments/default_arguments.py 29 16 45%
rasa/cli/arguments/evaluate.py 16 10 38%
rasa/cli/arguments/export.py 9 5 44%
rasa/cli/arguments/interactive.py 33 24 27%
rasa/cli/arguments/run.py 41 32 22%
rasa/cli/arguments/shell.py 8 3 62%
rasa/cli/arguments/test.py 55 41 25%
rasa/cli/arguments/train.py 65 45 31%
rasa/cli/arguments/visualize.py 8 5 38%
rasa/cli/arguments/x.py 9 5 44%
rasa/cli/data.py 98 65 34%
rasa/cli/evaluate.py 55 37 33%
rasa/cli/export.py 78 53 32%
rasa/cli/interactive.py 56 34 39%
rasa/cli/run.py 59 46 22%
rasa/cli/scaffold.py 87 68 22%
rasa/cli/shell.py 57 41 28%
rasa/cli/telemetry.py 35 23 34%
rasa/cli/test.py 95 70 26%
rasa/cli/train.py 78 58 26%
rasa/cli/utils.py 70 49 30%
rasa/cli/visualize.py 16 8 50%
rasa/cli/x.py 77 52 32%
rasa/constants.py 26 0 100%
rasa/core/__init__.py 4 0 100%
rasa/core/actions/__init__.py 0 0 100%
rasa/core/actions/action.py 444 325 27%
rasa/core/actions/constants.py 2 0 100%
rasa/core/actions/forms.py 194 143 26%
rasa/core/actions/loops.py 36 21 42%
rasa/core/actions/two_stage_fallback.py 69 40 42%
rasa/core/agent.py 202 132 35%
rasa/core/brokers/__init__.py 0 0 100%
rasa/core/brokers/broker.py 55 34 38%
rasa/core/brokers/file.py 30 16 47%
rasa/core/brokers/kafka.py 122 92 25%
rasa/core/brokers/pika.py 126 88 30%
rasa/core/brokers/sql.py 40 15 62%
rasa/core/channels/__init__.py 19 0 100%
rasa/core/channels/botframework.py 174 127 27%
rasa/core/channels/callback.py 46 23 50%
rasa/core/channels/channel.py 132 76 42%
rasa/core/channels/console.py 116 81 30%
rasa/core/channels/facebook.py 190 127 33%
rasa/core/channels/hangouts.py 155 106 32%
rasa/core/channels/mattermost.py 97 64 34%
rasa/core/channels/rasa_chat.py 71 43 39%
rasa/core/channels/rest.py 79 48 39%
rasa/core/channels/rocketchat.py 80 51 36%
rasa/core/channels/slack.py 254 187 26%
rasa/core/channels/socketio.py 120 85 29%
rasa/core/channels/telegram.py 139 98 29%
rasa/core/channels/twilio.py 88 60 32%
rasa/core/channels/twilio_voice.py 115 79 31%
rasa/core/channels/webexteams.py 67 41 39%
rasa/core/constants.py 36 0 100%
rasa/core/evaluation/__init__.py 0 0 100%
rasa/core/evaluation/marker.py 124 44 65%
rasa/core/evaluation/marker_base.py 324 183 44%
rasa/core/evaluation/marker_stats.py 91 61 33%
rasa/core/evaluation/marker_tracker_loader.py 43 27 37%
rasa/core/exceptions.py 9 2 78%
rasa/core/exporter.py 100 74 26%
rasa/core/featurizers/__init__.py 0 0 100%
rasa/core/featurizers/precomputation.py 120 79 34%
rasa/core/featurizers/single_state_featurizer.py 115 82 29%
rasa/core/featurizers/tracker_featurizers.py 296 213 28%
rasa/core/http_interpreter.py 36 24 33%
rasa/core/jobs.py 25 16 36%
rasa/core/lock.py 65 35 46%
rasa/core/lock_store.py 140 81 42%
rasa/core/migrate.py 181 154 15%
rasa/core/nlg/__init__.py 3 0 100%
rasa/core/nlg/callback.py 34 19 44%
rasa/core/nlg/generator.py 34 21 38%
rasa/core/nlg/interpolator.py 29 23 21%
rasa/core/nlg/response.py 85 65 24%
rasa/core/policies/__init__.py 0 0 100%
rasa/core/policies/ensemble.py 84 54 36%
rasa/core/policies/memoization.py 184 130 29%
rasa/core/policies/policy.py 176 94 47%
rasa/core/policies/rule_policy.py 417 311 25%
rasa/core/policies/ted_policy.py 542 422 22%
rasa/core/policies/unexpected_intent_policy.py 234 144 38%
rasa/core/processor.py 332 246 26%
rasa/core/run.py 94 64 32%
rasa/core/test.py 438 334 24%
rasa/core/tracker_store.py 513 315 39%
rasa/core/train.py 34 20 41%
rasa/core/training/__init__.py 21 17 19%
rasa/core/training/converters/__init__.py 0 0 100%
rasa/core/training/converters/responses_prefix_converter.py 41 41 0%
rasa/core/training/interactive.py 697 570 18%
rasa/core/training/story_conflict.py 117 80 32%
rasa/core/training/training.py 38 26 32%
rasa/core/utils.py 132 98 26%
rasa/core/visualize.py 26 26 0%
rasa/engine/__init__.py 0 0 100%
rasa/engine/caching.py 180 110 39%
rasa/engine/constants.py 8 0 100%
rasa/engine/exceptions.py 4 0 100%
rasa/engine/graph.py 193 113 41%
rasa/engine/loader.py 9 3 67%
rasa/engine/recipes/__init__.py 0 0 100%
rasa/engine/recipes/default_components.py 31 0 100%
rasa/engine/recipes/default_recipe.py 391 292 25%
rasa/engine/recipes/graph_recipe.py 29 14 52%
rasa/engine/recipes/recipe.py 26 11 58%
rasa/engine/runner/__init__.py 0 0 100%
rasa/engine/runner/dask.py 43 24 44%
rasa/engine/runner/interface.py 11 0 100%
rasa/engine/storage/__init__.py 0 0 100%
rasa/engine/storage/local_model_storage.py 116 52 55%
rasa/engine/storage/resource.py 37 19 49%
rasa/engine/storage/storage.py 56 7 88%
rasa/engine/training/__init__.py 0 0 100%
rasa/engine/training/components.py 49 21 57%
rasa/engine/training/fingerprinting.py 19 5 74%
rasa/engine/training/graph_trainer.py 66 46 30%
rasa/engine/training/hooks.py 52 29 44%
rasa/engine/validation.py 194 147 24%
rasa/exceptions.py 22 3 86%
rasa/graph_components/__init__.py 0 0 100%
rasa/graph_components/converters/__init__.py 0 0 100%
rasa/graph_components/converters/nlu_message_converter.py 14 2 86%
rasa/graph_components/providers/__init__.py 0 0 100%
rasa/graph_components/providers/domain_for_core_training_provider.py 24 10 58%
rasa/graph_components/providers/domain_provider.py 32 15 53%
rasa/graph_components/providers/forms_provider.py 21 4 81%
rasa/graph_components/providers/nlu_training_data_provider.py 28 13 54%
rasa/graph_components/providers/responses_provider.py 21 4 81%
rasa/graph_components/providers/rule_only_provider.py 23 8 65%
rasa/graph_components/providers/story_graph_provider.py 18 4 78%
rasa/graph_components/providers/training_tracker_provider.py 20 5 75%
rasa/graph_components/validators/__init__.py 0 0 100%
rasa/graph_components/validators/default_recipe_validator.py 139 90 35%
rasa/graph_components/validators/finetuning_validator.py 92 56 39%
rasa/jupyter.py 31 31 0%
rasa/model.py 47 33 30%
rasa/model_testing.py 147 114 22%
rasa/model_training.py 130 93 28%
rasa/nlu/__init__.py 4 0 100%
rasa/nlu/classifiers/__init__.py 1 0 100%
rasa/nlu/classifiers/classifier.py 2 0 100%
rasa/nlu/classifiers/diet_classifier.py 601 85 86%
rasa/nlu/classifiers/fallback_classifier.py 72 2 97%
rasa/nlu/classifiers/keyword_intent_classifier.py 85 3 96%
rasa/nlu/classifiers/logistic_regression_classifier.py 94 11 88%
rasa/nlu/classifiers/mitie_intent_classifier.py 72 19 74%
rasa/nlu/classifiers/regex_message_handler.py 19 0 100%
rasa/nlu/classifiers/sklearn_intent_classifier.py 134 8 94%
rasa/nlu/constants.py 31 0 100%
rasa/nlu/convert.py 17 10 41%
rasa/nlu/emulators/__init__.py 0 0 100%
rasa/nlu/emulators/dialogflow.py 11 11 0%
rasa/nlu/emulators/emulator.py 16 9 44%
rasa/nlu/emulators/luis.py 23 16 30%
rasa/nlu/emulators/no_emulator.py 5 1 80%
rasa/nlu/emulators/wit.py 14 14 0%
rasa/nlu/extractors/__init__.py 0 0 100%
rasa/nlu/extractors/crf_entity_extractor.py 272 26 90%
rasa/nlu/extractors/duckling_entity_extractor.py 81 11 86%
rasa/nlu/extractors/entity_synonyms.py 76 13 83%
rasa/nlu/extractors/extractor.py 148 7 95%
rasa/nlu/extractors/mitie_entity_extractor.py 117 14 88%
rasa/nlu/extractors/regex_entity_extractor.py 73 0 100%
rasa/nlu/extractors/spacy_entity_extractor.py 39 3 92%
rasa/nlu/featurizers/__init__.py 0 0 100%
rasa/nlu/featurizers/dense_featurizer/__init__.py 0 0 100%
rasa/nlu/featurizers/dense_featurizer/convert_featurizer.py 160 97 39%
rasa/nlu/featurizers/dense_featurizer/dense_featurizer.py 22 4 82%
rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py 201 147 27%
rasa/nlu/featurizers/dense_featurizer/mitie_featurizer.py 65 26 60%
rasa/nlu/featurizers/dense_featurizer/spacy_featurizer.py 62 4 94%
rasa/nlu/featurizers/featurizer.py 34 5 85%
rasa/nlu/featurizers/sparse_featurizer/__init__.py 0 0 100%
rasa/nlu/featurizers/sparse_featurizer/count_vectors_featurizer.py 302 88 71%
rasa/nlu/featurizers/sparse_featurizer/lexical_syntactic_featurizer.py 170 20 88%
rasa/nlu/featurizers/sparse_featurizer/regex_featurizer.py 108 18 83%
rasa/nlu/featurizers/sparse_featurizer/sparse_featurizer.py 5 0 100%
rasa/nlu/model.py 10 3 70%
rasa/nlu/persistor.py 122 84 31%
rasa/nlu/run.py 18 18 0%
rasa/nlu/selectors/__init__.py 0 0 100%
rasa/nlu/selectors/response_selector.py 262 85 68%
rasa/nlu/test.py 461 344 25%
rasa/nlu/tokenizers/__init__.py 0 0 100%
rasa/nlu/tokenizers/jieba_tokenizer.py 73 39 47%
rasa/nlu/tokenizers/mitie_tokenizer.py 32 2 94%
rasa/nlu/tokenizers/spacy_tokenizer.py 33 4 88%
rasa/nlu/tokenizers/tokenizer.py 102 20 80%
rasa/nlu/tokenizers/whitespace_tokenizer.py 42 3 93%
rasa/nlu/utils/__init__.py 15 8 47%
rasa/nlu/utils/bilou_utils.py 147 61 59%
rasa/nlu/utils/hugging_face/__init__.py 0 0 100%
rasa/nlu/utils/hugging_face/registry.py 11 11 0%
rasa/nlu/utils/hugging_face/transformers_pre_post_processors.py 67 67 0%
rasa/nlu/utils/mitie_utils.py 39 4 90%
rasa/nlu/utils/pattern_utils.py 53 15 72%
rasa/nlu/utils/spacy_utils.py 116 9 92%
rasa/plugin.py 27 9 67%
rasa/server.py 599 519 13%
rasa/shared/__init__.py 0 0 100%
rasa/shared/constants.py 82 0 100%
rasa/shared/core/__init__.py 0 0 100%
rasa/shared/core/constants.py 68 2 97%
rasa/shared/core/conversation.py 15 4 73%
rasa/shared/core/domain.py 727 348 52%
rasa/shared/core/events.py 682 324 52%
rasa/shared/core/generator.py 322 264 18%
rasa/shared/core/slot_mappings.py 82 49 40%
rasa/shared/core/slots.py 184 106 42%
rasa/shared/core/trackers.py 307 174 43%
rasa/shared/core/training_data/__init__.py 0 0 100%
rasa/shared/core/training_data/loading.py 35 20 43%
rasa/shared/core/training_data/story_reader/__init__.py 0 0 100%
rasa/shared/core/training_data/story_reader/story_reader.py 54 32 41%
rasa/shared/core/training_data/story_reader/story_step_builder.py 83 63 24%
rasa/shared/core/training_data/story_reader/yaml_story_reader.py 373 222 40%
rasa/shared/core/training_data/story_writer/__init__.py 0 0 100%
rasa/shared/core/training_data/story_writer/story_writer.py 13 4 69%
rasa/shared/core/training_data/story_writer/yaml_story_writer.py 157 123 22%
rasa/shared/core/training_data/structures.py 351 260 26%
rasa/shared/core/training_data/visualization.py 245 203 17%
rasa/shared/data.py 75 23 69%
rasa/shared/exceptions.py 38 17 55%
rasa/shared/importers/__init__.py 0 0 100%
rasa/shared/importers/importer.py 213 115 46%
rasa/shared/importers/multi_project.py 110 77 30%
rasa/shared/importers/rasa.py 44 16 64%
rasa/shared/importers/utils.py 12 3 75%
rasa/shared/nlu/__init__.py 0 0 100%
rasa/shared/nlu/constants.py 38 0 100%
rasa/shared/nlu/interpreter.py 4 4 0%
rasa/shared/nlu/training_data/__init__.py 0 0 100%
rasa/shared/nlu/training_data/entities_parser.py 74 14 81%
rasa/shared/nlu/training_data/features.py 120 71 41%
rasa/shared/nlu/training_data/formats/__init__.py 5 0 100%
rasa/shared/nlu/training_data/formats/dialogflow.py 97 61 37%
rasa/shared/nlu/training_data/formats/luis.py 38 26 32%
rasa/shared/nlu/training_data/formats/rasa.py 43 9 79%
rasa/shared/nlu/training_data/formats/rasa_yaml.py 253 120 53%
rasa/shared/nlu/training_data/formats/readerwriter.py 90 52 42%
rasa/shared/nlu/training_data/formats/wit.py 33 23 30%
rasa/shared/nlu/training_data/loading.py 69 8 88%
rasa/shared/nlu/training_data/lookup_tables_parser.py 9 7 22%
rasa/shared/nlu/training_data/message.py 151 31 79%
rasa/shared/nlu/training_data/schemas/__init__.py 0 0 100%
rasa/shared/nlu/training_data/schemas/data_schema.py 13 0 100%
rasa/shared/nlu/training_data/synonyms_parser.py 11 0 100%
rasa/shared/nlu/training_data/training_data.py 270 140 48%
rasa/shared/nlu/training_data/util.py 77 30 61%
rasa/shared/utils/__init__.py 0 0 100%
rasa/shared/utils/cli.py 22 13 41%
rasa/shared/utils/common.py 80 48 40%
rasa/shared/utils/io.py 217 73 66%
rasa/shared/utils/pykwalify_extensions.py 9 2 78%
rasa/shared/utils/schemas/__init__.py 0 0 100%
rasa/shared/utils/schemas/events.py 29 0 100%
rasa/shared/utils/validation.py 111 53 52%
rasa/telemetry.py 311 193 38%
rasa/utils/__init__.py 0 0 100%
rasa/utils/common.py 222 156 30%
rasa/utils/converter.py 13 13 0%
rasa/utils/endpoints.py 117 68 42%
rasa/utils/io.py 96 44 54%
rasa/utils/plotting.py 131 106 19%
rasa/utils/tensorflow/__init__.py 0 0 100%
rasa/utils/tensorflow/callback.py 44 5 89%
rasa/utils/tensorflow/constants.py 82 0 100%
rasa/utils/tensorflow/crf.py 79 57 28%
rasa/utils/tensorflow/data_generator.py 110 30 73%
rasa/utils/tensorflow/environment.py 61 46 25%
rasa/utils/tensorflow/exceptions.py 2 0 100%
rasa/utils/tensorflow/layers.py 355 171 52%
rasa/utils/tensorflow/layers_utils.py 15 9 40%
rasa/utils/tensorflow/model_data.py 302 56 81%
rasa/utils/tensorflow/model_data_utils.py 154 10 94%
rasa/utils/tensorflow/models.py 281 26 91%
rasa/utils/tensorflow/rasa_layers.py 235 51 78%
rasa/utils/tensorflow/transformer.py 211 138 35%
rasa/utils/tensorflow/types.py 3 0 100%
rasa/utils/train_utils.py 162 46 72%
rasa/validator.py 176 141 20%
rasa/version.py 1 0 100%
--------------------------------------------------------------------------------------------
TOTAL 27533 15211 45%
==================================== 174 passed, 275 warnings in 1218.15s (0:20:18) ====================================
@m-vdb One change I observed is that I am using Python 3.10 and the failing test uses 3.8. Not sure if that is intended, but different tests in the set use different Python versions (integration tests are using 3.10, so I installed that locally).
but it seems these tests are failing on Ubuntu for all the python versions we support: 3.7, 3.8, 3.9 and 3.10 😬
but it seems these tests are failing on Ubuntu for all the python versions we support: 3.7, 3.8, 3.9 and 3.10 😬
Right, then I don't know what is happening. The tests pass on my local Ubuntu system. Furthermore, this seems to be unrelated to my PR since I only changed validation logic.
@m-vdb I've re-tested on Ubuntu 22.04.1 LTS / Python 3.10 and the tests still succeed:
===================== 174 passed, 275 warnings in 739.26s (0:12:19) ======================
@m-vdb I just found similarly failing tests in PR https://github.com/RasaHQ/rasa/pull/12074, for example: https://github.com/RasaHQ/rasa/actions/runs/4232026432/jobs/7351360693
@timidri I'm going to go ahead and merge this PR, thanks for putting up with our flaky CI checks. We'll make sure to address these pains in the near future
@timidri I'm going to go ahead and merge this PR, thanks for putting up with our flaky CI checks. We'll make sure to address these pains in the near future
Thank you!