error when using certain text string in lime tutorial notebook

[cwmeijer]

It seems there is something wrong still while handling special characters. See the input and error below. The text ran fine when I removed the special characters.

review = "I have seen many movies but this sure must have been one of the best movies I ever saw. The acting was rubbish though."

---

ValueError Traceback (most recent call last) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5000,) + inhomogeneous part.

The above exception was the direct cause of the following exception:

RuntimeError Traceback (most recent call last) Cell In[16], line 5 1 # We're getting the explanation for the 'positive' class only, 2 # but dianna supports explaining for multiple labels in one go. 3 # It therefore always outputs a list of saliency maps. We want 4 # the first and only saliency map from this list here. ----> 5 explanation_relevance = dianna.explain_text(model_runner, review, model_runner.tokenizer, 6 'LIME', labels=[labels.index('positive')])[0] 7 explanation_relevance

File /mnt/c/Users/ChristiaanMeijer/OneDrive - Netherlands eScience Center/Documents/dianna/dianna/dianna/init.py:122, in explain_text(model_or_function, input_text, tokenizer, method, labels, **kwargs) 120 if kwargs: 121 raise TypeError(f'Error due to following unused kwargs: {kwargs}') --> 122 return explainer.explain( 123 model_or_function=model_or_function, 124 input_text=input_text, 125 labels=labels, 126 tokenizer=tokenizer, 127 **explain_text_kwargs, 128 )

File /mnt/c/Users/ChristiaanMeijer/OneDrive - Netherlands eScience Center/Documents/dianna/dianna/dianna/methods/lime_text.py:91, in LIMEText.explain(self, model_or_function, input_text, labels, tokenizer, top_labels, num_features, num_samples, **kwargs) 88 runner = utils.get_function(model_or_function, preprocess_function=self.preprocess_function) 89 explain_instance_kwargs = utils.get_kwargs_applicable_to_function( 90 self.explainer.explain_instance, kwargs) ---> 91 explanation = self.explainer.explain_instance(input_text, 92 runner, 93 labels=labels, 94 top_labels=top_labels, 95 num_features=num_features, 96 num_samples=num_samples, 97 **explain_instance_kwargs 98 ) 100 local_explanations = explanation.local_exp 101 string_map = explanation.domain_mapper.indexed_string

File ~/anaconda3/envs/dianna/lib/python3.10/site-packages/lime/lime_text.py:413, in LimeTextExplainer.explain_instance(self, text_instance, classifier_fn, labels, top_labels, num_features, num_samples, distance_metric, model_regressor) 406 indexed_string = (IndexedCharacters( 407 text_instance, bow=self.bow, mask_string=self.mask_string) 408 if self.char_level else 409 IndexedString(text_instance, bow=self.bow, 410 split_expression=self.split_expression, 411 mask_string=self.mask_string)) 412 domain_mapper = TextDomainMapper(indexed_string) --> 413 data, yss, distances = self.__data_labels_distances( 414 indexed_string, classifier_fn, num_samples, 415 distance_metric=distance_metric) 416 if self.class_names is None: 417 self.class_names = [str(x) for x in range(yss[0].shape[0])]

File ~/anaconda3/envs/dianna/lib/python3.10/site-packages/lime/lime_text.py:482, in LimeTextExplainer.__data_labels_distances(self, indexed_string, classifier_fn, num_samples, distance_metric) 480 data[i, inactive] = 0 481 inverse_data.append(indexed_string.inverse_removing(inactive)) --> 482 labels = classifier_fn(inverse_data) 483 distances = distance_fn(sp.sparse.csr_matrix(data)) 484 return data, labels, distances

Cell In[5], line 27, in MovieReviewsModelRunner.call(self, sentences) 24 tokenized_sentences.append(tokens_numerical) 26 # run the model, applying a sigmoid because the model outputs logits ---> 27 logits = self.run_model(tokenized_sentences) 28 pred = np.apply_along_axis(sigmoid, 1, logits) 30 # output two classes

File /mnt/c/Users/ChristiaanMeijer/OneDrive - Netherlands eScience Center/Documents/dianna/dianna/dianna/utils/onnx_runner.py:33, in SimpleModelRunner.call(self, input_data) 30 input_data = self.preprocess_function(input_data) 32 onnx_input = {input_name: input_data} ---> 33 pred_onnx = sess.run([output_name], onnx_input)[0] 34 return pred_onnx

File ~/anaconda3/envs/dianna/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:217, in Session.run(self, output_names, input_feed, run_options) 215 output_names = [output.name for output in self._outputs_meta] 216 try: --> 217 return self._sess.run(output_names, input_feed, run_options) 218 except C.EPFail as err: 219 if self._enable_fallback:

RuntimeError: Could not create tensor from given input list