Adapter recommender cannot load models

[whisere]

sklearn_sentence: How to configure recommenders page setting such as layer and feature, I tried SemPred and category with http://localhost:5000/sklearn_sentence but it doesn't work

server.add_classifier("sklearn_sentence", SklearnSentenceClassifier()) was done and runs okay

Also adapter_sent:

server.add_classifier(
     "adapter_sent",
     AdapterSentenceClassifier(
         base_model_name="bert-base-uncased", adapter_name="sentiment/sst-2@ukp", labels=["negative", "positive"]
     ),
 )

reports AttributeError: 'BertForSequenceClassification' object has no attribute 'load_adapter' error

[whisere]

need to run this for adapter_sent: pip install adapter-transformers

I used this step for both classifiers layers setting: https://inception-project.github.io/releases/0.19.0/docs/user-guide.html#sect_projects_layers and add netura to labels: labels=["negative", "netura", "positive"] to AdapterSentenceClassifier above Please let me know know if that is the correct steps for the recommenders

[jcklie]

You normally configure layer and feature in INCEpTION and the recommenders pick it up. The REST API ist described here. Are you using these with INCEpTION? Did you have a look at this notebook ? Sempred is not a sentence level layer, that should not work with sentence classification. You would need to create your own. The adapter recommender are not the easiest ones, I would not start with using these.

[whisere]

Thanks for the reply. Yes I am trying to use the recommenders in INCEpTION. All the instructions I found are for spacy_ner and spacy_pos, I have successfully set them up, but there are no instructions on how to setup the others on INCEpTION.

I have tried the above "I used this step for both classifiers layers setting: https://inception-project.github.io/releases/0.19.0/docs/user-guide.html#sect_projects_layers and add netura to labels: labels=["negative", "netura", "positive"] to AdapterSentenceClassifier above"

It seems working but not sure if the steps are right. I also need the adapter recommender for auto labelling sentence sentiment to start with. Thanks.

[jcklie]

What exactly is your issue? Adapter have a fixed tagset that depends on the pretrained model, if you want another tagset then you need to train your own. You need to ask the person who trained the adapter what the tagset is if it is not documented.

[whisere]

Thanks we can use the fixed tagset, so is it right to use this settings: https://inception-project.github.io/releases/0.19.0/docs/user-guide.html#sect_projects_layers in INCEpTION for using sklearn_sentence (SklearnSentenceClassifier) and adapter_sent (AdapterSentenceClassifier) that come with this git repository with tags/labels "negative", "netura", "positive"?

[whisere]

How do we find the adapter information such as the one below:

server.add_classifier( "adapter_sent", AdapterSentenceClassifier( base_model_name="bert-base-uncased", adapter_name="sentiment/sst-2@ukp", labels=["negative", "positive"] ), )

Thanks. Is it okay to add "netura" to the labels?

[whisere]

the above is from https://github.com/inception-project/inception-external-recommender/blob/master/wsgi.py

[jcklie]

Adapters come from adapterhub, so you need to find an adapter that suits your needs. They are pretrained, so you cannot just change the tagset.

https://adapterhub.ml/explore/ https://adapterhub.ml/explore/sentiment/sst-2/

I did not train them, so you need to ask the people there. If you need a different adapter, then you would need to train your own. You do not need a tagset for the Sklearn one, as they are trained on the fly on your data and compute the tagset based on the data you send/annotate.

[whisere]

Thank you very much, I will try that.

[whisere]

Any idea why it reported: No adapter with name 'sentiment/hinglish-twitter-sentiment@nirantk' was found in the adapter index.

when using https://adapterhub.ml/adapters/nirantk/bert-base-multilingual-uncased-hinglish-sentiment/

with server.add_classifier( "adapter_sent1", AdapterSentenceClassifier( base_model_name="bert-base-multilingual-uncased", adapter_name="sentiment/hinglish-twitter-sentiment@nirantk", labels=["negative", "netural", "positive"] ), )

Thanks!

[jcklie]

I do not know, you would need to ask the adapterhub people.

[whisere]

Please see https://github.com/Adapter-Hub/Hub/issues/27

[jcklie]

You could try to add the config parameter to https://github.com/inception-project/inception-external-recommender/blob/master/ariadne/contrib/adapters.py#L169

[whisere]

Thank you I will try!

[whisere]

I have added ,config="pfeiffer" on line 169 and 120 but still have the same problem raise EnvironmentError("No adapter with name '{}' was found in the adapter index.".format(specifier)) OSError: No adapter with name 'sentiment/hinglish-twitter-sentiment@nirantk' was found in the adapter index.

[whisere]

I found I have to change 168 from bert-base-uncased to bert-base-multilingual-uncased so: def _build_model(self): model = AutoModelForSequenceClassification.from_pretrained("bert-base-multilingual-uncased") self._adapter_internal_name = model.load_adapter(self._adapter_name, "text_task",config="pfeiffer") return model

but then sentiment/sst-2@ukp can't be added at the same time: raise EnvironmentError("No adapter with name '{}' was found in the adapter index.".format(specifier)) OSError: No adapter with name 'sentiment/sst-2@ukp' was found in the adapter index.

[whisere]

with def _build_model(self): model = AutoModelForSequenceClassification.from_pretrained("bert-base-multilingual-uncased") self._adapter_internal_name = model.load_adapter(self._adapter_name, "text_task",config="pfeiffer") return model

and removed sentiment/sst-2@ukp from start script it downloaded something but it also reports another error:

raise OSError("Unable to load weights from pytorch checkpoint file. ") OSError: Unable to load weights from pytorch checkpoint file.

[jcklie]

I fixed some things and pushed a new example, can you try again?

[whisere]

Thank you! It still reported the same error: raise OSError("Unable to load weights from pytorch checkpoint file. ") OSError: Unable to load weights from pytorch checkpoint file.

I was thinking it might be torch problem so I did pip3 install --upgrade torch Then it reported: ImportError: cannot import name 'SAVE_STATE_WARNING' under torch 1.18

so I did pip install torch==1.6.0 --use-feature=2020-resolver it reported after installation: ERROR: torchvision 0.6.0 requires torch==1.5.0, but you'll have torch 1.6.0 which is incompatible.

but the recommender is started up at 5000 port without problems, I will see if there is issues with inception.

[whisere]

It seems working at inception side, I will try other adapters too. Many thanks!!

[whisere]

received this error when starting with two remote adapters but only enabling one:

inception/inception-external-recommender-master/ariadne/contrib/adapters.py", line 179, in predict

label = self._label_map[label_id]

KeyError: 4980

inception/inception-external-recommender-master/ariadne/contrib/adapters.py", line 179, in predict

label = self._label_map[label_id]

KeyError: 1901

start scripts:

server.add_classifier( "adapter_sent2", AdapterSentenceClassifier( base_model_name="bert-base-uncased", adapter_name="sentiment/sst-2@ukp", labels=["negative", "netural", "positive"],config="pfeiffer", ), )

#server.add_classifier("sbert", SbertSentenceClassifier()) server.add_classifier( "adapter_sent1", AdapterSentenceClassifier( # base_model_name="bert-base-uncased", adapter_name="sentiment/sst-2@ukp", labels=["negative", "positive"] "bert-base-multilingual-uncased", "sentiment/hinglish-twitter-sentiment@nirantk", labels=["negative", "netural", "positive"], config="pfeiffer", ), )

I later removed adapter_sent1 and left only adapter_sent2 in start script it is still the same.

[jcklie]

You need to find out what the labels and ids of the adapter are and change the part where we map numerical ids to string labels.

[whisere]

Thanks! Where should I make this change? the sentiment/sst-2@ukp remote adapter doesn't seem to work anymore in inception with the keyerror above, started with script:

server.add_classifier(
"adapter_sent2",
AdapterSentenceClassifier(
base_model_name="bert-base-uncased", adapter_name="sentiment/sst-2@ukp", labels=["negative", "neutral", "positive"],config="pfeiffer",
),
)

sentiment/hinglish-twitter-sentiment@nirantk below is working okay

server.add_classifier(
"adapter_sent1",
AdapterSentenceClassifier(
"bert-base-multilingual-uncased",
"sentiment/hinglish-twitter-sentiment@nirantk",
labels=["negative", "neutral", "positive"],
config="pfeiffer",
),
)

[jcklie]

I think it is

https://github.com/inception-project/inception-external-recommender/blob/master/ariadne/contrib/adapters.py#L159

I assumed that label ids start from 0

[whisere]

Thanks! I still don't quite get the change needs to be made but will investigate. sentiment/sst-2@ukp was working fine before the latest code change while sentiment/hinglish-twitter-sentiment@nirantk was not, but now sentiment/hinglish-twitter-sentiment@nirantk is work while sentiment/sst-2@ukp is not...