Dimitris Papadopoulos

To answer my own question, what I did to load the finetuned checkpoint is the following: ``` from utils_nlp.models.transformers.sequence_classification import Processor, SequenceClassifier MODEL_NAME = "bert-base-cased" CACHE_DIR = '/local/path/where/base/model/will/be/downloaded/' model =...

Exactly! A pair of sentences as input, and one of three labels (contradiction, entailment, neutral) as output. An example that could benefit from this widget can be found at the...

Exact same model, same setup. The model is sharded (2 A100s) and served with 2 adapters. An initial call without adapter is processed successfully, but after 1-2 calls the server...

Thanks for the support! The latest docker image has solved CUDA errors in my case. However, I notice the following strange behaviour, regardless of the `adapter-memory-fraction ` value: When I...

@tgaddair It is using NVLink (NC A100 v4-series instance). 2 adapters, rank 32. They need to be called sequentially as part of a pipeline, therefore any latency in swapping directly...