Error when using `stopping_criteria` with `model.generate()`

Open aghyad-deeb opened this issue 9 months ago • 2 comments

Note that the issue only happens when using remote=True

Code

import os
from dotenv import load_dotenv
from nnsight import CONFIG, LanguageModel

load_dotenv(override=True)


CONFIG.API.APIKEY = os.getenv("NDIF_KEY")
model_id, remote, device = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B", True, "auto"
# model_id, remote, device = "HuggingFaceTB/SmolLM2-135M-Instruct", False, "mps"
model = LanguageModel(model_id, device_map=device, trust_remote_code=True)

from transformers import StoppingCriteria, StoppingCriteriaList

class StopOnTokens(StoppingCriteria):
    def __init__(self, stop_token_ids):
        self.stop_token_ids = stop_token_ids

    def __call__(self, input_ids, scores, **kwargs):
        for stop_id in self.stop_token_ids:
            if input_ids[0][-1] == stop_id:
                return True
        return False
stopping_criteria = StoppingCriteriaList([StopOnTokens(stop_token_ids=[128014])])

with model.generate("hi", remote=remote, stopping_criteria=stopping_criteria):
    out =  model.generator.output.save()
model.tokenizer.decode(out[0])

Output

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[39], line 19
     16 stopping_criteria = StoppingCriteriaList([StopOnTokens(stop_token_ids=[end_of_think_token])])
     18 # with model.generate("hi", remote=remote, stopping_criteria=stopping_criteria, max_new_tokens=1000):
---> 19 with model.generate("hi", remote=remote, stopping_criteria=stopping_criteria):
     20 # with model.generate("hi", remote=remote, stopping_criteria=stopping_criteria):
     21 # with model.generate("hi", remote=remote, max_new_tokens=1000):
     22 # prompt = "hi"
     23 # temp = [
     24 #     {"role": "user", "content": prompt},
     25 # ]
     26 # input = model.tokenizer.apply_chat_template(temp, tokenize=True, add_generation_prompt=True)
     27 # with model.generate(input, remote=remote, eos_token_id=end_of_think_token, max_new_tokens=1000):
     28     out =  model.generator.output.save()
     29 model.tokenizer.decode(out[0])

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/intervention/contexts/interleaving.py:96, in InterleavingTracer.__exit__(self, exc_type, exc_val, exc_tb)
     92     self.invoker.__exit__(None, None, None)
     94 self._model._envoy._reset()
---> 96 super().__exit__(exc_type, exc_val, exc_tb)

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/tracing/contexts/tracer.py:25, in Tracer.__exit__(self, exc_type, exc_val, exc_tb)
     21 from .globals import GlobalTracingContext
     23 GlobalTracingContext.try_deregister(self)
---> 25 return super().__exit__(exc_type, exc_val, exc_tb)

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/tracing/contexts/base.py:82, in Context.__exit__(self, exc_type, exc_val, exc_tb)
     78 graph = graph.stack.pop()
     80 graph.alive = False
---> 82 self.backend(graph)

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/intervention/backends/remote.py:77, in RemoteBackend.__call__(self, graph)
     72 def __call__(self, graph: Graph):
     74     if self.blocking:
     75 
     76         # Do blocking request.
---> 77         result = self.blocking_request(graph)
     79     else:
     80 
     81         # Otherwise we are getting the status / result of the existing job.
     82         result = self.non_blocking_request(graph)

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/intervention/backends/remote.py:289, in RemoteBackend.blocking_request(self, graph)
    280 sio.connect(
    281     self.ws_address,
    282     socketio_path="/ws/socket.io",
    283     transports=["websocket"],
    284     wait_timeout=10,
    285 )
    287 remote_graph = preprocess(graph)
--> 289 data, headers = self.request(remote_graph)
    291 headers["session_id"] = sio.sid
    293 # Submit request via

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/intervention/backends/remote.py:60, in RemoteBackend.request(self, graph)
     58 def request(self, graph: Graph) -> Tuple[bytes, Dict[str, str]]:
---> 60     data = RequestModel.serialize(graph, self.format, self.zlib)
     62     headers = {
     63         "model_key": self.model_key,
     64         "format": self.format,
   (...)     67         "sent-timestamp": str(time.time()),
     68     }
     70     return data, headers

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/schema/request.py:43, in RequestModel.serialize(graph, format, _zlib)
     38 @staticmethod
     39 def serialize(graph: Graph, format:str, _zlib:bool) -> bytes:
     41     if format == "json":
---> 43         data = RequestModel(graph=graph)
     45         json = data.model_dump(mode="json")
     47         data = msgspec.json.encode(json)

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/nnsight/schema/request.py:30, in RequestModel.__init__(self, memo, *args, **kwargs)
     28 def __init__(self, *args, memo: Dict = None, **kwargs):
---> 30     super().__init__(*args, memo=memo or dict(), **kwargs)
     32     if memo is None:
     34         self.memo = {**MEMO}

File ~/Documents/coding/reasoning_or_performing/venv/lib/python3.12/site-packages/pydantic/main.py:214, in BaseModel.__init__(self, **data)
    212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215 if self is not validated_self:
    216     warnings.warn(
    217         'A custom validator is returning a value other than `self`.\n'
    218         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
    219         'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
    220         stacklevel=2,
    221     )

ValidationError: 41 validation errors for RequestModel
graph.GraphModel
  Input should be a valid dictionary or instance of GraphModel [type=model_type, input_value=<nnsight.tracing.graph.gr...h object at 0x157db3c80>, input_type=Graph]
    For further information visit https://errors.pydantic.dev/2.10/v/model_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.MemoReferenceModel
  Input should be a valid dictionary or instance of MemoReferenceModel [type=model_type, input_value=&lt;InterventionNode at 0x157db2a50&gt;, input_type=InterventionNode]
    For further information visit https://errors.pydantic.dev/2.10/v/model_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.int
  Input should be a valid integer [type=int_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/int_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.float
  Input should be a valid number [type=float_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/float_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.str
  Input should be a valid string [type=string_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/string_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.bool
  Input should be a valid boolean [type=bool_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/bool_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`tagged-union[MemoReferenceModel,...,SliceModel,TensorModel,TupleModel,ListModel,DictModel,FunctionModel,EllipsisModel,InterventionGraphModel,SubGraphModel,GraphModel]`
  Unable to extract tag using discriminator 'type_name' [type=union_tag_not_found, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/union_tag_not_found
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-before[check_memo(), any]
  Value error,  [type=value_error, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/value_error
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[Node]]
  Input should be an instance of Node [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[InterventionGraph]]
  Input should be an instance of InterventionGraph [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[SubGraph]]
  Input should be an instance of SubGraph [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[Graph]]
  Input should be an instance of Graph [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[builtin_function_or_method]
  Input should be an instance of builtin_function_or_method [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[function]
  Input should be an instance of function [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[method_descriptor]
  Input should be an instance of method_descriptor [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].custom-error[is-instance[type]]
  Input should be a type [type=is_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[slice]]
  Input should be an instance of slice [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[Tensor]]
  Input should be an instance of Tensor [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.`function-after[to_model(), tuple[any, ...]]`
  Input should be a valid tuple [type=tuple_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/tuple_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.int
  Input should be a valid integer [type=int_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/int_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.float
  Input should be a valid number [type=float_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/float_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.str
  Input should be a valid string [type=string_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/string_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.bool
  Input should be a valid boolean [type=bool_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/bool_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`tagged-union[MemoReferenceModel,...,...,TensorModel,...,...,DictModel,FunctionModel,EllipsisModel,InterventionGraphModel,SubGraphModel,GraphModel]`
  Unable to extract tag using discriminator 'type_name' [type=union_tag_not_found, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/union_tag_not_found
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-before[check_memo(), any]
  Value error,  [type=value_error, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/value_error
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[Node]]
  Input should be an instance of Node [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[InterventionGraph]]
  Input should be an instance of InterventionGraph [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[SubGraph]]
  Input should be an instance of SubGraph [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), is-instance[Graph]]
  Input should be an instance of Graph [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[builtin_function_or_method]
  Input should be an instance of builtin_function_or_method [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[function]
  Input should be an instance of function [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].is-instance[method_descriptor]
  Input should be an instance of method_descriptor [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]].custom-error[is-instance[type]]
  Input should be a type [type=is_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[slice]]
  Input should be an instance of slice [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[inner(), is-instance[Tensor]]
  Input should be an instance of Tensor [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.`function-after[to_model(), tuple[any, ...]]`
  Input should be a valid tuple [type=tuple_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/tuple_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]]
  Input should be a valid list [type=list_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/list_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), dict[any,any]]
  Input should be a valid dictionary [type=dict_type, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/dict_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), list[any]].values.0.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[<lambda>(), is-instance[ellipsis]]
  Input should be an instance of ellipsis [type=is_instance_of, input_value=<__main__.StopOnTokens object at 0x157dae900>, input_type=StopOnTokens]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[to_model(), dict[any,any]]
  Input should be a valid dictionary [type=dict_type, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/dict_type
graph.function-after[to_model(), is-instance[Graph]].nodes.2.function-after[inner(), is-instance[Node]].kwargs.stopping_criteria.`union[function-before[check_memo(), any],function-after[inner(), is-instance[Node]],function-after[to_model(), is-instance[InterventionGraph]],function-after[to_model(), is-instance[SubGraph]],function-after[to_model(), is-instance[Graph]],function-after[to_model(), union[is-instance[builtin_function_or_method],is-instance[function],is-instance[method_descriptor],custom-error[is-instance[type]]]],function-after[inner(), is-instance[slice]],function-after[inner(), is-instance[Tensor]],function-after[to_model(), tuple[any, ...]],function-after[to_model(), list[any]],function-after[to_model(), dict[any,any]],function-after[<lambda>(), is-instance[ellipsis]]]`.function-after[<lambda>(), is-instance[ellipsis]]
  Input should be an instance of ellipsis [type=is_instance_of, input_value=[<__main__.StopOnTokens object at 0x157dae900>], input_type=StoppingCriteriaList]
    For further information visit https://errors.pydantic.dev/2.10/v/is_instance_of

I'm currently using the eos_token_id argument as a workaround but wanted to highlight this. Thanks!

Mar 14 '25 15:03 aghyad-deeb

Hi, StoppingCriteria and StoppingCriteriaList are currently not supported when using the remote backend. A similar ticket was raised last year, and I don't think it was decided that we would support it (#137). We have a "whitelist" which is basically just a list of supported types/functions which can be used on our remote server, the error message you got came from stopping_criteria failing validation.

That said, the error message you got is very uninformative, so I raised a ticket to address this: #388.

Mar 27 '25 14:03 MichaelRipa

@MichaelRipa Thanks for your reply! In case this is helpful, here's why I think it's particularly useful to have support for StoppingCriteria. When having the remote=True option, there seems to be some data transfer from local to the remote that takes non-trivial time both at the start of the request and at the end. As I noticed this, I started looking into ways to minimize the number of these transitions (e.g., using model.generate() over manually sampling in the code). I think StoppingCriteria can be helpful for this purpose as you can tell the model exactly when to stop generating and hopefully eliminate some unnecessary overhead from connecting to the remote again. The example I have in particular is getting one token after the end of reasoning token for a model. I ended up doing this by passing the end of thinking token as the eos_token_id argument, but I thought StoppingCriteria can be helpful in other cases.

Mar 27 '25 19:03 aghyad-deeb

Closing this as #137 already ask for this feature

Oct 15 '25 19:10 Butanium