sdk-python icon indicating copy to clipboard operation
sdk-python copied to clipboard

[FEATURE] Context Limit Property on Model Interface

Open westonbrown opened this issue 1 month ago • 0 comments

Problem Statement

The Model abstract class does not expose the model's context window limit. Applications must hard-code context limits or maintain separate model-to-limit mappings.

The SDK handles ContextWindowOverflowException reactively but provides no way to query capacity beforehand. This makes it impossible to implement threshold-based management like "compress at 70% of context window" (see #555).

Proposed Solution

Add a context_limit property to the Model interface:

class Model(ABC):
    @property
    @abstractmethod
    def context_limit(self) -> int:
        """Maximum context window size in tokens."""
        pass

Model providers implement with known values:

class BedrockModel(Model):
    MODEL_CONTEXT_LIMITS = {
        "anthropic.claude-sonnet-4-20250514-v1:0": 200000,
        "amazon.nova-pro-v1:0": 300000,
    }
    
    @property
    def context_limit(self) -> int:
        return self.MODEL_CONTEXT_LIMITS.get(self.model_id, 128000)

Use Case

  1. Dynamic threshold calculation - threshold = model.context_limit * 0.7
  2. UI progress indicators - display context usage percentage
  3. Model selection - choose models based on context requirements
  4. ConversationManager configuration - auto-tune window sizes
usage_pct = current_tokens / agent.model.context_limit
if usage_pct > 0.7:
    agent.conversation_manager.reduce_context(agent)

Alternative Solutions

Applications can maintain their own model-to-limit mappings, but this is error-prone and does not stay updated with new model releases.

Additional Context

This feature, combined with a token estimation API (#1294), enables implementation of #555 (Proactive Context Compression). Related: #637 (max tokens handling).

westonbrown avatar Dec 05 '25 18:12 westonbrown