Add decorators for cross-cutting architecture information

[jpgough-ms]

CALM Schema Change Proposal

Target Schema:

Create a new decorators.json schema file for decorator definitions that can be referenced by CALM architectures.

Description of Change:

This change creates a separate decorator schema to attach supplementary information to multiple nodes and relationships simultaneously. Decorators exist outside core architecture documents to avoid bloating them while solving cross-cutting information needs that current metadata cannot address.

Decorators enable:

• Information that spans multiple components
• Typed categorization with predefined and custom types
• Free-form JSON data storage
• Centralized cross-cutting documentation

Use Cases:

1. User Guides: Reference multiple nodes and relationships to explain component interactions
2. Business Context: Attach business information to individual nodes or component groups
3. Threat Models: Document security threats across multiple components
4. Security Information: Store security data that crosses component boundaries
5. Deployment Information: Add deployment details affecting multiple components
6. Cross-cutting Documentation: Reference multiple architecture elements for supplementary information

Example Use Cases:

Guide Decorator:

{
  "unique-id": "user-onboarding-guide",
  "type": "guide",
  "applies-to": ["frontend-app", "auth-service", "user-database", "frontend-to-auth-connection"],
  "data": {
    "title": "New User Registration Flow",
    "description": "Step-by-step guide for implementing user registration",
    "documentation-url": "https://docs.company.com/guides/user-registration",
    "steps": [
      "User submits form via frontend-app",
      "Frontend calls auth-service via frontend-to-auth-connection", 
      "Auth-service validates and stores user in user-database"
    ]
  }
}

Threat Model Decorator:

{
  "unique-id": "sql-injection-threat",
  "type": "threat-model",
  "applies-to": ["api-service", "database", "api-to-db-connection"],
  "data": {
    "threat-id": "TM-001",
    "description": "SQL injection vulnerability in API to database communication",
    "severity": "high",
    "mitigation": "Use parameterized queries and input validation",
    "affected-flows": ["user-login", "data-retrieval"]
  }
}

Current Limitations:

The current CALM schema has metadata at the node and relationship level, but this doesn't adequately address cross-cutting concerns because:

1. No Cross-Reference Capability: Metadata cannot reference multiple nodes/relationships
2. Information Duplication: Related information duplicates across metadata sections
3. No Central Discovery: No single location for cross-component information
4. Inconsistent Structure: Same information types structure differently across nodes

Proposed Schema Changes:

Create a new decorators.json schema file with the following structure:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://calm.finos.org/release/1.1/meta/decorators.json",
  "title": "CALM Decorators Schema",
  "description": "Schema for CALM architecture decorators",
  "type": "object",
  "properties": {
    "decorators": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "unique-id": {
            "type": "string",
            "description": "Unique identifier for this decorator"
          },
          "type": {
            "anyOf": [
              {
                "enum": ["guide", "business", "threat-model", "security", "deployment"]
              },
              {
                "type": "string"
              }
            ],
            "description": "Type of decorator - predefined types or custom string"
          },
          "applies-to": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "minItems": 1,
            "description": "Array of unique-ids referencing nodes, relationships, flows, or other architecture elements"
          },
          "data": {
            "type": "object",
            "description": "Free-form JSON object containing the decorator's data"
          }
        },
        "required": ["unique-id", "type", "applies-to", "data"],
        "additionalProperties": false
      }
    }
  },
  "required": ["decorators"]
}

Backward Compatibility:

This change is fully backward compatible:

• Existing Documents Valid: All current CALM documents validate against the new schema
• No Breaking Changes: The decorators property is optional
• No Migration Required: Architectures adopt decorators incrementally
• Tool Compatibility: Existing tools ignore the decorators section

Validation Strategy:

Schema Validation Only:

• Decorators must follow the JSON schema structure
• applies-to references must be strings (tools handle referential integrity)
• Required properties must be present
• No business rules or constraints at schema level

Test Cases to Add:

• Valid decorator with single applies-to reference
• Valid decorator with multiple applies-to references
• Valid decorator with predefined type enum value
• Valid decorator with custom type string
• Invalid decorator missing required properties
• Invalid decorator with empty applies-to array

Example Documents: Create sample CALM architectures demonstrating each decorator type with realistic use cases.

Implementation Impact:

Existing Tools:

• CALM CLI: No impact - tools ignore decorators section
• Validators: Automatically validate decorator schema structure
• Documentation Generators: Can use decorator data in future enhancements

Documentation:

• Update CALM specification with decorators concept
• Add decorator usage examples
• Document decorator type and structure best practices

Related Schemas:

• Create new decorators.json schema file
• No changes required to existing schema files (core.json, interface.json, control.json, etc.)
• Architecture documents can optionally reference decorator files but are not required to

CALM Hub Integration:

• Future enhancement to display and search decorator information
• Potential for decorator-based architecture browsing and filtering

Version Strategy:

Target Version: 1.1.0 (minor version bump)

• Additive change with no breaking changes
• Follows semantic versioning
• Supports incremental adoption

Timeline:

• Add to current development branch
• Include in next minor release cycle
• No urgent timeline dependencies

Additional Context:

Decorators add contextual information to CALM architectures while keeping core architecture documents focused and clean. This addresses the growing size problem of real-world architecture documents while improving documentation, security analysis, and operational understanding.

The type system (enum + string) standardizes common decorator types and allows custom extensions for specific organizational needs.

External Decorator Approach:

Decorators exist in separate decorators.json files independent of architecture documents. This approach provides several benefits:

• Document Size Management: Keeps core architecture documents focused and prevents bloat
• Modularity: Decorator files are maintained independently from architecture definitions
• Reusability: Decorator collections can be shared across multiple architectures
• Optional Decoration: Architecture documents don't need to know they're being decorated
• Collaborative Editing: Different teams can maintain decorators and architectures separately

Example Decorator File (decorators.json):

{
  "$schema": "https://calm.finos.org/release/1.1/meta/decorators.json",
  "decorators": [
    {
      "unique-id": "user-onboarding-guide",
      "type": "guide",
      "applies-to": ["frontend-app", "auth-service", "user-database"],
      "data": {
        "title": "User Registration Flow",
        "steps": [...]
      }
    }
  ]
}

Optional Architecture Reference:

{
  "$schema": "https://calm.finos.org/release/1.1/meta/calm.json",
  "unique-id": "trading-system",
  "name": "Trading System Architecture",
  "nodes": [...]
}

Architectures can optionally reference decorators, but decorators can reference architectures without requiring changes to the architecture document itself.

Additional Context:

Decorators add contextual information to CALM architectures while separating core elements from supplementary data. This improves documentation, security analysis, and operational understanding.

The type system (enum + string) standardizes common decorator types and allows custom extensions for specific organizational needs.

External Decorator Storage:

Decorators can be stored in separate decorators.json files and referenced using standard JSON referencing. This provides several benefits:

• Modularity: Separate decorator files can be maintained independently from core architecture definitions
• Reusability: Decorator collections can be shared across multiple architectures
• Organization: Large decorator sets can be organized in dedicated files without cluttering architecture documents
• Collaborative Editing: Different teams can maintain decorators and architectures separately

Example Reference Pattern:

{
  "$schema": "https://calm.finos.org/release/1.1/meta/calm.json",
  "unique-id": "trading-system",
  "name": "Trading System Architecture", 
  "decorators": {
    "$ref": "./decorators.json#/decorators"
  },
  "nodes": [...]
}

Implementation Checklist:

• [ ] New decorators.json schema drafted and validated
• [ ] Example CALM documents created demonstrating each decorator type
• [ ] Schema validation tests written for decorator structure
• [ ] Documentation updated to include decorator concept and examples
• [ ] Migration guide created (minimal - just adoption examples)
• [ ] Validate no impact on existing CALM documents

[rocketstack-matt]

Big fan of this idea, but I think we need a way to get the additional information out of the core document as it's already getting very big modeling some real world systems (indeed I think we need to figure out how to break it into multiple docs).

Should we instead have decorators as external pointers and then apply the decorator schema on those? I also think, like controls, it may make sense to be able to apply decorators as multiple levels. We could either do this by reversing the direction (architecture elements point to their decorators) or by introducing a dot notation of what the decorators are referencing back in the main architecture?

[rocketstack-matt]

Update, I missed the additional context section referencing external decorators, think this is the better approach, although still a question of whether the main doc even needs to know it's being decorated.

[jpgough-ms]

We could either do this by reversing the direction (architecture elements point to their decorators) or by introducing a dot notation of what the decorators are referencing back in the main architecture?

I think with this point, I like the fact the nodes don't have to know about what decorates them. We have this challenge today where we have a bunch of information we want to decorate with (i.e. at the pattern level), but don't necessarily want the user of the architecture to see all of it. We would use this to investigate outages (for example) or it could be detail that is beyond what the user of the pattern would understand.

Update, I missed the additional context section referencing external decorators, think this is the better approach, although still a question of whether the main doc even needs to know it's being decorated.

I think we might be in async agreement. Let's create a new decorators.json

[jpgough-ms]

@rocketstack-matt I've edited the issue to incorporate your feedback, thank you 😃

[LeighFinegold]

I think its quite powerful to add decorators from perspective of applying to multiple nodes.

I think where it will become interesting is around the tooling and its ability to be extensible to usage context. For example, how do I validate that the decorator data section is valid? Do I need to validate that such decorators are not apply to certain nodes?

The other angle is around the overlap between controls and what is defined as a decorator or not. In the business context example for example, a lot of those could be considered controls. Perhaps its ok to define that twice in relation to what high level we expect vs what we end up implementing?

Originally when using CALM i had created domain definitions which I attached to domain nodes via meta. In the end, through discussion with @rocketstack-matt there was a thought that it was overkill to keep such business context as json information. I wonder if the above changes that?

[rocketstack-matt]

I think it in large part depends what folks are trying to achieve . . . let's discuss on Office Hours

[rocketstack-matt]

@pmerrison can we get your feedback on this issue, especially around how it would fit in for what you want to do with threat modeling.

[LeighFinegold]

I have thinking about this more. I think if we go back to a mode where we do use JSON and decorate anything with such data, nothing stops us providing templates for Solution Architect Documents for an LLM to then try and extract from the model and do a first draft to allow a user to then amend accordingly subsequently afterwards.

[LeighFinegold]

Another thought - shouldn't we amend this proposal such that the data attach can follow a defined schema i.e. add a data-definition-url property which documents the schema.

{
  "$schema": "https://calm.finos.org/release/1.1/meta/decorators.json",
  "decorators": [
    {
      "unique-id": "user-onboarding-guide",
      "type": "guide",
      "applies-to": ["frontend-app", "auth-service", "user-database"],
      "data-definition-url": "onboarding-guide-schema.json"
      "data": {
        "title": "User Registration Flow",
        "steps": [...]
      }
    }
  ]
}