Model Context Protocol (MCP) and commands for getting component metadata

[JuroOravec]

This is bit of a publicity stunt. Right now people are going about Model Context Protocol (MCP), which is kind of like a standardized interface for adding tools to LLMs (e.g. accessing data from a database).

We could have something similar to help people using LLMs to write and design our components.

Some functionalities could be:

• Commands to:
  • Get all components
    • AKA components list command
  • Get all component trees
    • Search Python code for all calls of MyComponent.render() or MyComponent.render_to_response(), to find all root components.
  • Display component trees
    • Given a component, would search its template for all child components, displaying it all as a tree
  • Get parent components
    • Given a component, find all "parent" components, so all components that have this component in their template
  • Get component's info, like the template, js, css, typing (if given), defaults (if given), views (if given), etc

NOTE: IMO this would make most sense to develop in two steps - First the logic that builds up the graph of which components are parents / children of whose, and a way to traverse that graph. Second step would then be to define commands that interact with that graph.

MCP's quickstart says that it can be used to define promtps. So we could use this to prime the LLM to write high-quality components.

---

Personally, it's questionable how much value would this add, but if we could say that we're the first frontend framework (or at least in Python) that has an MCP integration, that could be catch some eyeballs.

[EmilStenstrom]

Agreed that the long term value of this is questionable, but still a fun idea! :) Can it be done as a plugin?

[Benoss]

I think adding a llms.txt https://llmstxt.org to start with will add a lot of value before doing a custom MCP server. This will also allow the new LLM models to be better at writing Django Componants There are already a few MCP servers that can read llms.txt standard so we can get instant value as well.

Second step would maybe be just a way to generate a MD file from an existing Django project via a manage command. Python manage.py components list-md this can be saved locally in the repo and used by existing LLM without having to write a full MCP server

Last step would be a custom MCP server that can serve documentation, list existing components, generate new components boilerplate.

[JuroOravec]

@Benoss Sounds really intereting! Do you have a setup where the LLM makes use of the llms.txt? Or could you recommend a video that showcases it? Also love the specific steps you outlined, makes it feel more palpable.

I had a quick look at the llms.txt docs. It feels like it would be a similar level of complexity like implementing the benchmarking for django-components.

However, IMO the technical part of this is the easy part. As I'm writing this out I'm struggling to grasp how to think about this. Comparing it with the benchmarking, there I knew what I wanted to achieve - some way to quantify and visualise the time to render. It's harder to imagine what we would expect from this.

The use case I imagine is that I am writing some code, and maybe I want to make some changes to the HTML to add / change a component. And I want LLM to be able help me with that, so that at minimum I get faster autocompletion, and in the better case the LLM agent can make these changes by themselves.

The question then for me is - How can we structure the 1. llms.txt, 2. llms-full.txt, or 3. the components' markdown reference from the list-md command, such that an LLM model is better at outputting the correct syntax when using those files.

In other words, how can we test that what we write as additional context / instructions for LLMs will actually help the model to write better suggestions?

Basically my question is - How to test prompt engineering?

I asked Gemini 2.5 Pro and got the suggestion to add tests where we would give models tasks such as:

• Simple: "Generate a ButtonComponent with the label 'Submit'."
• With Options: "Create a CardComponent with the title 'User Info' and add a disabled submit ButtonComponent labeled 'Save' in its footer slot."
• Modification: "Take this code snippet <...> and change the AlertComponent's level parameter to 'error'."
• More Complex: "Create a Django template snippet that uses a UserListComponent to display a list of users, where each user is shown using a UserProfileCard component." (This requires the LLM to understand nesting).

I also asked to outline how such tests could look like. Click to expand

# test_component_generation.py

import pytest
import pathlib
from typing import Optional

# --- Hypothetical LLM Client ---
# Replace this with your actual LLM API call implementation
def call_llm(user_prompt: str, system_prompt: Optional[str] = None) -> str:
    """
    Placeholder for calling the LLM API.

    Args:
        user_prompt: The main instruction for the LLM.
        system_prompt: Optional context or instructions for the LLM's behavior.

    Returns:
        The raw text response from the LLM.
    """
    print(f"\n--- Calling LLM ---")
    if system_prompt:
        print(f"System Prompt: {system_prompt[:100]}...") # Print beginning
    print(f"User Prompt: {user_prompt}")

    # In a real scenario, you would use libraries like:
    # - openai.ChatCompletion.create(...)
    # - anthropic.Anthropic().messages.create(...)
    # - google.generativeai.GenerativeModel(...).generate_content(...)
    # - langchain chains or agents

    # Simulate LLM response based on whether context was provided
    if system_prompt and "ButtonComponent" in system_prompt:
        # Simulate a better response when context is available
        response = '{% load component_tags %}\n{% component "ButtonComponent" label="Submit" %}'
        print(f"Simulated LLM Response (with context): {response}")
    else:
        # Simulate a potentially naive or incorrect response without context
        response = '<button type="submit">Submit</button> '
        print(f"Simulated LLM Response (without context): {response}")
    print(f"--- LLM Call End ---")
    return response

# --- User's Validation Logic ---
# Replace this with your actual rendering check
def check_output_renders_correctly(llm_output: str) -> bool:
    """
    Placeholder for the user's logic to validate the LLM output.
    This function should attempt to render the snippet using the
    Django template engine and the component framework, checking
    if it produces valid output without errors.

    Args:
        llm_output: The code snippet generated by the LLM.

    Returns:
        True if the output is valid and renders correctly, False otherwise.
    """
    print(f"Checking if output renders correctly: {llm_output}")
    # --- Your Validation Logic Goes Here ---
    # Example steps:
    # 1. Create a minimal Django template string containing llm_output.
    # 2. Load necessary component tags.
    # 3. Get the template engine.
    # 4. Try to render the template with an empty context.
    # 5. Catch template syntax errors, component registration errors, etc.
    # 6. (Optional) Check if the rendered HTML contains expected elements.
    # For this example, we'll just do a basic check:
    if '{% component "ButtonComponent"' in llm_output and 'label="Submit"' in llm_output:
        print("Output seems syntactically plausible.")
        return True
    else:
        print("Output does NOT look like the expected component tag.")
        return False
    # --- End of Validation Logic ---


# --- Pytest Fixtures ---

@pytest.fixture(scope="module")
def component_context() -> str:
    """Loads the component reference Markdown file."""
    context_file = pathlib.Path("components_reference.md")
    if not context_file.exists():
        # Create a dummy file if it doesn't exist for the example to run
        print(f"Warning: {context_file} not found. Creating dummy file.")
        dummy_content = """
        # Component Reference

        ## ButtonComponent
        - **Description:** Renders a button element.
        - **Parameters:**
            - `label` (string, required): The text displayed.
            - `type` (string, optional, default: "button"): button|submit|reset.
        - **Slots:** None
        - **Usage:**
            ```html+django
            {% component "ButtonComponent" label="Click Me" %}
            ```
        """
        context_file.write_text(dummy_content, encoding="utf-8")
        # pytest.skip(f"Component reference file not found: {context_file}")

    try:
        return context_file.read_text(encoding="utf-8")
    except Exception as e:
        pytest.fail(f"Failed to load component context: {e}")


@pytest.fixture
def base_prompt() -> str:
    """The specific task prompt for the LLM."""
    return 'Generate a Django template snippet for a ButtonComponent with the label "Submit".'


# --- Pytest Test Functions ---

def test_llm_without_context(base_prompt):
    """
    Tests the LLM's response *without* providing the component reference context.
    We expect this might be less accurate.
    """
    print("\n--- Running Test: Without Context ---")
    llm_output = call_llm(user_prompt=base_prompt, system_prompt=None)

    assert isinstance(llm_output, str), "LLM did not return a string."
    assert llm_output.strip(), "LLM returned an empty string."

    # User's validation check
    is_correct = check_output_renders_correctly(llm_output)
    # We don't necessarily assert True here, as the point is to *compare*
    # results with/without context. You might assert False if you expect
    # failure without context, or just log the result.
    print(f"Validation Result (Without Context): {is_correct}")
    # assert is_correct # Optional: Assert only if you expect success/failure


def test_llm_with_context(base_prompt, component_context):
    """
    Tests the LLM's response *with* the component reference context
    provided via the system prompt. We hope this is more accurate.
    """
    print("\n--- Running Test: With Context ---")
    system_prompt = f"""
You are an expert Django developer specializing in the django-components framework.
Use the following component reference to generate accurate template snippets.

--- COMPONENT REFERENCE START ---
{component_context}
--- COMPONENT REFERENCE END ---

Generate only the Django template code snippet as requested by the user.
"""
    llm_output = call_llm(user_prompt=base_prompt, system_prompt=system_prompt)

    assert isinstance(llm_output, str), "LLM did not return a string."
    assert llm_output.strip(), "LLM returned an empty string."

    # User's validation check
    is_correct = check_output_renders_correctly(llm_output)
    # Here, we would typically assert True, as we expect the context
    # to enable the LLM to succeed.
    print(f"Validation Result (With Context): {is_correct}")
    assert is_correct, "LLM output failed validation even with context."

# To run this test:
# 1. Make sure you have pytest installed (`pip install pytest`).
# 2. Save this code as a Python file (e.g., `test_component_generation.py`).
# 3. Create a dummy `components_reference.md` file in the same directory or let the fixture create it.
# 4. Run `pytest -s` in your terminal in that directory (`-s` shows print statements).

Here's the full convo: https://g.co/gemini/share/fdc91e88c0e0

---

So with all of this, the support for llms.txt and similar could be IMO broken down into 3 parts:

1. Technical part - e.g. implementing llms.txt, adding pure markdown alternatives for docs pages, the list-md command, etc..
2. A setup for testing whether LLMs can generate correct outputs with the info we provide.
3. Defining the "user stories" we want to cover so we can test for them.

Ideas for some users stories could be:

1. Edit component in HTML to add / modify / remove a prop / slot
2. Edit component in py to add / modify / remove a prop / slot
3. Given a parent component that renders some text and two buttons, refactor the buttons into a standalone component (and insert it into parent's template).
4. Write a component to do X (e.g. a form with 2 fields).
5. JS / CSS variables.

[Benoss]

Here is a good MCP server explaining a bit more the use case. https://github.com/langchain-ai/mcpdoc

You are right ideally your ai assistant: create me a button using Django components. But the power comes more when you use a coding assistant like cline or too code where the LLM does the plan and break down all the UI parts then with the power of MCP would generate the snippets and use them to build an entire feature by editing multiple files in the codebase. While I think we are not there yet in terms of autonomous coding, seeing how much progress was done in the last 6 months, the libraries that are AI friendly will probably be used more and more