[Question]: What to do if the model isn't following the pattern prompts?

[mpstaton]

What is your question?

First, @danielmiessler thanks so much for making fabric. I've been trying hard for the last six months to really integrate AI into my work and fabric shows the most promise by far. I spent 15 years as a professional VC, so I mean it when I say you are "really onto something."

After reading the documentation and playing for days, I'm still not able to get DeepSeek R1 to follow the user-generated patterns. I'm not sure if this is a user error (me), a bug, or DeepSeek R1 just doesn't follow instructions. It seems to be "winging it" no matter what the pattern is.

I've read through all the commands of fabric, but I don't see one to 'assimilate' new patterns, so I'm assuming as long as I create a folder in the .config/fabric/patterns dir it should be picking up on it.

Forgive all the content below, but I want you to have all the context.

Here's my dir tree. ./config --fabric ----patterns ------extract_lossless_wisdom --------system.md --------user.md --------[template].md --------README.md

Here's the main prompt, but I've also given "user context" and even created a README and a template file trying to get it to structure output like the following:

---

My system.md file:

IDENTITY and PURPOSE

You are a wisdom, lessons, and how-to extraction service for a technology adoption advisory service. Your organization works with companies that are looking to adopt and put into practice state of the art technology. Your clients aim to make wise, prudent, but bold decisions to adopt new technology in pursuit of faster innovation speeds, better collaboration, more insightful product design, more efficient and up-to-date software engineering practices, and radically increase product adoption, customer satisfaction and organizational productivity.

Your clients are not only interested in wisdom related to the role of technology in the for modern companies, but also how to gain a competitive advantage by choosing the "best technology stack" and putting technology tools to work with best practices.

When learning of any new tool, programming language, cloud service, app, enterprise SaaS, web framework, code library, AI model, AI wrapper, or anything related, you must be able to reason about and explain the following:

TLDR

What is this "thing" or "things" -- the subject, or the subjects of the video -- this product or service, this trend or practice, this or app or library... What is it exactly? How to most succintly and accurately describe it in a way that older, less-tech savvy people can understand?

Why should your clients care? How can it help employees do their jobs better? Which employees would benefit? Or who would it benefit, and how will it benefit their company?

A quick, one paragraph summary of the video you are processing for this task. Just to relate

Deeper Context

What is the broader context of this "thing" -- how long has it been around? When was this company founded? What "category" is it in? What market does it serve? Who is their ideal customer? Who needs it and who wants it? What kind of value propositions does their marketing make? How do the YouTube creators making the video feel about it?

For their ideal customer, what are their alternatives? Who are their competitors? It is helpful to detail a pretty exhaustive list of comparable, analogous, and competitive services. Of these comparables, which ones are "market leaders," and which ones are challengers? Which ones are premium, which ones are cheaper, which ones are free and or open source?

Why would your client seriously consider this "thing" being discussed in the video over the comparables you have listed? What other comparables should be considered and why? What are "other companies" or "other people" adopting and why?

Take a step back and think step by step about how to achieve the best result possible as defined in the steps below. You have a lot of freedom to make this work well.

OUTPUT SECTIONS

1. You set up a frontmatter section in the top of the markdown. In it, you place the complete youtube url, the compete title of the video, the channel, and the timestamp of when you reviewed the video. Use the following naming conventions:

2. this_youtube_video_url:

3. video_title:

4. video_channel:

5. timestamp:,

6. First create a level 5 header (h5, or #####) of "Mentioned in this Video," under the header you will just list keywords, with each word capitalized, and each keyword (or two or three words) surrounded by square brackets like '[[' ']]'. For example, if Marketing Automations and Omnichannel Marketing are discussed you would type [[Marketing Automations]], [[Omnichannel Marketing]]. (The square brackets are an extended markdown for connected, networked documents.) Start a new section underneath with "---"

7. You create a level one header (#) and insert the title of this video. Underneath, provide a summary of the content around 50, including who is presenting and the content being discussed. Use a link to the channel in markdown if you can.

8. You extract any "competitive analysis" or "consumer or customer user opinion" about the "things" that are mentioned. Detail any commentary that evangelizes or dismisses, likes or dislikes, or compares one thing to another thing.

9. You extract the most insightful and interesting overall recommendations that can be collected from the content into a section called "### Recommendations:" Start a new section underneath with "---"

10. You extract any step-by-step instructions in this video. Detail them step by step in a section called "### How-To:""

11. You extract all mentions of media, papers, blogs, writing, thought leaders, other youtubers, and other sources of inspiration mentioned by the speakers into a section called "### References:". This should include any and all references to something that the speaker mentioned.

12. You extract the 15-30 most insightful and interesting quotes from the input into a section called "Quotes:". Use the exact quote text from the input, and if you can tell which speaker (if there are multiple speakers) specify which speaker. Put it in bullet fomat and surround it by quotes. An example: "To be or not to be. That is the question" -- William Shakespeare

OUTPUT INSTRUCTIONS

1. Put the frontmatter in the top of the markdown.
2. You only output Markdown.
3. Use the template discussed above. For clarity, here it is again:

---
this_youtube_video_url: ${value}
video_title: ${value}
video_channel: ${value}
timestamp: ${value}
---
##### Mentioned in this Video
[[Topic 1]], [[Product 1]], [[Product 2]], [[Topic 2]]

---

# {video_title}
Summary around 50 words.

Competitive analysis. Comparables. Audience or users.

### Recommendations
Recommendations

### How-To
Step by step instructions

---

### References 
References

### Quotes
Quotes

1. Do not write as if you are writing a personal letter. This is a professional memo from our technology advisory organization.
2. Switch between prose that has a voice that is authoritative and technical, and a voice that has a simple way of explaining complex things.
3. Do not give warnings or notes.
4. You use numbered lists, not bullets.
5. Do not repeat yourself. We want to be detailed but we don't want to waste anyone's attention.
6. Do not start items with the same opening words.

---

---

The DeepSeek R1 output to your extract_wisdom prompt:

---

this_video_url: https://youtube.com/watch?v=z1Eq0xlVs3g this_video_id: z1Eq0xlVs3g

Alright, let me try to figure out how to approach this. The user has given a detailed outline of an episode they're planning on hosting, with placeholders for titles and other details. My task is to act as an AI assistant and help them fill in these gaps.

First, I need to understand the structure of their existing episode. It's divided into segments: Introduction, Development, Challenges, Testing, Q&A, and Conclusion. Each segment has specific points that need to be filled out with content from the host, like Shing or TJ.

The user provided an initial response where they took over as host, but the placeholders were just "[Name]". My job is to help them elaborate on each part of the episode based on their outline and the knowledge I have. I should make sure that each segment is detailed enough to provide value while keeping it engaging for the listeners.

I also need to consider what might be missing in their outline. Maybe they didn't include specific details about certain topics, or perhaps some points were cut off or not fully covered. By providing more information, I can make the episode more comprehensive and informative.

Another thing is to think about the audience. Since this is a developer show focused on browsers, it's likely aimed at tech enthusiasts who are familiar with JavaScript but new to CSS or HTML. So, I should tailor the content to be helpful for them without being too technical.

I also need to consider how each segment flows into the next. For example, in the Development segment, after explaining how to write a unit test, discussing testing best practices makes sense. Similarly, in Challenges, addressing common issues and solutions can keep the episode dynamic.

I should make sure that each section is concise but informative, avoiding unnecessary jargon unless it's explained clearly. Also, I need to think about potential pitfalls that listeners might encounter, like how to write clean CSS or avoid common JavaScript errors.

Overall, my goal is to transform their outline into a lively, engaging episode by adding relevant content and addressing key issues in the development process of web development with JavaScript and HTML.

Certainly! Here's a detailed version of your episode outline, filled in with appropriate content based on your initial thinking:

---

[Host Name]

Introduction

• Welcome to [Your Show Name], where we discuss everything related to web development with JavaScript. Join us as we explore the tools, techniques, and best practices that make your back-end code shine.
• Q&A: How long is this episode? The YouTube retention analysis shows it's been 100 million views since its launch on June 1st.

---

Development [Host Name]: Today, we’ll dive into how to write clean JavaScript and CSS that’s both functional and maintainable. From best practices to optimizing your code, let’s tackle this together!

[Host Name]: Let’s start with something simple—writing a unit test for your new functionality. This will help ensure your code works as expected.

[Host Name]: So, you have your function, and you want to write a test for it. How do you go about that? Well, the best approach is to create a small test case with known inputs and outputs. Start with something basic like:

function greeting() {
    return "Hello, world!";
}

<!DOCTYPE html>
<html>
<head>
<script>
</script>
<body>
<div class="container">
    <h1>Greeting</h1>
    <p>Hello, world!</p>
</div>
</body>
</html>

[Host Name]: Once you’ve written your test case, open it in a browser and run it. If there are any errors, that’ll let me know what’s wrong with your code.

[Host Name]: Now, for the coding part—writing clean JavaScript and CSS. I’ll break this down into sections to keep things manageable.

---

Challenges [Host Name]: Writing clean JavaScript is one thing, but making it efficient isn’t another. How can I ensure my code doesn’t get too big or slow?

[Host Name]: One common issue is variable scoping. If a variable isn’t defined where needed, your code might throw an error. I’ll explain how to use let and const to keep things clear.

[Host Name]: Another challenge is avoiding infinite loops. How do you detect when something isn’t working as expected?

[Host Name]: To address this, I can wrap the test in a loop that runs until it either fails or meets its criteria. Something like:

function test() {
    let count = 0;
    while (true) {
        // Check if input is provided
        const input = prompt('Enter your message');
        if (!input) break;

        // Perform the action
        setTimeout(() => {
            alert('Test passed!');
            count++;
        }, 1000);
        
        if (count > 5) {
            throw new Error('Test failed: Failed after 6 attempts.');
        }
    }
}

// Run the loop multiple times
for (let i = 0; i < 3; i++) {
    test();
}

[Host Name]: But what if I’m not sure how to structure my code? Maybe using a template or library could help. Oh, right! The MDN Web Docs have some useful tips on organizing JavaScript in projects.

---

Testing [Host Name]: Testing is the backbone of any project, but it can also be error-prone if done poorly. How do I write tests that cover all edge cases?

[Host Name]: One effective approach is to use tools like [Name]'s Test runner or [Name]'s覆盖率 plugin. These tools help you identify where your code might fail.

[Host Name]: Let’s say I’m stuck on a particular test case. How do I handle that situation? Well, I can reach out for help or discuss it in real-time with my team.

---

Conclusion [Host Name]: Writing clean JavaScript and CSS is crucial for making your code maintainable and efficient. By understanding best practices, avoiding pitfalls, and using tools to aid me along the way, I’ll ensure that my projects are not only functional but also future-proof.

[Host Name]: Thank you for joining us on [Your Show Name]. We look forward to seeing what you create next!

---

This version of your episode is more detailed and engaging, providing valuable content while keeping it informative. Let me know if you’d like further refinements!

---

---

The DeepSeek R1 Ouput to my extract_lossless_wisdom pattern

---

this_video_url: https://youtube.com/watch?v=z1Eq0xlVs3g this_video_id: z1Eq0xlVs3g

Alright, I'm trying to figure out how to approach this. The user has provided a detailed episode transcript about TypeScript and web development, and they've asked me to act as an AI helper. They want me to outline specific questions that could be raised based on this episode.

First, I need to understand what the episode covers. It's about TypeScript for JavaScript development, its features like static type checking, performance improvements, async/await support, and how it enhances accessibility with Web Workers. The host also mentions integrating with frameworks like React, Angular, Vue, etc.

Now, thinking about potential questions that could come up after this episode:

1. Static Type Checking and Performance: What are the main benefits of TypeScript in terms of performance? How does static typing help catch bugs before runtime?

2. New Features Like async/await: What makes JavaScript support for async/await a big deal today? Are there any limitations or challenges with these features?

3. Integration with Web Workers: How does TypeScript enable the use of Web Workers in modern projects? Does it leverage React's capabilities, and how does that affect performance?

4. Enhanced Accessibility: Beyond Adobe Flash, what other accessibility features are being introduced with TypeScript? Are there any new ways to improve screen reader support or keyboard navigation?

5. Real-World Applications: Are there actual projects where TypeScript has made a significant impact in terms of development speed and code quality? Maybe something related to web frameworks or large-scale applications.

6. Learning Curve for NewDevelopers: How does the learning curve for TypeScript compare to JavaScript? Are there any tips or resources that could make it easier for someone new to TypeScript?

7. Community and Tools: What's the current state of TypeScript community support? Are there specific tools, frameworks, or libraries developers are leveraging in their projects?

8. Future Updates and Changes: How is TypeScript evolving? Are there plans for future features or updates that could further enhance accessibility or performance?

9. Integration with Other Technologies: Is TypeScript being used alongside other modern technologies like React and Vue.js? If so, how does that integration impact the overall development experience?

10. Accessibility and Special Populations: Besides Adobe Flash, are there other major browser extensions or features being introduced to improve accessibility for different user groups, such as individuals with visual impairments or those using assistive technologies?

I think these questions cover both the technical aspects of TypeScript and its impact on accessibility in web development. They also touch upon how it's being integrated into modern frameworks and its future developments.

Questions Raised About TypeScript and Web Development

1. Static Type Checking and Performance

   • What are the main benefits of TypeScript in terms of performance? How does static typing help catch bugs before runtime?

2. New Features Like async/await

   • What makes JavaScript support for async/await a big deal today? Are there any limitations or challenges with these features?

3. Integration with Web Workers

   • How does TypeScript enable the use of Web Workers in modern projects? Does it leverage React's capabilities, and how does that affect performance?

4. Enhanced Accessibility

   • Beyond Adobe Flash, what other accessibility features are being introduced with TypeScript? Are there any new ways to improve screen reader support or keyboard navigation?

5. Real-World Applications

   • Are there actual projects where TypeScript has made a significant impact in terms of development speed and code quality? Maybe something related to web frameworks or large-scale applications.

6. Learning Curve for NewDevelopers

   • How does the learning curve for TypeScript compare to JavaScript? Are there any tips or resources that could make it easier for someone new to TypeScript?

7. Community and Tools

   • What's the current state of TypeScript community support? Are there specific tools, frameworks, or libraries developers are leveraging in their projects?

8. Future Updates and Changes

   • How is TypeScript evolving? Are there plans for future features or updates that could further enhance accessibility or performance?

9. Integration with Other Technologies

   • Is TypeScript being used alongside other modern technologies like React and Vue.js? If so, how does that integration impact the overall development experience?

10. Accessibility and Special Populations

    • Besides Adobe Flash, are there other major browser extensions or features being introduced to improve accessibility for different user groups, such as individuals with visual impairments or those using assistive technologies?

These questions provide a comprehensive overview of TypeScript's current state, future potential, and its impact on accessibility in web development.

[mpstaton]

@danielmiessler I think I may have found the issue. I'm not sure it's a bug, but possibly an oversight.

The .env file that comes with fabric has the following values:

PATTERNS_LOADER_GIT_REPO_URL=https://github.com/danielmiessler/fabric.git PATTERNS_LOADER_GIT_REPO_PATTERNS_FOLDER=patterns

However, it doesn't have any environment variables for the local pattern directory.

I haven't gone through your source code, and I have no experience with Go so I am not sure if I could even help or verify my hunch.

But, my hunch is that I keep getting the same output because fabric is defaulting to hitting your GitHub patterns repo. That would explain how no matter what I do to change the patterns in the config/fabric/patterns dir, I keep getting the same kind of result from fabric.

Is there a setting somewhere that's just not in the docs? Or, maybe I should commit my own patterns and then point the url there? (actually, fuck that's a good idea). Except, now that I am thinking of it, I created a pattern folder with my own pattern name, extract_lossless_wisdom and it ran as if it knew what that meant. So, maybe this isn't the issue?

No idea. Would appreciate if you could chime in.

Best,

Michael

[mmsrubar]

Are you running the DeepSeek R1 model with ollama or some other local system? I recently struggled with a very similar issue and found that ollama uses a default context window size of about 2048 tokens so that my system prompt never even gets to the model. Using fabric command line option --modelContextLength= helped.

[ksylvan]

@mpstaton you wrote:

PATTERNS_LOADER_GIT_REPO_URL=https://github.com/danielmiessler/fabric.git PATTERNS_LOADER_GIT_REPO_PATTERNS_FOLDER=patterns

However, it doesn't have any environment variables for the local pattern directory.

The .env fragments there mostly tell the fabric setup process where to get the patterns from. Those patterns are then installed on your local machine.

Fabric, running on your machine, will only look in the $HOME/.config/fabric/patterns/ directory for patterns. It does not do any lookups based on the github URL.

So if you want to add a custom pattern, you can add a subdirectory there, with your custom patterns.

See, for example: https://github.com/ksylvan/custom-fabric

[mpstaton]

@mpstaton you wrote:

PATTERNS_LOADER_GIT_REPO_URL=https://github.com/danielmiessler/fabric.git PATTERNS_LOADER_GIT_REPO_PATTERNS_FOLDER=patterns However, it doesn't have any environment variables for the local pattern directory.

The .env fragments there mostly tell the fabric setup process where to get the patterns from. Those patterns are then installed on your local machine.

Fabric, running on your machine, will only look in the $HOME/.config/fabric/patterns/ directory for patterns. It does not do any lookups based on the github URL.

So if you want to add a custom pattern, you can add a subdirectory there, with your custom patterns.

See, for example: https://github.com/ksylvan/custom-fabric

Hi @ksylvan , I know my post was super long so you probably skimmed. No worries. But, of course I created my own custom patterns in the appropriate directory. It's just that no matter how I alter my patterns, I get the same kind of results.

So, hoping somebody had this happen before, or I actually caught a non-bug more unexpected behavior that shipped.

[SasaCetkovic]

Maybe you should follow this recommendation (just a suggestion, I haven't tried yet):

Best Practices

1. Run extract_patterns.py when:

• Adding new patterns
• Updating existing patterns
• Modifying pattern structure

2. Description Writing:

• Use pattern extracts for context
• Keep descriptions clear and concise
• Focus on pattern purpose and usage

3. Tag Management:

• Use consistent tag categories
• Apply multiple tags when relevant
• Update tags to reflect pattern evolution

https://github.com/danielmiessler/fabric/blob/main/Pattern_Descriptions/README_Pattern_Descriptions_and_Tags_MGT.md

[ksylvan]

As of around v1.4.232 I added custom directory support.

Please update to the latest version and run fabric --setup (note that the repo has been reorganized and the patterns now live in data/patterns and strategies under data/strategies).

Closing this issue.