Can't use template string as test arg value

[nashgc]

Can't use template string as test arg value

example:

template_string Article() #" Test article with a lot of text ..."#


test extract_words {
  functions [ExtractWords]
  args {
    article #"
      {{ Article () }}
    "#
  }
}

test check_article {
  functions [CheckArticle]
  args {
    article #"
      {{ Article () }}
    "#
  }
}

[linear[bot]]

BAML-244 Can't use template string as test arg value

[Elijas]

+1 would be great for holding large strings in a separate file

[henryk1229]

+1 as well - is there an existing workaround for this?

[Elijas]

You can use this workaround

template_string TestArticle1() #" 
    Test article with a lot of text
"#

template_string GetArticle(article: string) #"
    {% if article == "TestArticle1" %}{{ TestArticle1() }}
    {% else %}{{ article }}
    {% endif %}
"#

function ExtractWords(article: string) -> string {
  client Default
  prompt #"
    {{ GetArticle(article) }}
    
    {{ ctx.output_format }}
  "#
}

test test1 {
  functions [ExtractWords]
  args {
    article TestArticle1
  }
}

[hanselke]

+1, the workaround seems clunky, gona have to write test wrappers for every function + then now test it. the jinja pipeline doesnt run on test input strings? best i've got is

// Our main function to analyze procurement documents
function AnalyzeProcurementDocument(user_input: string) -> ProcurementAnalysis {
  client Granite3
  prompt #"
    {{ AnalyzeProcurementDocumentPrompt(user_input)}}
  "#
}


function TestAnalyzeProcurementDocument(type: string) -> ProcurementAnalysis {
  client Granite3
  prompt #"
    {{ AnalyzeProcurementDocumentPrompt(
     GetTemplateMd(type)
    )}}
  "#
}

test Laptop_ProcurementDocumentTest {
  functions [TestAnalyzeProcurementDocument]
  args {
    type "laptop"
  }
}
// Test case for the function
test IcePack_ProcurementDocumentTest {
  functions [TestAnalyzeProcurementDocument]
  args {
    type "icepack"
  }
}

could live with that, be nice if we can just use template strings in test inputs tho.

[prrao87]

Ok, this has been causing me huge headaches - managing tests and duplicating strings everywhere all across my BAML file makes it look really ugly and tests to be far less usable. Can template_strings be repurposed to serve as containers to store long strings?