detection-rules icon indicating copy to clipboard operation
detection-rules copied to clipboard

[Bug] TOML string outputs are not properly escaped

Open pberba opened this issue 2 months ago • 2 comments

Describe the Bug

When the triple double quotes are used to output raw strings is used, the TOML is not checked for escaped characters and the rendered output is

To Reproduce

If I have a rule with a query DSL filter, that is sufficient long and an escaped character, this will cause the formatter to incorrectly output this as a raw string

{
  "query_string": {
    "query": "file.path: \"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\" file.path: Hello\\:World"
  }
}

And the TOML will look like this


[[rule.filters]]

[rule.filters."$state"]
store = "appState"
[rule.filters.meta]
alias = "Custom Filter"
disabled = false
index = "apm-*-transaction*,auditbeat-*,endgame-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-*,-*elastic-cloud-logs-*"
key = "query"
negate = false
type = "query_string"
value = 'file.path: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" file.path: Hello\:World'
[rule.filters.query.query_string]
query = """
file.path: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" file.path: Hello\:World
"""

Expected Behavior

The output should always be escaped or the condition for raw should include checking for \ characters.

https://github.com/elastic/detection-rules/blob/66a0b6b97c47957e5019d681943f4ff8ed3470ac/detection_rules/rule_formatter.py#L145

Screenshots

Screenshots

Sample Filter

Image

Corrupted TOML

Image

Desktop - OS

None

Desktop - Version

No response

Additional Context

No response

pberba avatar Oct 03 '25 06:10 pberba

Running into this issue as well which is causing a downstream app which relies on this code to fail :(

Do we have any proposed solution for this?

willbrownecs avatar Nov 24 '25 21:11 willbrownecs

I would also like to see this resolved. It's occuring in 4 out of almost 2000 rules in our environment.

FinalFrag avatar Nov 27 '25 15:11 FinalFrag