Support dynamic filters in `custom_query` (elasticsearch, OpenSearch)

[tstadel]

Is your feature request related to a problem? Please describe. Currently we only support filters in custom_query if we write proper placeholders into the custom_query for each filter field (e.g. $years). That translates into not being able to use dynamic filters at query time easily. You would need to pass a different custom_query if you want to change the filter. Also you cannot use more complex filter expressions using filters but rather have to rebuild them in the custom_query. For production use-cases this is not an appropriate solution effectively disabling Haystack's filters for custom queries.

Describe the solution you'd like Filters should work for custom_query in the same way as for standard queries. As elasticsearch/OpenSearch queries can be quite complex and it's not at all straight forward where to insert the filter statement, best would be to let the user set a placeholder for filters like we already do for query. This would be a breaking change as it's not compatible with the way we handle filters for custom_query today. As mentioned this should not be a bummer as they are rather useless in the current implementation.

Describe alternatives you've considered

• insert filter expression automatically without placeholder (as mentioned there are many ways to insert filter strings in those queries, so this might be pretty fragile)

Additional context To further facilitate the handling of custom_query as it's getting more and more important for production use-cases we should think about supporting dicts instead of strings (or both) like proposed in https://github.com/deepset-ai/haystack/issues/3489.