mako
mako copied to clipboard
sqlalchemy
option to apply filters before the defaults ?
Migrated issue, originally created by Anonymous
Maybe I am wrong here, but shouldn't default filters such as unicode() be applied last? The fact that it's done the exactly opposite way makes it a pain, as we are limited to deal with the string/unicode representation of whatever variable is passed to the filter.
Hence, I propose the following change:
diff -r 29d9a098a70465450563e9c07ed613e227886936 mako/codegen.py
--- a/mako/codegen.py Fri Aug 05 17:45:51 2011 -0400
+++ b/mako/codegen.py Mon Aug 08 11:14:48 2011 +0200
@@ -669,7 +669,7 @@
if self.compiler.pagetag:
args = self.compiler.pagetag.filter_args.args + args
if self.compiler.default_filters:
- args = self.compiler.default_filters + args
+ args = args + self.compiler.default_filters
for e in args:
# if filter given as a function, get just the identifier portion
if e == 'n':
What do you think?
Cheers,
Pedro
Michael Bayer (@zzzeek) wrote:
Each of the u, h, and x filters expect a string. So if i said:
${x | h}
expecting "x" to be HTML encoded, then "x" happens to be an integer or some other non-string that is coerced by unicode(), kaboom.
The patch above breaks the one test we have in this regard which checks this:
def test_convert_str(self):
"""test that string conversion happens in expressions before sending to filters"""
t = Template("""
${x | trim}
""")
assert flatten_result(t.render(x=5)) == "5"
now I am absolutely aware what a PITA this is for non-string oriented filters like formatters. All throughout our templates, we just have a lot of "n" going on.
I have the impression that a better idea would be needed here. ${x | y} has "y" as a "filter" - it processes output. In practice, we use ${x | y} a lot for '''formatting'''. This is a different role. I'm seeing the "Filtering and Buffering" chapter being renamed "Filtering, Formatting, and Buffering" in the docs. Syntax change ?
${x | formatter1, formatter2, ... | filter1, filter2, ...}
${x & formatter1, formatter2, ... | filter1, filter2, ...}
?
then again, the usual approach:
${x | formatter1, n}
is actually more efficient than having the formatter + the unicode. The "formatters" we use already return a fully composed string, so the extra "unicode" call would be overhead (and all those unicode calls do add up). So, not sure I want to do anything here, perhaps update the docs to discuss the roles of "filter"/"formatting" regardless.
Anonymous wrote:
An alternative could be placing letting the conversion to string be done by those filters, no? If h and trim are expecting strings, they should make sure all the arguments are properly converted to strings beforehand.
Pedro
Michael Bayer (@zzzeek) wrote:
Replying to [comment:2 guest]:
An alternative could be placing letting the conversion to string be done by those filters, no? If
handtrimare expecting strings, they should make sure all the arguments are properly converted to strings beforehand.
but that is also wasteful and redundant, as well as an implicit behavior I don't like very much, for each filter to also embed string conversion within it. the filtering of unicode etc. is the primary performance bottleneck in mako.
Anonymous wrote:
Replying to [comment:3 zzzeek]:
I see. Too bad then. Still, a solution that would be easier than changing the whole syntax could be finding a way of implying the n in some of the filters without having to specify it manually. Maybe something like this could work?
def json(obj):
# ...
json.exclusive = True
I don't know about the performance penalty though.
Michael Bayer (@zzzeek) wrote:
having the formatter advertise as such might be nice, though again that's a runtime check since filters/formatters can be passed into the template.
A special character, perhaps:
${mydata | >json}
Anonymous wrote:
([email protected] here — can I create a new account for Trac somehow so I can get followups by email?)
I wanted to add a json filter and hoped to make it usable as ${x | json} without having to add any extra n filters. I only realized that the default_filters are applied before other filters after messing around with this for a day.
Could the name be changed to initial_filters or something similar? That is, the name could be made clearer to indicate that the filters are not ''defaults'' (that can be overwritten later) but unconditionally applied filters.
Adding "formatters" would be nice — anything that lets us work on the Python values before they're turned into strings.
I'll now go and change my ${x | json} fragments into ${json(x)} in the templates. That '''does''' work since it is compiled into what I need: filters.html_escape(json(x)). My filter outputs a unicode subclass which defines __html__ to return self, which makes the html_escape a no-op.
Michael Bayer (@zzzeek) wrote:
my plan is entirely to move this trac to the bitbucket repo, but I don't have a completed script that can convert all the issues to their import format (I have one I was writing, but it's a big project).
Feel free to just discuss on the mailing list, this trac is pretty out of the loop.
Michael Bayer (@zzzeek) wrote:
oh but name change, i dont really think we can change any names. mako is installed very broadly in projects that don't really want to look at their mako code very often so i dont think backwards incompatible anything is possible right now.
Michael Bayer (@zzzeek) wrote:
if "default_filters" is ambiguous, i dont see why we can't just make sure the documentation makes these clear. in my mind "default" means, "the filters that get applied in all cases no matter what".
where would "formatters" go exactly, im not understanding that ? the "turning things into strings" is what default_filters does right now, if you put a function of your own in there, it replaces the "turning things into strings" with whatever you want. the proposal here is to somehow extend the "|" syntax to allow filters before or after the default chain.
Anonymous wrote:
(Martin here). I'm not a native English speaker, so my understanding of "default" may be wrong. I had just expected it to be the filters that would be applied ''unless'' an expression specified other filters. Like default arguments for Python: def foo(x=10) defines a function where x is 10 ''unless'' I specify otherwise when I call the function.
It doesn't matter much, though, and I agree that updating the documentation would be the way to go here. You should definitely not make backwards incompatible changes ("renaming" the keyword argument could be done by adding a new one and only documenting that but still accepting the old one).
With regard to formatters or filters: when I first read about Mako filters, I thought that ${x | foo} was equivalent to ${foo(x)}. This was further reinforced when I saw that you can define a function inside a template and use that as a filter. The crucial difference, though, is the order of application and that wasn't clear to me until I read create_filter_callable in codegen.py.
Anonymous wrote:
PS: Thanks for the hint about the mailinglist, I'll use that next time!
Changes by Michael Bayer (@zzzeek):
- changed title from "default filters should be applied last" to "option to apply filters before the defaults ?"
How about adding two more settings, fallback_filters and final_filters, in addition to default_filters?
You could have fallback_filters be used if there are no other filters specified. This captures the understanding of default voided in this request here. It also allows for fallback_filters to perform some escaping or formatting that is right most of the time, and to specify an alternate formatting or escaping in situations where it is not.
And you'd have final_filters applied to the result. I would suggest doing so unconditionally. That would address #272 which could use final_filters=['str'] to achieve conversion of everything to strings. It would also allow use as a safety feature: some form of safe HTML library could force-escape its inputs unless they are objects from that library which have been constructed in some other fashion that ensures proper escaping.
Just re-read all this.
my first thought is that whatever we do here should fix #171 and #272 at the same time as these seem to be asking for almost the same thing.
my next thought is, if we have: fallback_filters (if no filters) -> default filters (if no n) -> template filters from "|" -> final_filters, that looks like a rule chain to me, which the proposal here is to hardcode.
can we instead have an actual filter rule chain and just build up a default that looks like the above, but folks can go in there and build whatever chain they want and/or we can document certain kinds of chains etc.?
I'm not sure how these chains could be best specified. Let's say they want to run "u" and "trim" if no other template filters are given, "sanitize" if no "n" is given, then the template specified filters, then a final filter called "stringify". Three new "meta" filters fallback(), default(), and template() would be control flow containers. A specification might look like:
from mako import filters
TemplateLookup(filter_chain=[filters.fallback(filters.u ,filters.trim), filters.default(filters.sanitize), filters.template(), filters.stringify)
I'm not sure about "filters.u" vs "u" but the main thing is, instead of having three keyword arguments that refer to a hardcoded flow that would need to be documented, have an open-ended rule chain that is largely self documenting and can be changed and extended.
