Apply the_content filter to wysiwyg output for shortcodes and oembed

[adriantoll]

Putting the discussion here in case anyone finds it of interest...

Currently wysiwyg fields have the filters wp_kses_post (potentially) and wpautop (always) applied before saving to the database. This means that the field content has <p> and </p> tags, which are added by TinyMCE, saved into the database field itself. This is unlike standard post content wysiwyg fields, which have <p> and </p> tags stripped before being saved to the database, and then added again on output by the the_content filter.

The fact that DCF doesn't apply the_content to wysiwyg output means that shortcodes aren't converted. Additionally, if the_content is applied to the output, shortcode are converted but oembeds are not. This is because the oembed regexp looks for an oembed URL directly preceded by a carriage return, whereas the field output contains lines that start with <p> tags, e.g. <p>http://www.youtube.com/...

The idea is to strip <p> and </p> tags when saving the field and to apply the_content to the output, which will automatically enabled shortcodes and oembeds. This would also enable the use of things like the Gravity Forms button which is also displayed alongside the Add Media button with the media_buttons setting in wysiwyg_options.

[As a side note, the "Add Media" button above wysiwyg fields works as expected as it just adds the HTML tags for that image.]

the_content applies the following filters as far as I can see...

wptexturize - http://codex.wordpress.org/Function_Reference/wptexturize convert_smilies - http://codex.wordpress.org/Function_Reference/convert_smilies convert_chars - http://codex.wordpress.org/Function_Reference/convert_chars wpautop - http://codex.wordpress.org/Function_Reference/wpautop shortcode_unautop - http://wpseek.com/shortcode_unautop/ prepend_attachment - http://wpseek.com/prepend_attachment/ capital_P_dangit - http://codex.wordpress.org/Function_Reference/capital_P_dangit do_shortcode - http://codex.wordpress.org/Function_Reference/do_shortcode

... and also does the oembeds.

It looks like the_content therefore doesn't do much of significance in terms of backwards compatibility, although there will of course be other plugins which hook into that filter to do extra filtering.

The main question seems to be whether this should be default behaviour, or whether this behaviour should have to be turned on explicitly. The fact that the_content doesn't do a great deal, and the built-in post content field has this applied by default, would argue for it to be default behaviour. I suppose there may be cases of unexpected behaviour when used with other plugins, but if the plugin currently doesn't convert shortcodes or oembed URLs, then those won't be included in anyone's existing DCF wysiwyg content fields anyway. I would argue the same is likely to be true of other plugins in the vast majority of cases.

The database fields for wysiwyg content will have <p> and </p> tags in them. When the_content is applied to fields like this, these tags are processed without any issue, so there are no conflicts with existing stored fields as far as I can see.

Therefore, my conclusion would be to strip <p> and </p> tags when saving wysiwyg fields and apply the_content to the output. Any other opinions?

[gyrus]

Haven't got much time to give to this at the moment, but my feeling is that I'm not sure about the range of possible issues if we include these changes as defaults.

For existing sites that just add a WYSIWYG field, then "manually" apply the_content filter on output, I gather you're saying that applying the_content automatically wouldn't hurt because applying the filter twice gives the same result as applying it once. Has this been tested to confirm?

However, even if this has been tested, the simple fact is we have no idea what different things people are doing to the output from WYSIWYG fields, and therefore have no idea how adding this as a default might affect things. Sure, we might come up with a bunch of "known unknowns" and rule them out as problems, but in rolling out software updates it's the "unknown unknowns" that are the problem. The simplest way of avoiding these is to make new behaviours "opt-in".

Check out the allowed_html parameter too (see http://sltaylor.co.uk/wordpress/developers-custom-fields-docs/). Would you be just stripping <p> tags for this? What about other tags that TinyMCE includes? If we had a new parameter, something like store_wysiwyg_markup (defaulting to false), how might this clash with allowed_html?

For the output, I guess we'd be adding a new optional $apply_content_filter parameter for slt_cf_field_value() and slt_cf_all_field_values() (defaulting to false). We might also have a constant, like SLT_CF_USE_GMAPS, to trigger this globally without having to add the new parameter to each call to slt_cf_field_value() and slt_cf_all_field_values().

[adriantoll]

Fair enough - you know more about the inner workings of the plugin and Wordpress than I do, so if it requires more thought let's park it for now. I have it working on the site I needed to by stripping the

tags and then applying the_content, so there's no urgency from this end.

[gyrus]

I'll definitely leave this open, I'll be in touch soon if I get some time to work on this. Hopefully something can make it into the next release.

[adriantoll]

Closing as DCF is not being actively developed any more.