New Liquid format method

[shopmike]

Liquid Format

Liquid::Template.parse(template).format

This is a work in progress but I wanted to keep visibility and start discussions around it. I'm going to spend some of my spare time on this as I believe its a strong way to move Liquid forward. There is still a fair bit to do but shouldn't take more than a couple of weeks given that I'm not spending much time on this.

Why is this needed

1. Removal of the :lax and :warn parsers - I would like to think a v5 or v6 of Liquid will deprecate both these parsing modes. The main idea behind the format command is it provides a way to parse with the :lax or :warn parsers and then output as :strict compatible syntax. So a possible way to move from v4 to v5 is simply running the format command across previous templates. An online conversion tool could also be provided for end users by liquid platforms so they convert templates outside of the system.

2. Provides an upgrade path - There have been requests over time for specific features that make sense in Liquid however they are impossible to implement without impacting previous templates. A simple example of this is brackets within an if statement, people currently have these in their template but they currently don't do anything, introduction of this will likely change the rendering output and the format tool can be a way to provide the upgrade tools to make breaking changes.

3. Provides systems with consistent formatting - One of the best features of Go is the inbuilt go fmt command. For me this shows a great example that by having an in-built format command systems can then leverage it to provide a better experience. For example a theme editor can run the format command on save and make the whitespace around tags and filters uniform so that future developers have a consistent experience.

4. Provides a way to debug - By running code through the format command, you actually get a chance to see how Liquid is processing your template. Its rather common to realise that for some reason Liquid is actually just ignoring most of your code.

What's left to do

White Space

• [x] Retain whitespace in an IgnoredWhitespace token so format can recover.
• [x] Determine if a tag has white space controls
• [x] Handle Whitespace control with inner control tags

Expressions

• [x] Detect when a string should be a single quote or double quote
• [x] Handle integers
• [x] Handle ranges
• [x] Handle floats
• [x] Handle literals

Tags

• [x] Assign
• [x] Break
• [x] Capture
• [x] Case
• [x] Comment
• [x] Continue
• [x] Cycle
• [x] Decrement
• [x] For
• [x] If
• [x] Ifchanged
• [x] Include
• [x] Increment
• [x] Raw
• [x] Table_row
• [x] Unless
• [ ] Liquid
• [ ] Render

Nodes

• [x] Raise error if any node doesn't support format

Discussion topics

1. I decided to put these in the core of the system, I feel like this method can be forgotten if it was just a side project and its compatibility was lost. Also by putting the methods directly on tags it makes it easy to add this support to existing tags. However I would like to know if anyone thinks there is a better approach

2. The whitespace control I'm going to modify to output a tag that has an empty render method like comments but stores what that whitespace originally looked like. Any feedback or comments on this will be helpful?

@pushrax @ashmaroli @fw42 @samdoiron @dylanahsmith @Thibaut @tobi

[tobi]

Very cool.

[fw42]

The general idea of rewriting bad templates into good templates makes a lot of sense to me. I remember there might have been a prototype (hackdays?) for this already a few years ago (@pushrax?) but it never got shipped.

Do we care about the scenario of someone having an old template stored offline somewhere (maybe a theme they bought and downloaded to their own computer) and in five years from now they come back and expect that theme to still work?

[tobi]

Yea offline backwards compatibility has to be ensured. But at the very very least we can integrate this as a format button into the online editor and the CLI tools. We should probably format the html around the liquid as well in those cases though.

• tobi CEO Shopify

On Fri, Aug 9, 2019 at 10:16 AM Florian Weingarten [email protected] wrote:

The general idea of rewriting bad templates into good templates makes a lot of sense to me. I remember there might have been a prototype (hackdays?) for this already a few years ago (@pushrax https://github.com/pushrax?) but it never got shipped.

Do we care about the scenario of someone having an old template stored offline somewhere (maybe a theme they bought and downloaded to their own computer) and in five years from now they come back and expect that theme to still work?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Shopify/liquid/pull/1124?email_source=notifications&email_token=AAAACWZUPLLJXCPAFIF4IPDQDV35NA5CNFSM4IKPJ4W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD36ZRWI#issuecomment-519936217, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAACWZOCI737SJNIUKNPZ3QDV35NANCNFSM4IKPJ4WQ .

[shopmike]

This is where I think an online tool would be beneficial that goes along with this, just thinking a really basic heroku type setup that just runs ruby and you can upload multiple liquid files and get an output. I think if you upgrade a system, the error message could possibly point to this location suggesting you may be using an outdated syntax.

[tobi]

That would be too byzantine for merchants who are often non-technical and treat these theme zip files as found artifacts of unknown power and functionality.

• tobi CEO Shopify

On Fri, Aug 9, 2019 at 10:22 AM Mike Angell [email protected] wrote:

This is where I think an online tool would be beneficial that goes along with this, just thinking a really basic heroku type setup that just runs ruby and you can upload multiple liquid files and get an output. I think if you upgrade a system, the error message could possibly point to this location suggesting you may be using an outdated syntax.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Shopify/liquid/pull/1124?email_source=notifications&email_token=AAAACW24YT5S3I7NGVJU3R3QDV4RBA5CNFSM4IKPJ4W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD36Z72I#issuecomment-519938025, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAACWYZTZ4IC6IEYSJVQMLQDV4RBANCNFSM4IKPJ4WQ .

[evulse]

Actual zip files of themes I think is an easier one as if it fails to load you can pretty much guarantee it was valid code before, so you can just host a service internally that ships the zip off when it fails and converts it and then loads it in. For me working with bigger merchants the guys that have to do to the most work are the ones that run it through version control. They will need to upload all their files to a service, pull them down add them back in and commit.

[shopmike]

This is a duplication of https://github.com/Shopify/liquid/pull/561 . This pull request does consider changes that have happened in Shopify since the other pull request was written. So next steps are to take the best of both and merge. Good news is this completes the 2 tags that yet to be completed.

[shopmike]

Needs to now support {% liquid %} and {% render %}. Added to TODO list