Hermetic Liquid Template Recording and Replay System

[tobi]

This PR implements a comprehensive hermetic recording system for Liquid templates that enables perfect deterministic replay of template executions with massive size optimizations.

What This Is

The Hermetic Template Recorder captures complete template execution state - including all variable access patterns, filter calls, and file system interactions - and stores it in optimized JSON recordings that can be replayed with perfect fidelity. This enables:

• Multi implementation testing - This makes it trivial to verify that multiple liquid implementations render the same outputs with the ability to selectively turn on/off filters as they are getting implemented in new versions.
• Production sampling - Record real production liquid runs, then re-run them later for testing
• Deterministic template testing - Record once, replay identically every time
• Performance analysis - Isolate template rendering from data fetching
• Template debugging - Inspect exact variable access patterns and filter chains
• CI/CD optimization - Use recordings instead of complex database setups

Key Features

🎯 Perfect Roundtrip Fidelity

• Zero differences between original and replayed output
• Captures both Liquid Drop objects and plain Hash/Array variables
• Hermetic replay uses pure data structures (no Drop object reconstruction)

📦 Massive Size Optimization

• Semantic key-based filter recording reduces file sizes by ~95% compared to full object space dump
• Smart data extraction from Drop objects
• Efficient JSON serialization of complex nested structures

🛠 Developer-Friendly CLI Tools

# Record template execution
bundle exec ruby bin/liquid-record output.json vogue product

# Verify perfect roundtrip
bundle exec ruby bin/liquid-record --verify output.json vogue product
✅ Verification PASSED - outputs match perfectly!

🔧 TrackableHash/Array Wrappers

• Intercepts property access on non-Drop objects
• Captures complete variable interaction patterns
• Maintains data type integrity throughout recording/replay cycle

How to Use

Basic Recording

# Record template execution

Liquid::TemplateRecorder.record(/tmp/recording.json) do
  # ...
  template = Liquid::Template.parse(source).render(assigns)
  # .... 
end

CLI Workflow

# Record a template with variables
bundle exec ruby bin/liquid-record recording.json template_dir variable_set

# Verify the recording works perfectly
bundle exec ruby bin/liquid-record --verify recording.json template_dir variable_set

# Replay without original data/database
replayer = Liquid::TemplateRecorder.replay_from('recording.json')
output = replayer.render

Integration Testing

# Unit test roundtrip functionality
def test_cli_roundtrip
  # Record template
  system("bundle exec ruby bin/liquid-record test.json dropify product")
  
  # Verify perfect replay
  replayer = Liquid::TemplateRecorder.replay_from('test.json')
  assert_equal expected_output, replayer.render
end

Technical Implementation

Drop-Free Replay Architecture

During recording, the system extracts complete underlying data from Liquid Drop objects and stores it as plain JSON. During replay, only Hash/Array structures are used - no Drop objects are reconstructed. This ensures:

• No dependency on original object models
• Faster replay performance
• Perfect data isolation

Semantic Key Generation

Filter calls are identified by semantic keys like product.title|upcase[0] instead of sequential indices, enabling:

• Resilient replay across template modifications
• Massive file size reduction through deduplication
• Better debugging and inspection capabilities

Smart Data Merging

The system merges original template assigns with dynamically accessed Drop properties using intelligent conflict resolution that preserves Hash structures and prevents type conversion issues.

Test Plan

• [x] Unit tests for CLI roundtrip workflow
• [x] Integration tests for recording/replay cycles
• [x] Verification of zero-diff output matching
• [x] Performance testing with complex nested data structures
• [x] Edge case handling for circular references and deep nesting

The system achieves perfect hermetic recording with 55KB JSON files containing complete execution state, enabling reliable template testing and analysis workflows.

• make theme_runner actually useful outside of the performance benchmarks
• the performance runner wasn't actually working before
• also fix profiler
• Apply suggestions from code review
• memory-profiler update
• more profile cleanups
• Implement hermetic template recording and replay system
• Add ProductDrop class for proper Liquid Drop object handling
• Reorder JSON structure and add CLI tools for better usability
• Implement TrackableHash/Array wrappers for hermetic variable recording
• Fix loop event processing to prevent Hash-to-Array conversion
• Add comprehensive CLI roundtrip unit test
• Update replayer and file system components for hermetic recording
• Update integration tests and test helper for hermetic recording

[isaacbowen]

:))))))))))))))))

[ianks]

This is really cool. I spent some time integrating this into our storefront to test it out, and here are some things that came up:

1. Multi-template architectures Our storefront uses JSON templates that render multiple section templates in a single request. The current recorder design captures a single template source, but in practice we often have dozens of section templates rendering together. It might be helpful to support recording multiple templates in a single recording session, perhaps with a templates array that tracks each parsed template along with its name/path.

2. Template parsing flow variations We noticed the recorder hooks into Liquid::Template.parse (class method), but in our codebase we create template instances first and then call parse on them:

template = Liquid::Template.new
template.parse(source, options)

The instance method interception works, but the RecordingTemplate wrapper approach might need some adjustments for this pattern.

3. Section rendering context For applications using section-based architectures, it would be valuable if the recording could capture which sections were rendered and in what order. This would make it easier to trace execution flow in complex page layouts.

4. File path resolution The file system recording works well. One thing we noticed is that the paths in our file system reads are often relative to different roots, so having some metadata about the base path context would help with replay accuracy.

5. Template source capture timing When templates are parsed through adapter layers or caching mechanisms, the source might not be available at the Liquid::Template.parse call site. Consider capturing source at multiple points in the template lifecycle to handle these cases.

The JSON schema is well structured and the recorder's approach to tracking variable access and filter calls is solid. Would be happy to discuss extending this to handle multi-template scenarios if you're interested.