Experimental V2 renderer

[jhollinger]

An experimental serializer/renderer for the proposed V2 base. Rapidly evolving. I know it looks large, but most of that is tests. It's broken up into small commits so we can easily pull out pieces and PR them individually if we want. Notes:

• A simple formatter DSL in Blueprints
• Extractors are extensions
• The serializer dogfoods the reflection API
• The serializer dogfoods extension hooks: all built-in options come from built-in extensions
• Several optional extensions built in (should we have a blueprinter-contrib repo for these?)
• Hook design solves https://github.com/procore-oss/blueprinter-activerecord/issues/30
• Can be significantly faster than V1 - up to 60%! (details below)
• Blueprints get instantiated:
  • During each render
  • A single instance is shared throughout the render (e.g. a single WidgetBlueprint instance)
  • Field blocks, if blocks, formatter blocks, etc. are instance-evaled against the blueprint instance
  • This allows for helper methods and state during the render

Performance

The perf characteristics of V1 and V2 vary greatly based on:

1. The total number of fields and associations in a Blueprint.
2. The relative number of fields vs associations in a Blueprint.
3. How many items are being rendered overall.

# 2 fields, 1 objects, 0 collections
10,000 widgets 10x: V2 50.64% faster (0.3797 sec)
1,000 widgets 100x: V2 48.85% faster (0.3582 sec)
500 widgets 100x: V2 46.48% faster (0.1675 sec)
250 widgets 100x: V2 45.56% faster (0.0819 sec)
100 widgets 250x: V2 39.26% faster (0.0707 sec)
25 widgets 500x: V2 13.27% faster (0.0121 sec)
5 widgets 1000x: V2 56.76% slower (0.0473 sec)
1 widgets 1000x: V2 87.57% slower (0.0599 sec)

# 10 fields, 5 objects, 2 collections
10,000 widgets 10x: V2 59.70% faster (4.6962 sec)
1,000 widgets 100x: V2 57.14% faster (4.1727 sec)
500 widgets 100x: V2 56.30% faster (2.0148 sec)
250 widgets 100x: V2 54.93% faster (0.9509 sec)
100 widgets 250x: V2 53.53% faster (0.9159 sec)
25 widgets 500x: V2 49.31% faster (0.4208 sec)
5 widgets 1000x: V2 25.31% faster (0.0953 sec)
1 widgets 1000x: V2 46.88% slower (0.0715 sec)

# 100 fields, 50 objects, 25 collections
10,000 widgets 10x: V2 62.32% faster (77.7100 sec)
1,000 widgets 100x: V2 69.06% faster (87.1926 sec)
500 widgets 100x: V2 68.95% faster (43.3407 sec)
250 widgets 100x: V2 68.18% faster (20.9669 sec)
100 widgets 250x: V2 60.04% faster (14.8659 sec)
25 widgets 500x: V2 62.47% faster (7.7011 sec)
5 widgets 1000x: V2 56.03% faster (2.7653 sec)
1 widgets 1000x: V2 30.53% faster (0.3391 sec)

Note that on the very small end of things (few items with few fields) V2 is actually slower. But we're talking fractions of a ms.

[jhollinger]

@sandstrom

* Implement association as it's own thing, instead of having that be just a field. Fields are inherently different from associations, and it'll be easier to work with if that's reflected in the internal code as well. [related issue](https://github.com/procore-oss/blueprinter/issues/407)

I think this is accounted for. I've actually split "assocation" into "singular" and "multiple" variants (currently with the placeholder names object and collection). They're implemented as separate structs from field and can be fetched separately from the reflection API.

* Separate internal options from external ones, instead of having some 'blessed' or 'special' options among user-defined ones. [related issue](https://github.com/procore-oss/blueprinter/issues/405)

This is a little tricky given the hook-based design. To ensure that extensions can be just as powerful as most built-in functionality, all built-in options (if, unless, default, root, extractor, etc) are implemented with extension hooks. But I suppose we could still store those options separately if there's a consensus around it. I did move blueprint and view out of options, but for slightly different reasons.

[sandstrom]

I think this is accounted for. I've actually split "assocation" into "singular" and "multiple" variants (currently with the placeholder names object and collection). They're implemented as separate structs from field and can be fetched separately from the reflection API.

Ah, I see now. I was looking at this PR diff, but the true diff seems to be this one:

https://github.com/procore-oss/blueprinter/compare/main...jh/2.0-renderer

This is a little tricky given the hook-based design. To ensure that extensions can be just as powerful as most built-in functionality, all built-in options (if, unless, default, root, extractor, etc) are implemented with extension hooks. But I suppose we could still store those options separately if there's a consensus around it. I did move blueprint and view out of options, but for slightly different reasons.

I'm just providing feedback here, based on some thoughts I had when we implemented Blueprinter in our codebase.

I understand, though I guess I also slightly disagree with the idea to implement built-in functionality using extension hooks. I think I would have taken a different path, but it's also quite subjective so take this with a grain of salt 😄

Overall, great to see the new V2 taking shape! 🎉

[lessthanjacob]

Awesome work here @jhollinger!

Since we're aligned on the general direction, could we start pulling out more targeted branches from these changes? It'd be great to tie them back to some of the long-standing issues we've had open.

Additionally, I was hoping to jump back in and get more "hands on" again with this work, but didn't want to clobber into anything from this branch if it's accounted for here!

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sandstrom]

not stale

[sandstrom]

Awesome! 🎉