huge performance-penalty when using extends/partials vs. inline rendering

[mugwump]

I just converted an inline-rendering example to using partials:

collection @elements
attribute :id, :subject, :description, :project_id, :parent_id

node ...
node ...
....

where the code was inlined into the collection-view, to using partials:

collection @elements
extends "element/element"

which yields the exact same output, but the performance drops drastically with huge collections:

from

Completed 200 OK in 680ms (Views: 318.7ms | ActiveRecord: 22.9ms)

for the inline-view

to

Completed 200 OK in 2267ms (Views: 1919.2ms | ActiveRecord: 22.6ms)

for the views using extends.

Anything I can do about it?! Anything I need to cache here? Are the partials re-read for every item in the collection??

[mugwump]

PS: It gets a little better with

config.cache_all_output = true
config.cache_sources = true 
config.cache_engine = Rabl::CacheEngine.new
config.perform_caching = true

but I never even get near the inline-performance...

[Aerlinger]

Yup, happens with me too. Some responses are close to ~5 seconds and 90% of that time is spent rendering rabl view. Not sure what the simple solution is.

[nesquena]

How many items are you guys rendering out of curiosity to get the count to 5 seconds?

[Aerlinger]

It's rather large query. User -> has many games -> has many turns

[nesquena]

Yeah partial and extends can be pretty slow, it's not as fast as inline performance. It could be optimized certainly, but I don't tend to work with APIs with data sets where I return more than say ~100 items at once (with pagination) so for me the rabl views are so fast its a complete non-issue. I'd love for someone else to take a look at some point and see if the extends / partial can be optimized.

[Aerlinger]

I think the problem may be the deep nesting. Only return a few items (games) though:

user {
  games: [
    game: {
      turns: {
        turn: {
           receiver: (user) {
              ...
           }.
        }
      }
    }
  ]
}

so there are three nested extends.

[nesquena]

I see yeah I've seen that more than 2 levels deep of extends combined with tons of root level items can be a recipe for poor performance with rabl. As I mentioned, it's outside my use case but I am happy to work with others to make this faster. Check out rabl-rails for one attempt that never made it back to the mainline but is faster for complex responses with very similar syntax.

[mugwump]

I'm not sure, that this is actually caused by the nesting: We're not nesting deep, only have many items to render (about 1000) - probably the actual reading of the partials may be the problem?!

On a lighter note: It is possible to make most of the issues go away by just constructing nested hashes inside a node - this probably gets it nearer to the rabl_rails-implementation - which I guess works by constructing a hash first and rendering in a second step (only browsed over the docs here, so I maybe wrong...). And using presenters you can achieve similar results.

[toxaq]

I've turned to using helpers to avoid this.

module RablHelpers
  def game_attributes(rabl
     rabl.attributes [:id, :name, :score]
  end
end

Then in index.json.rabl

collection @games

game_attributes(self)

And in show.json.rabl

 object @games

 game_attributes(self)

This avoids calling extends which unfortunately relies on creating a new Rabl::Engine for every single item in the collection.

[nesquena]

That's not a bad idea, thanks for sharing

[kuon]

I have the same issue, so I went the helper route too.

I think the simpler fix would be to implement something like include("path") that would act more like a C preprocessor include and have a very simple pre processor that build 1 large file out of smaller files.

[nesquena]

Yeah that's true a rabl pre processor is a good suggestion. All the syntax and would keep the speed same as a flat file. — Nathan Esquenazi CodePath Co-founder http://thecodepath.com

On Wed, Dec 18, 2013 at 3:18 PM, kuon [email protected] wrote:

I have the same issue, so I went the helper route too.

I think the simpler fix would be to implement something like include("path") that would act more like a C preprocessor include and have a very simple pre processor that build 1 large file out of smaller files.

Reply to this email directly or view it on GitHub: https://github.com/nesquena/rabl/issues/500#issuecomment-30891223

[abrandoned]

This issue may be largely abandoned, but we have built some basic "compilation" rake tasks and verification tasks that help us take advantage of Rabl without the performance penalty outlined above (using a similar mechanism as the preprocessor, but relegated to a rake task that we run in our CI pipeline) ... if anything is interested in using or contributing https://github.com/abrandoned/rabl-extend-compiler

[nesquena]

@abrandoned This is really neat, thanks for sharing!