Investigate perf benefits of using jsonapi

[snewcomer]

• [ ] investigate perf benefits with https://github.com/jeregrine/jsonapi

[snewcomer]

Name                    ips        average  deviation         median         99th %
ja_serializer        4.02 K        0.25 ms    ±24.68%        0.23 ms        0.48 ms
jsonapi              0.21 K        4.87 ms   ±105.40%        4.18 ms        9.60 ms

Comparison:
ja_serializer        4.02 K
jsonapi              0.21 K - 19.60x slower

TaskListView with a lot of tasks shows that jsonapi is significantly slower. However, when not dealing with relationships, jsonapi is faster.

Name                ips        average  deviation         median         99th %
jsonapi        213.45 K        4.69 μs  ±3440.93%           4 μs          10 μs
existings       38.04 K       26.29 μs   ±283.89%          23 μs          50 μs
Comparison: 
jsonapi             213.45 K
ja_serializer       38.04 K - 5.61x slower

[mitchellhenke]

For the tests, is each rendering producing the same result?

I ask because the defaults between the two libraries are different, and so one may be rendering more or less data in different situations 🙂

[snewcomer]

Hey @mitchellhenke. Yeah jsonapi is rendering more data.

Here is the benchee example.

Moreover, this is based on the seeds for Tasks.

So a TaskList will have many tasks (~50). Here is an example payload with one task:

jsonapi %{:data => %{attributes: %{inbox: false, inserted_at: #DateTime<2017-11-02 15:33:10Z>, name: "Test task list", order: 1000, updated_at: #DateTime<2017-11-02 15:33:10Z>}, id: "1704", links: %{self: "/task-list/1704"}, relationships: %{project: %{data: %{id: "5814", type: "project"}, links: %{related: "/project/5814", self: "/task-list/1704/relationships/project"}}, tasks: %{data: [%{id: "1680", type: "task"}], links: %{related: "/task", self: "/task-list/1704/relationships/tasks"}}}, type: "task-list"}, :included => [%{attributes: %{}, id: "5814", links: %{self: "/project/5814"}, relationships: %{}, type: "project"}, %{attributes: %{archived: false, body: nil, created_at: #DateTime<2017-11-02 15:33:10.223630Z>, created_from: "code_corps", inserted_at: #DateTime<2017-11-02 15:33:10Z>, markdown: "A test task", modified_at: #DateTime<2017-11-02 15:33:10.223641Z>, modified_from: "code_corps", number: 1, order: 1000, status: "open", title: "Test task", updated_at: #DateTime<2017-11-02 15:33:10Z>}, id: "1680", links: %{self: "/task/1680"}, relationships: %{comments: %{data: %{id: nil, type: "comment"}, links: %{related: "/comment/", self: "/task/1680/relationships/comments"}}}, type: "task"}], :links => %{}}

ja_serializer %{"data" => %{"attributes" => %{"inbox" => false, "inserted-at" => #DateTime<2017-11-02 15:33:10Z>, "name" => "Test task list", "order" => 1000, "updated-at" => #DateTime<2017-11-02 15:33:10Z>}, "id" => "1704", "relationships" => %{"project" => %{"data" => %{"id" => "5814", "type" => "project"}}, "tasks" => %{"data" => [%{"id" => "1680", "type" => "task"}]}}, "type" => "task-list"}, "jsonapi" => %{"version" => "1.0"}}

So the thought is putting together this much data is causing the difference? I think you are correct.

[snewcomer]

@mitchellhenke also if you would like, do you want me to make a PR to jsonapi to make those options configurable and then run the same benchmarks?

[joshsmith]

Yeah, as much as we can make the comparison apples-to-apples here, the better informed we can be.

[snewcomer]

ok I'll look into this and ping ya back later today.

[snewcomer]

@JoshSmith @mitchellhenke yep you are right.

Name               ips        average  deviation         median         99th %
ja_serializer       27.56 K       36.29 μs    ±52.96%          32 μs          92 μs
jsonapi        14.26 K       70.13 μs    ±40.93%          61 μs         167 μs

Comparison: 
ja_serializer       27.56 K
jsonapi        14.26 K - 1.93x slower

jsonapi payload %{:data => %{attributes: %{inbox: false, inserted_at: #DateTime<2017-11-03 19:19:22Z>, name: "Test task list", order: 1000, updated_at: #DateTime<2017-11-03 19:19:22Z>}, id: "285", relationships: %{project: %{data: %{id: "963", type: "project"}}, tasks: %{data: [%{id: "281", type: "task"}]}}, type: "task-list"}}

[snewcomer]

Name               ips        average  deviation         median         99th %
jsonapi       176.20 K        5.68 μs  ±1025.94%           5 μs          15 μs
existing       30.49 K       32.80 μs   ±116.45%          29 μs          74 μs

Comparison: 
jsonapi       176.20 K
existing       30.49 K - 5.78x slower

@JoshSmith pulled in jsonapi master on this branch w/ a file priv/repo/benchit.exs. Now things are making more sense. Tested with ~50 tasks and ~1450 tasks...Still approx. the same results.

jsonapi payload %{:data => %{attributes: %{inbox: false, inserted_at: #DateTime<2017-11-08 22:45:29Z>, name: "Test task list", order: 1000, updated_at: #DateTime<2017-11-08 22:45:29Z>}, id: "2748", relationships: %{project: %{data: %{id: "10047", type: "project"}}, tasks: %{data: [%{id: "2713", type: "task"}]}}, type: "task-list"}, :included => [], :links => %{}}

[mitchellhenke]

This looks great, and both appear to be quite fast. Thanks for doing these benchmarks and contributing improvements to JSONAPI!

[snewcomer]

@begedin FYI. Will wait for your thoughts before tackling.

[begedin]

Looks like it's an overall improvement in speed, so I like that. I also like that it's less rails-like and more elixir-like.

A potential difficulty is that our controller tests, primarily the code we use to reduce controller test boiler plate, are more coupled to ja_serializer than we would like.

@snewcomer I see you've been working with the task view/controller. Since that is the case, how would you feel about submitting an example PR that switches the task controller/view to json API fully, including the tests?

[snewcomer]

@begedin Yeah I can definitely do that! I'll try to get something done by Monday.

[joshsmith]

Is it possible we can mitigate the pain of the controller test boilerplate by instead using https://github.com/DockYard/json_api_assert?