Dynamic Includes

[gwincr11]

The next step in building out active record dynamic includes and the new load tree debug view as proposed in https://github.com/rails/rails/pull/45071 this build on the load tree from https://github.com/rails/rails/pull/45161 and add the dynamic includes functionality.

The dynamic includes functionality is a feature that allows you to automatically load the associations for a model when you iterate over the record and hit an association that is not yet loaded.

Example Usage

These examples are taken from the tests added to the PR. Without dynamic includes enabled, the following implementation would execute 9 queries:

developers = Developer.where(id: [developer.id, developer2.id])
assert_queries(9) do
  developers.each do |d|
    d.ship.parts.each do |part|
      part.trinkets.each do |t|
        t
      end
    end
  end
end

SQL Log

D, [2022-05-05T16:18:46.432004 #3280] DEBUG -- :   Developer Load (0.2ms)  SELECT "developers"."id", "developers"."name", "developers"."salary", "developers"."firm_id", "developers"."mentor_id", "developers"."legacy_created_at", "developers"."legacy_updated_at", "developers"."legacy_created_on", "developers"."legacy_updated_on" FROM "developers" WHERE "developers"."id" IN (?, ?)  [["id", 1], ["id", 11]]
D, [2022-05-05T16:18:46.432893 #3280] DEBUG -- :   Ship Load (0.1ms)  SELECT "ships".* FROM "ships" WHERE "ships"."developer_id" = ? LIMIT ?  [["developer_id", 1], ["LIMIT", 1]]
D, [2022-05-05T16:18:46.433554 #3280] DEBUG -- :   ShipPart Load (0.1ms)  SELECT "ship_parts".* FROM "ship_parts" WHERE "ship_parts"."ship_id" = ?  [["ship_id", 2]]
D, [2022-05-05T16:18:46.434335 #3280] DEBUG -- :   Treasure Load (0.1ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_id" = ? AND "treasures"."looter_type" = ?  [["looter_id", 1], ["looter_type", "ShipPart"]]
D, [2022-05-05T16:18:46.434911 #3280] DEBUG -- :   Treasure Load (0.1ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_id" = ? AND "treasures"."looter_type" = ?  [["looter_id", 2], ["looter_type", "ShipPart"]]
D, [2022-05-05T16:18:46.435559 #3280] DEBUG -- :   Ship Load (0.0ms)  SELECT "ships".* FROM "ships" WHERE "ships"."developer_id" = ? LIMIT ?  [["developer_id", 11], ["LIMIT", 1]]
D, [2022-05-05T16:18:46.436479 #3280] DEBUG -- :   ShipPart Load (0.1ms)  SELECT "ship_parts".* FROM "ship_parts" WHERE "ship_parts"."ship_id" = ?  [["ship_id", 751016585]]
D, [2022-05-05T16:18:46.437258 #3280] DEBUG -- :   Treasure Load (0.1ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_id" = ? AND "treasures"."looter_type" = ?  [["looter_id", 3], ["looter_type", "ShipPart"]]
D, [2022-05-05T16:18:46.438098 #3280] DEBUG -- :   Treasure Load (0.1ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_id" = ? AND "treasures"."looter_type" = ?  [["looter_id", 4], ["looter_type", "ShipPart"]]

When enabling dynamic includes it happens in 4:

developers = Developer.where(id: [developer.id, developer2.id])

ActiveRecord.enable_dynamic_includes do
  assert_queries(4) do
    developers.each do |d|
      d.ship.parts.each do |part|
        part.trinkets.each do |t|
          t
        end
      end
    end
  end
end

SQL Log

D, [2022-05-05T16:18:46.388805 #3280] DEBUG -- :   Developer Load (0.2ms)  SELECT "developers"."id", "developers"."name", "developers"."salary", "developers"."firm_id", "developers"."mentor_id", "developers"."legacy_created_at", "developers"."legacy_updated_at", "developers"."legacy_created_on", "developers"."legacy_updated_on" FROM "developers" WHERE "developers"."id" IN (?, ?)  [["id", 1], ["id", 11]]
D, [2022-05-05T16:18:46.424796 #3280] DEBUG -- :   Ship Load (0.3ms)  SELECT "ships".* FROM "ships" WHERE "ships"."developer_id" IN (?, ?)  [["developer_id", 1], ["developer_id", 11]]
D, [2022-05-05T16:18:46.425470 #3280] DEBUG -- : Dynamically preloaded: Developer.ship
D, [2022-05-05T16:18:46.427809 #3280] DEBUG -- :   ShipPart Load (0.2ms)  SELECT "ship_parts".* FROM "ship_parts" WHERE "ship_parts"."ship_id" IN (?, ?)  [["ship_id", 2], ["ship_id", 751016585]]
D, [2022-05-05T16:18:46.428172 #3280] DEBUG -- : Dynamically preloaded: Developer.ship.parts
D, [2022-05-05T16:18:46.430243 #3280] DEBUG -- :   Treasure Load (0.2ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_type" = ? AND "treasures"."looter_id" IN (?, ?, ?, ?)  [["looter_type", "ShipPart"], ["looter_id", 1], ["looter_id", 2], ["looter_id", 3], ["looter_id", 4]]
D, [2022-05-05T16:18:46.430748 #3280] DEBUG -- : Dynamically preloaded: Developer.ship.parts.trinkets

In some scenarios it is ideal to not preload all the data for a particular algorithm, if perhaps it has a very expensive database query associated and perhaps a select runs prior to loading the data to limit the selected data further in memory, you may want to turn off the dynamic includes in such a scenario. This is an exceptional case but is possible.

The below example will take 7 queries:

ActiveRecord.enable_dynamic_includes do
  developers = Developer.where(id: [developer.id, developer2.id])
  assert_queries(7) do
  developers.each do |d|
    assert ActiveRecord.dynamic_includes_enabled?
    ActiveRecord.disable_dynamic_includes do
      d.ship.parts.each do |part| # This should n+1
        assert_not ActiveRecord.dynamic_includes_enabled?
          ActiveRecord.enable_dynamic_includes do
            part.trinkets.each do |t|
              t
              assert ActiveRecord.dynamic_includes_enabled?
            end
         end
         assert_not ActiveRecord.dynamic_includes_enabled?
       end
     end
     assert ActiveRecord.dynamic_includes_enabled?
   end
 end
end

SQL Log

D, [2022-05-05T16:18:46.439329 #3280] DEBUG -- :   Developer Load (0.2ms)  SELECT "developers"."id", "developers"."name", "developers"."salary", "developers"."firm_id", "developers"."mentor_id", "developers"."legacy_created_at", "developers"."legacy_updated_at", "developers"."legacy_created_on", "developers"."legacy_updated_on" FROM "developers" WHERE "developers"."id" IN (?, ?)  [["id", 1], ["id", 11]]
D, [2022-05-05T16:18:46.440002 #3280] DEBUG -- :   Ship Load (0.1ms)  SELECT "ships".* FROM "ships" WHERE "ships"."developer_id" = ? LIMIT ?  [["developer_id", 1], ["LIMIT", 1]]
D, [2022-05-05T16:18:46.440643 #3280] DEBUG -- :   ShipPart Load (0.1ms)  SELECT "ship_parts".* FROM "ship_parts" WHERE "ship_parts"."ship_id" = ?  [["ship_id", 2]]
D, [2022-05-05T16:18:46.441803 #3280] DEBUG -- :   Treasure Load (0.1ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_type" = ? AND "treasures"."looter_id" IN (?, ?)  [["looter_type", "ShipPart"], ["looter_id", 1], ["looter_id", 2]]
D, [2022-05-05T16:18:46.442134 #3280] DEBUG -- : Dynamically preloaded: Developer.ship.parts.trinkets
D, [2022-05-05T16:18:46.442986 #3280] DEBUG -- :   Ship Load (0.1ms)  SELECT "ships".* FROM "ships" WHERE "ships"."developer_id" = ? LIMIT ?  [["developer_id", 11], ["LIMIT", 1]]
D, [2022-05-05T16:18:46.443639 #3280] DEBUG -- :   ShipPart Load (0.1ms)  SELECT "ship_parts".* FROM "ship_parts" WHERE "ship_parts"."ship_id" = ?  [["ship_id", 751016585]]
D, [2022-05-05T16:18:46.445137 #3280] DEBUG -- :   Treasure Load (0.2ms)  SELECT "treasures".* FROM "treasures" WHERE "treasures"."looter_type" = ? AND "treasures"."looter_id" IN (?, ?)  [["looter_type", "ShipPart"], ["looter_id", 3], ["looter_id", 4]]
D, [2022-05-05T16:18:46.445454 #3280] DEBUG -- : Dynamically preloaded: Developer.ship.parts.trinkets

[OuYangJinTing]

This is a good feature. If it be merged main, I will try to use it instead of goldiloader gem. But this seems to break ActiveRecord::Associations::Preloader::Branch feature, maybe should to leave a note in guides.