Add embedded associations in ActiveModel

[mansakondo]

Relational databases are very powerful. Their power comes from their ability to...

• Preserve data integrity with a predefined schema.
• Make complex relationships through joins.

But sometimes, we can stumble accross data that don't fit in the relational model. We call this kind of data: semi-structured data. When this happens, the things that makes relational databases powerful are the things that gets in our way, and complicate our model instead of simplifying it.

That's why document databases exist, to model and store semi structured data. However, if we choose to use a document database, we'll loose all the power of using a relational database.

Luckily for us, relational databases like Postgres and MySQL now has good JSON support. So most of us won't need to use a document database like MongoDB, as it would be overkill. Most of the time, we only need to denormalize some parts of our model. So it makes more sense to use simple JSON columns for those, instead of going all-in, and dump your beloved relational database for MongoDB.

Currently in Rails, we can have full control over how our JSON data is stored and retrieved from the database, by using the Attributes API to serialize and deserialize our data.

That's what this extension does, in order to provide a convenient way to model semi-structured data in a Rails application.

Usage

Let's say that we need to store books in our database. We might want to "embed" data such as parts, chapters and sections without creating additional tables. By doing so, we can retrieve all the embedded data of a book in a single read operation, instead of performing expensive multi-table joins.

We can then model our data this way:

class Book < ApplicationRecord
  include ActiveModel::Embedding::Associations

  embeds_many :parts
end

class Book::Part
  include ActiveModel::Embedding::Document

  attribute :title, :string

  embeds_many :chapters
end

class Book::Part::Chapter
  include ActiveModel::Embedding::Document

  attribute :title, :string

  embeds_many :sections
end

class Book::Part::Chapter::Section
  include ActiveModel::Embedding::Document

  attribute :title, :string
  attribute :content, :string
end

And display it like this (with nested attributes support out-of-the-box):

# app/views/books/_form.html.erb
<%= form_with model: @book do |book_form| %>
  <%= book_form.fields_for :parts do |part_fields| %>

    <%= part_fields.label :title %>
    <%= part_fields.text_field :title %>

    <%= part_fields.fields_for :chapters do |chapter_fields| %>
      <%= chapter_fields.label :title %>
      <%= chapter_fields.text_field :title %>

      <%= chapter_fields.fields_for :sections do |section_fields| %>
        <%= section_fields.label :title %>
        <%= section_fields.text_field :title %>
        <%= section_fields.text_area :content %>
      <% end %>
    <% end %>
  <% end %>

  <%= book_form.submit %>
<% end %>

Validations

class SomeModel < ApplicationRecord
  include ActiveModel::Embedding::Associations

  embeds_many :things

  validates_associated :things
end

class Thing
  include ActiveModel::Embedding::Document

  embeds_many :other_things

  validates_associated :other_things
end

class OtherThing
  include ActiveModel::Embedding::Document

  attribute :some_attribute, :string

  validates :some_attribute, presence: true
end

things = Array.new(3) { Thing.new(other_things: Array.new(3) { OtherThing.new } }
record = SomeModel.new things: things

record.valid? # => false
record.save # => false

record.things.other_things = Array.new(3) { OtherThing.new(some_attribute: "present") }

record.valid? # => true
record.save # => true

Custom collections

class SomeCollection
  include ActiveModel::Embedding::Collecting
end

class Thing
  # ...
end

class SomeModel
  include ActiveModel::Embedding::Document

  embeds_many :things, collection: "SomeCollection"
end

some_model = SomeModel.new things: Array.new(3) { Thing.new }
some_model.things.class
# => SomeCollection

Custom types

# config/initializers/types.rb
class SomeType < ActiveModel::Type::Value
  def cast(value)
    value.cast_type = self.class
    super
  end
end

ActiveModel::Type.register(:some_type, SomeType)

class SomeOtherType < ActiveModel::Type::Value
  attr_reader :context

  def initialize(context:)
    @context = context
  end

  def cast(value)
    value.cast_type = self.class
    value.context = context
    super
  end
end

class Thing
  attr_accessor :cast_type
  attr_accessor :context
end

class SomeModel
  include ActiveModel::Embedding::Document

  embeds_many :things, cast_type: :some_type
  embeds_many :other_things, cast_type: SomeOtherType.new(context: self)
end

some_model = SomeModel.new(
  things: Array.new(3) { Thing.new },
  other_things: Array.new(3) { Thing.new }
)

some_model.things.first.cast_type
# => SomeType
some_model.other_things.first.cast_type
# => SomeOtherType
some_model.other_things.first.context
# => SomeModel

Associations

embeds_many

Maps a JSON array to a collection.

Options:

• :class_name: Specify the class of the documents in the collection. Inferred by default.
• :collection: Specify a custom collection class which includes ActiveModel:: Embedding::Collecting (ActiveModel::Embedding::Collection by default).
• :cast_type: Specify a custom type that should be used to cast the documents in the collection. (the :class_name is ignored if this option is present.)

embed_one

Maps a JSON object to a document.

Options:

• :class_name: Same as above.
• :cast_type: Same as above.

:warning: Warning

Embedded associations should only be used if you're sure that the data you want to embed is encapsulated. Which means, that embedded associations should only be accessed through the parent, and not from the outside. Thus, this should only be used if performing joins isn't a viable option.

Read this section from the original README (and this article) for more insights on the use cases of this feature.

Concepts

Document

A JSON object mapped to a PORO which includes ActiveModel::Embedding::Document. Usually part of a collection.

Collection

A JSON array mapped to an ActiveModel::Embedding::Collection (or any class that includes ActiveModel::Embedding::Collecting). Stores collections of documents.

Embedded associations

Models structural hierarchies in semi-structured data, by "embedding" the content of children directly in the parent, instead of using references like foreign keys. See Embedded Data Models from MongoDB's docs.

Semi-structured data

Data that don't fit in the relational model.

Semi-structured data is a form of structured data that does not obey the tabular structure of data models associated with relational databases or other forms of data tables, but nonetheless contains tags or other markers to separate semantic elements and enforce hierarchies of records and fields within the data. Therefore, it is also known as self-describing structure. - Wikipedia

[rafaelfranca]

Thank you so much for the pull request and all the work you put on this. I have someone in my team working in the same idea, but with a very different API and implementation, so it is very likely that I'll not merge this PR.

I'll keep it open though, in case our implementation end up not working as we like.

I'll update this PR with more information.

Thank you again.

[mansakondo]

Ok thanks for the reply.

[rafaelfranca]

I think there are more feature here than what I was planning to implement like embedding in JSON columns. As we have the API we want I'll probaly ask here if you are interested in porting this implementation to use the new API.

[mansakondo]

Yeah sure with pleasure.

[rails-bot[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mansakondo]

Hi @rafaelfranca. Since this PR was closed, I wanted to know if you and your team are still working on a way to model semi-structured data in Rails ?

[rafaelfranca]

Yes! The PRs are open right now #44380 #44324

[mansakondo]

Ok thanks for letting me know.