Fake collections

[Grafikart]

I'm encountering a problem with a deep relation and I was wondering if it could be solved within neo4jrb scope.

Sometimes I have specific relations like this

(:Tutorial)<-[:TEACH|:USE]<-(:Technology)
(:Tutorial)<-[:INCLUDE]-(:Chapter)<-[:INCLUDE]-(:serie)

I was wondering if it was possible to declare a fake relation in the model

has_many :virtual, :technologies
has_one :virtual, :formation

Then we would be able to collect with a specific alias

    self
      .query_as(:t)
      .with(:t)
      .optional_match('rel = (t)-[r:TEACH|:REQUIRE]->(tech:Technology)')
      .with(:t, :tech, :r)
      .optional_match('(t)<-[:INCLUDE*]-(f:Formation)')
      .return('t, collect(tech) as technologies_collection, collect(r) as requirements, f as formation_collection')

The model would create automatically the accessors for the subcollections.

t # would return the Tutorial
t.technologies # Would return Technology collection
t.formation # Would return the Formation

[cheerfulstoic]

I think that there are a lot of ways to go about this ;)

Firstly, you should know about the rel_length options (documented here)

as(:t).branch { technologies(:tech, :r) }.formation(:f, nil, rel_length: :any)
      .pluck(:t,
      technologies_collection: 'collect(tech)',
      requirements: 'collect(r)',
      formation_collection: 'f')

But, of course, you would need to define the associations. The formations association should be pretty easy:

has_one :in, :formation, type: :INCLUDE

For technologies, it might be tricker because of the multiple relationships. I don't remember if we ever implemented this:

has_many :out, :technologies, type: [:TEACH, :REQUIRE]

If that doesn't work you can probably hack it like this:

has_many :out, :technologies, type: "TEACH|:REQUIRE"

But maybe you're also wanting something higher level so that you don't need to do the collects. For the technologies association you'd probably be fine because the gem auto-loads associations (though you can do it all with one query using with_associations), but the variable length relationship part won't, I don't think, play well with auto-loading because variable-length relationships are defined on query, not on the association.

[jorroll]

Not 100% sure I understand the question, but to add to cheerfulstoic's response:

You can create a scopes on your Tutorial model such as

scope :formation, -> do
  query_as(:tutorial).
  match("(tutorial)<-[:INCLUDE]-(:Chapter)<-[:INCLUDE]-(serie)").
  proxy_as(Formation, :serie)
end

or

scope :technology, -> do
  query_as(:tutorial).
  match("(tutorial)<-[:TEACH|:USE]-(tech:Technology)").
  proxy_as(Technology, :tech)
end

And you could use it like Tutorial.technology or Tutorial.new.technology. This also supports chaining like Tutorial.new.technology.something_else. But something I don't think you can do is

tutorial = Tutorial.new
technology = Technology.new
tutorial.technology << technology

Also, it looks like your query could be cleaned up slightly like:

 self
      .query_as(:t)
      .optional_match('(t)-[r:TEACH|:REQUIRE]->(tech:Technology)')
      .break
      .optional_match('(t)<-[:INCLUDE*]-(f:Formation)')
      .return('t, collect(tech) as technologies_collection, collect(r) as requirements, f as formation_collection')

[jorroll]

@cheerfulstoic would has_many :out, :technologies, type: "TEACH|:REQUIRE" break assignment? (i.e. model.technologies << technology)

[cheerfulstoic]

@thefliik Yeah, good point. Though even if the array syntax was supported, I don't know what we would do for assignment. It might have to default to the first type specified in the array... That alone makes me think that we didn't implement this feature.

Regarding scopes, it's a great point. I would also mention that you can use class methods as well like:

def self.technology
  all.query_as(:tutorial).
  match("(tutorial)<-[:TEACH|:USE]-(tech:Technology)").
  proxy_as(Technology, :tech)
end

[Grafikart]

The assignment is not a problem since I handle relation creation with an ActiveRel (so it's not a problem if we loose the ability to do <<). The problem with scope is that I end up returning a collection of Technology or Formation. The goal was to go around the autoloading, managing the loading of relation myself (adding some specificities on the relations).

A class method is my current solution, I get everything I want with one query and hydrate records accordingly (adding attr_accessor :technologies, :formation). The goal is to get as much as possible with one query.

  # This could be optimised but I didn't knew about break() at the time
  scope :listing, -> (limit) {
    self
      .query_as(:t)
      .with(:t)
      .optional_match('rel = (t)-[r:TEACH|:REQUIRE]->(tech:Technology)')
      .with(:t, :tech, :r)
      .optional_match('(t)<-[:INCLUDE*]-(f:Formation)')
      .return('t, collect(tech) as technologies, collect(r) as requirements, f as formation')
  }

The problem can be even more complexe if we want to add conditions on relation

 self
      .query_as(:t)
      .optional_match('(t)-[r:TEACH|:REQUIRE]->(tech:Technology)')
      .where('r.version > 2')
      .break
      .optional_match('(t)<-[:INCLUDE*]-(f:Formation)')
      .return('t, collect(tech) as technologies_collection, collect(r) as requirements, f as formation_collection')

But maybe my use case is too specific, and I should keep doing the hydratation myself.

[jorroll]

So you're basically trying to load a model with_associations() but you're finding that the with_associations() API is too limiting? Or are you not familiar with the with_associations() method? (It looks like your queries are too complex for with_associations() anyway, which is something I often run into myself.)

So it's still unclear to me how has_many :virtual, :technologies would improve things for you? Wouldn't that be functionally equivalent to adding the :technologies accessor? I guess I can think of one difference: that you could still chain with the virtual association. Is that the only limitation you're running into with your current method?

[Grafikart]

@thefliik I'm aware of with_associations() but the API is indeed too limiting for complex queries where you would want to choose how to hydrate a relation. The has_many is a way to tell the model to look for a field "XXXX_collection" when returning things. But I think I am doing things the wrong way and I shouldn't rely too much on an ORM for these specific use cases where I return collections of data.

Feel free to close this issue if you think the same

[cheerfulstoic]

Yeah, the underlying Query / QueryProxy stuff is there so that you can do more complex stuff when the level APIs break down. But I could also see being able to create associations which are more complex which would then work with with_associations. Happy to leave the issue around in case somebody wants to take a stab at it.

[Grafikart]

I'll try to implement the first thing I need. (multiple labels + rel_length)

has_many :out, :technologies, type: [:USE, :TEACH]
has_one :in, :formation, type: :INCLUDE, rel_length: 2

is there a guide to setup the test environment ? (I'm familiar with rspec but don't know how should I prepare my neo4j database.

[jorroll]

I think the contributing file explains it all. If I remember correctly, it's not really any different than setting up a rails app (which might not be a helpful analogy, if you don't use rails).

This being said, I think the rel_length bit was 99% implemented by #1485. I think all you'd need to do is whitelist rel_length as a new configuration option.

[cheerfulstoic]

:+1: on the CONTRIBUTING.md. Feel free to drop by the Gitter channel if you need help.