Nested attributes

[warebot]

Does Dynamoid currently support working with nested attributes

I failed to find anything in the documentation/README (or specs) that pertains to that.

[pboling]

I am not sure. I haven't used them myself. @bglusman ?

[bglusman]

Nope, I haven't either but also didn't know dynamo supported it, doesn't look like a brand new feature either, though not sure what a dynamoid api to support that would look like... @warebot can you give an example of how you would like to use this and what your application is? Possibly we can add it or you could add and PR yourself but I doubt it's already supported. I'm on mobile at the moment but can give a quick look through code base later and see if I just don't know about it, both Pete and I are new maintainers, not original authors.

[warebot]

@pboling @bglusman Thank you for your prompt replies. Our current application requires us to persist and query documents that are n-level(s) deep, where n is arbitrary i.e nested attributes/documents.

e.g.

pseudo-code

bob = {
  name: 'Bob',
  phone_numbers: {
    home: 123,
    mobile: 456
  }
} 

=> persist(bob)
=> query(phone_numbers: home: 123) # defines the document path to the attribute we are querying on.
_bob_

So the ORM would probably need to support Hash object fields that are actually persisted as a nested document as opposed to serialized (to allow us to later query those documents)

[bglusman]

I can confirm that Dynamoid does not handle deserializing Maps and Lists stored in Dynamo as such, though it does provide access to them as strings you could manually parse, and if stored correctly you might be able to construct index queries that will find them correctly since that all happens server side, but hard to say without digging into the indexing code. Indexes are a bit strange (at least to me) in Dynamo, with several intertwined concepts of Global and Secondary indexes and projected attributes, so I'm not sure how easily you can construct the specific indexes you'd need for your usecase with or without Dynamoid to allow querying efficiently, without a full table scan, but it would certainly be nice to know and have support for this in Dynamoid. I doubt if Pete or I will have time to add that functionality though, nor do I think it would be wise without having a concrete application to make sure it was a useful end to end feature. If you need any support adding the feature(s) yourself though we'd be happy to try and help when we have time. You agree @pboling ?

[bglusman]

Also meant to add, currently Dynamoid needs support for the map types, which is probably the first thing to change here, but I think it already has support for arrays [lists in dynamo] and sets... here's the code for dump_field types and undump_field types which is basically this list:

:string, :integer, :number, :array, :raw, :set, :datetime, :boolean, :serialized

Possibly adding map and/or hash to that list will make it "just work", since I don't see any special handling to map from these types to the complete list here (which is a little curious since this official gem supports maps but DOES have mapping to "M" etc ... almost wonder if we could/should refactor dynamoid to rely on that for much of it's persistence/indexes logic etc since it's aws maintained and likely to be kept up to date, but that might be a larger project and is certainly a tangent from this question)

For getting the indexing part to work, if it doesn't "just work" (no idea if indexing lists will This article on projection expressions may help though. Spent too long looking into this but hope it helps/curious to know how far from what you need current support is. Also, even though maps/hashes are not supported as a built in type, when put inside an array they work fine and create a nested map correctly, so that may actually work for you.

[pboling]

@bglusman -

wonder if we could/should refactor dynamoid to rely on that for much of it's persistence/indexes logic etc since it's aws maintained and likely to be kept up to date, but that might be a larger project and is certainly a tangent from this question)

I think that is a worthwhile idea, so I created issue for it.

@warebot I definitely don't have time to work on this much, aside from providing what little insight I have. I am new to DynamoDB myself, and using this gem as a crutch, like the rest of us. For my limited use cases of DynamoDB, this gem already does what I need.

DynamoDB is confusing as hell.

[warebot]

@pboling @bglusman

Actually the :raw type looks pretty interesting. I wasn't aware of its existence since it was not listed as a field type in the README. It also looks like undump_field will try to "deserialize" it as a Hash too... 💭

I will try to explore this approach and will definitely update you if I come across any findings/results that are insightful.

Thanks again for looking into this.

[warebot]

@pboling @bglusman

The :raw field was indeed what I was looking for. It was just missing from the frozen version that we were using 1.2.1.

We were able to persist and perform queries on nested attributes.

e.g. User.where(doc: {a: 4})

Thanks again 👍

[bglusman]

@warebot just a heads up, you may well be doing a full table scan when you do that, don't think where can automatically use indexes, even if present/relevant to your query, see #96

[warebot]

@bglusman I appreciate the heads up. Unfortunately, given the arbitrary nature of our documents, this is something we probably won't be able to escape.

[nbulaj]

This looks like what I requested in #608 , but I need something like User.where(settings: { "color.not_null": true }) which doesn't work in my case :(