redis-om-node icon indicating copy to clipboard operation
redis-om-node copied to clipboard

Add support for Schema type array of objects

Open richardg2o opened this issue 3 years ago • 3 comments

Currently the Schema only supports the following types: number, string, boolean, string[] Redis also supports keys having an array of objects.

However, redis-om doesn't work with that, it stores it inside redis as shown in the image below.

image

richardg2o avatar Jan 21 '22 13:01 richardg2o

Its actualy pretty easy to fix.

In schema.js, just remove typechecking, redis already knows what type it is and stores it accordingly.

Schema.js:53
------------
if (isArray) {
     this.entityData[fieldAlias] = value.map((v) => v.toString());
} else {
    this.entityData[fieldAlias] = value;
}

Above code can simply be replated with

this.entityData[fieldAlias] = value;

In redis it works perfectly now image

richardg2o avatar Jan 21 '22 13:01 richardg2o

Creating more deeply nested objects is something I would like to add to Redis OM as right now the implementation is flat. However, doing this is not as simple as one might think. Redis OM has to work with both Hashes and JSON documents. Hashes are flat, JSON is not. Redis OM has to work with RediSearch—adding indices for fields within nested objects in JSON. Doing this with RediSearch requires adding JSONPath queries and RediSearch has some limitations on the types of nesting it can handle. What about sub-objects within arrays of sub-objects? The complexity quickly spirals and creating a robust solution, while doable, requires a fair bit of design and premeditation. This is why I didn't implement this feature for the preview. It will probably be a breaking change.

guyroyse avatar Jan 24 '22 19:01 guyroyse

Can't you simply check if user has set dataStructure: 'JSON'? And then in schema.js handle the two based on their structure?

richardg2o avatar Jan 25 '22 09:01 richardg2o

This was fixed with the release of Redis OM 0.4.0 Beta. Install it with npm install redis-om@beta.

guyroyse avatar Mar 20 '23 18:03 guyroyse

@guyroyse would you mind providing an example of how to use this? Most notably, I'd like to store an array of objects for which I don't know the shape. Also, I'd like this to be possible at any depth. 🤔

humblecoder avatar Jul 13 '23 06:07 humblecoder

The new version does not require that you specify all things in a Schema. So, you can save whatever you want. You can index fields within an array of objects but those fields can only be TAG fields behind the scenes, which means in Redis OM that they can only be of type string[]. I plan to add number[] and boolean[] in the future near.

For example, say you had the following JSON:

{
  "artist": "Mushroomhead",
  "genres": [ "metal" ],
  "albums": [
    {
      "title": "The Righteous & The Butterfly",
      "year": 2014
    },
    {
      "title": "A Wonderful Life",
      "year": 2020
    },
    {
      "title": "Beautiful Stories for Ugly Children",
      "year": 2010
    }
  ]
}

You could create a Schema like this:

const schema = new Schema('artist', {
  artist: { type: 'string', path: '$.artist' },
  genres: { type: 'string[]', path: '$.genres[*]` },
  album_titles: { type: 'string[]', path: '$.albums[*].title' },
}

Note that the path is optional for artist and genres as Redis OM will generate it for you. But it is requred to dip down below the root of the JSON object and so if required for the albums.

guyroyse avatar Jul 13 '23 15:07 guyroyse

The new version does not require that you specify all things in a Schema

Understood, but in the event that I simply wanted to specify the path for an array of "unknown shaped" objects, how would I? For example, if I simply specify album_titles: { type: 'string[]', path: '$.albums[*]' } I'm unable to send random objects. I receive the following error:

Unexpected value for field ... of type 'string[]' in JSON at '$.album[*]'.

Am I required to "punch through" with at least one potential field value so that any other fields are then accepted?

I'm just looking to distinguish between "edge case" and "impossible" here. 🫠

humblecoder avatar Aug 10 '23 19:08 humblecoder