how to add new index to old ones

[hamidrabedi]

So i have these indexes on the first startup:

	if err := r.Repository.CreateIndex(context.Background(), func(schema om.FtCreateSchema) rueidis.Completed {
		return schema.
			FieldName("$.user.owner").As("owner").Tag().
			FieldName("$.user.name").As("name").Tag().
			Build()
	}); err != nil {
		logs.Warning(err)
	}

now I need to add a new index:

	if err := r.Repository.CreateIndex(context.Background(), func(schema om.FtCreateSchema) rueidis.Completed {
		return schema.
			FieldName("$.user.owner").As("owner").Tag().
			FieldName("$.user.name").As("name").Tag().
			FieldName("$.user.id").As("id").Tag().
			Build()
	}); err != nil {
		logs.Warning(err)
	}

But I get an index already exists error. I need my Redis to be persistent and can't flush everything.

[rueian]

Hi @hamidrabedi,

You need to use FT.ALTER on your own currently, just like what CreateIndex does under the hood.

https://github.com/redis/rueidis/blob/5135acdc65c815cdce8e8b0c12c1e5aa939504bd/om/json.go#L139

We may also add an AlterIndex method to the repository interface.

[hamidrabedi]

Yes that would be a nice feature to have, writing raw code or removing and recreating indexes doesn't seem to be a good solution.

[imvtsl]

Hey @rueian Can I have a go at this issue?

Just to clarify few things before I begin working on the fix: 1- We want to add AlterIndex method to this repository. 2- This method would perform the same functionality as FT.ALTER.

Is this a correct understanding of the requirement?

[rueian]

Hey @rueian Can I have a go at this issue?

Just to clarify few things before I begin working on the fix: 1- We want to add AlterIndex method to this repository. 2- This method would perform the same functionality as FT.ALTER.

Is this a correct understanding of the requirement?

Sure, thanks for your interest in this.

1. Add AlterIndex to the type Repository[T any] interface and implement it in both type JSONRepository[T any] struct and type HashRepository[T any] struct. The implementations should be similar to the existing CreateIndex.
2. Yes, the implementations should use the FT.ALTER command by r.client.B().FtAlter().
3. Come up with test cases to show that the implementations work in the hash_test.go and json_test.go files.

[imvtsl]

Thanks for the quick response. I will begin working on this.

[imvtsl]

Hi @rueian Just wanted to give an update. I created above PR and verified that it is working as expected by checking manually using redis-cli. I am working on test cases for the change. Thanks!

[hamidrabedi]

@imvtsl will there be automatic update if index fields were changed ? that would be nice

[rueian]

Hi @hamidrabedi, do you have a feeling about what an automatic update API should look like?

[hamidrabedi]

@rueian, suppose you have a new Redis instance and you call the create index function. If the index has not been created before, it will be created.

Obviously, most of the time there will be updates to your projects, and you might need to remove or add indexes. So instead of using CreateIndex, I want to use something like CreateOrUpdateIndex. This function checks the existing indexes in Redis and ensures that my indexes are present; if not, it adds them.

Regarding deleting an index, I have observed that many migration libraries do not handle deletion for various reasons, so users should handle deleting indexes themselves. However, we could have an option for it—like an argument in the CreateOrUpdateIndex function that, if provided by the user, would indicate that index deletion should happen as well.

This is just a basic usage that users can encounter, right now for this problem all I can do is to call flushall on initialization of my project, or handling the index Updates by connecting to the redis. I can't think of any other usecases at the moment, but I think basic auto update will do the trick for now.

[rueian]

This looks like it requires a set of high-level APIs, probably including our own indexes DSL.

But I am thinking of another approach, how about always creating a new index with FT.CREATE and transparently redirecting application queries to it by using FT. ALIASADD? I think that will be much easier.

[hamidrabedi]

Oh yeah much easier, or like dropping and recreating the index, but I think alias is better and you have lower downtime because the old index won't be affected. can delete the old index afterwards.

[SoulPancake]

Is there still work to be done here? @rueian

[rueian]

Yes, I think we need a new smarter API for creating and aliasing a new index. The API should:

1. Be similar to the existing CreatIndex.
2. Optionally check if a new index is needed.
3. Create a new index.
4. Alias the new index.
5. Drop the old index.

I guess this may require an additional state management so please let me know how would you like to approach this.

[rueian]

Closed by https://github.com/redis/rueidis/pull/813