Support Master/Slave & multiple databases

[jackbit]

Hi Bro,

I would like to ask, is mebla support master / slave database from current mongoid.yml configuration ? how i can switch searching in mebla to my slaves or other databases in reading data in the same application? what will mebla sync the updates from databases?

Many Thanks, Yacobus

[jackbit]

sorry for asking again, what is whiny_indexing for in your specs/fixtures/model.rb ?

thanks, yacobus

[cousine]

Hi Yacobus,

I haven't tested mebla with master/slave configuration, but it should theoretically work since it works on mongoid's layer not the database directly.

As for whiny_indexing, if this is set to true mebla will throw an error if indexing fails.

Regards, Omar

[jackbit]

Hi thanks for your answer, I have several issue with mebla:

1. I have 16 millions records in mongDB, when index data, it takes ages and not completed because get KILL signal from server. If i used replication database, will membla index both data in master and slaves? is it good if i dont auto index data?
2. when the index data not complete but models are indexed, i got null results from mebla. what's happen? elasticsearch service is running. I tried to use regex in query like ModeName.search("keyword: /DELL/i"). not working too.
3. I dont see options to switch searching to slave database, in mongoid, we can do like ModelName.where("blabla").enslave, i think mebla will connect to master not slave, since the slave parameter is not enabled in query. I dont want put enslave global in model. If i put ModelName.search("foo: not doe").enslave, it gives me error.

Thanks, yacobus

[cousine]

Hi,

Sorry for the late answer.

1. I am working on that issue, it has to do mainly with how elasticsearch does bulk indexing so I am trying to find a way around it, as for replication, it won't matter at this stage, since mebla just loops over the records and extracts the indexing data from the model object.
2. If the data indexing is not complete this means that existing records in the database are not in elasticsearch's index, but if you add a new record it will be added to the index.
3. I will look into that in the next few days and investigate what will happen if master/slave is used in mongoid, most probably this will need some changes in the API but I'll try to make it non-obtrusive as much as possible, about the enslave method it won't work since #search returns mebla::result_set object

[jackbit]

Hey,

Thanks for your advance actions, mebla will be rocket, since many rails developer use mongoid nowdays. we are impatient to see next version. once again, thanks millions.

yacobus

[jackbit]

once again, it would be supper great if there an option in rake to index model and data by providing options which database that you want to use for elasticsearch. in my research, many people use replication for super huges records and use slave for searching/reading and leave master alone in writing. It will give benefit : when indexing to slave only, it will not extract records from master and not annoy its performance.

You are rocking bro :)