monstache icon indicating copy to clipboard operation
monstache copied to clipboard

How to sync data based on perticular time period

Open BASHEER786 opened this issue 4 years ago • 3 comments

Hi,

Required urgent suggestion. Can you please suggest how to filter collection of documents to sync from mongo to Elasticsearch based on time period(like last six months documents) using created date filed created in our document.

Thanks, Basheer.

BASHEER786 avatar May 21 '21 12:05 BASHEER786

@BASHEER786 I think the easier way to do that is to create a view in MongoDB that only returns documents where created date if greater than a date 6 months ago. Then reference the name of that view instead of the collection in the direct-read-namespaces.

direct-read-namespaces = [ "six_months_ago_view" ]

rwynn avatar May 23 '21 03:05 rwynn

@rwynn Thank you for your quick response will it work for real time production. we need to sync data from mongo to Elasticsearch and then we need to watch each and every event using change streams.

BASHEER786 avatar May 24 '21 05:05 BASHEER786

Hi Rwynn,

Thank you for your quick response. if we plan to move with monstache to sync data from mongodb to elasticsearch in real time production do we get any production support from your side.waiting for your response.

Thanks & Regards, Basheer.

On Sun, May 23, 2021 at 8:57 AM Ryan Wynn @.***> wrote:

@BASHEER786 https://github.com/BASHEER786 I think the easier way to do that is to create a view in MongoDB that only returns documents where created date if greater than a date 6 months ago. Then reference the name of that view instead of the collection in the direct-read-namespaces.

direct-read-namespaces = [ "six_months_ago_view" ]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rwynn/monstache/issues/516#issuecomment-846495583, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ32WI76SE5A2JUXXG52IB3TPBYYLANCNFSM45JB3H4Q .

BASHEER786 avatar May 24 '21 06:05 BASHEER786