parseable icon indicating copy to clipboard operation
parseable copied to clipboard

Multiple parseable servers sharing the same S3 backend

Open tweeter0830 opened this issue 1 year ago • 3 comments
trafficstars

Will parseable work if I have multiple instances of the parseable application running independently and ingesting log data, each backed by the same S3 bucket or will I run into some syncing issues?

tweeter0830 avatar Dec 12 '23 16:12 tweeter0830

Thanks for the question @tweeter0830 , we've not yet tested this scenario, but yes, most probably there will be race scenario. Do you mind explaining the use case a bit more? May be we can suggest an alternative solution

nitisht avatar Dec 12 '23 17:12 nitisht

ty for getting back to me so quickly. I have multiple clusters which I'd like to add logging to. My plan was to start a paresable instance per cluster and have that write to the shares S3 locations. The correct way to do this would be to set up one paresable instance local to the shared S3, but that's hard for me given my configuration.

tweeter0830 avatar Dec 12 '23 17:12 tweeter0830

I got the same concern. We're considering Parseable as an alternative to Loki. We've not found information on Parseable HA or load-balanced arch in the doc (or anywhere else BTW). Is there a way for multiple Parseable instances to share the same retention config, RBACs infos, backend storage., .. ? Thanks anyway, it looks like a very promising piece of code.

fromcloudbase avatar Dec 15 '23 10:12 fromcloudbase

This is done with #730 , will be available in the next release v1.0.0. Right now this is under heavy testing. We'll update the docs at the time of release. Closing the issue as done.

nitisht avatar Apr 12 '24 09:04 nitisht

Feel free to reopen if there are doubts or questions.

nitisht avatar Apr 12 '24 09:04 nitisht