es2loki
es2loki copied to clipboard
ktsstudio
Export online
The code only migrate data from ES to Loki, is possible or what is the difficult to let job export online?
Instead finish job, listen the current index and put this doc into loki online.
Great job.
Thank you
Hello! Great question. I think it is possible to add such functionality, but needs some refactoring. But I have a question - in my mind the process of continuous logs forwarding from es to loki is just unnecessary - you can simply spin up a promtail for "new" logs in parallel with filebeat and transfer all the old logs using es2loki up until the date when promtail started transferring logs. It is the case of when if you're migrating from Elasticsearch to Loki for example. What is your case? Why promtail (or any other log collector) seems to be not enough?
Hello, thank you for the reply, yes, in most cases the promtail will be the solution, in my case, the application put the logs directly in ES, and as logstash do not have persistence of position, I can't use it to expose logs from ES to Loki, and promtail can't use ES as input stream, that is my case.
Thank you