mongoriver icon indicating copy to clipboard operation
mongoriver copied to clipboard

Support for Mongo driver >= 2.0

Open karthikmuralidharan opened this issue 9 years ago • 4 comments

Please correct me if I am wrong, but does the current gem support Mongo ruby driver v2 and above ? I had to downgrade it to 1.9.2 for it to work.

karthikmuralidharan avatar Nov 28 '15 13:11 karthikmuralidharan

I know this is an old issue, but I was wondering how much work would be needed to update to mongo driver >= 2.0 ? I know there's some breaking changes, and even though I'm not very familiar with Ruby I can try to help.

Any thoughts @den-stripe ?

poislagarde avatar Aug 30 '17 14:08 poislagarde

Hey @poislagarde ! I have not tried to run it with mongo driver >= 2.0. So not sure what is required to make it work.

den-stripe avatar Sep 05 '17 15:09 den-stripe

Mosql/Mongoriver doesn't work well with mongodb 3.6. It often gets below exception at line 170 of tailer.rb file (@cursor.next)

PlanExecutor killed: CappedPositionLost: CollectionScan died due to position in capped collection being deleted. Last seen record id:

So, may you advise how to upgrade mongoriver compatible with new mongodb driver ? P.S: When I rerun the mosql script after its failure, it works again without any data lost, so I tried to add a rescue block as below but it doesn't work:

     begin
        while !@stop && !state.break? && @cursor.has_next?
          state.increment
          record = @cursor.next
          case database_type
          when :mongo
            blk.call(record, state)
          when :toku
            converted = Toku.convert(record, @upstream_conn)
            converted.each do |converted_record|
              blk.call(converted_record, state)
            end
          end
        end
      rescue
        puts "exception cursor.next"
        state.increment
      end

thucnc avatar Jan 01 '18 12:01 thucnc

I'm using ->mosql->mongoriver for mongo to pg oplog streaming, and the above PR is working for me. Still doing more testing, but have streamed many many thousands of records accurately so far. Where I'd be more worried about the above PR is some connection scenarios I have may not have tried, and TokuMX.

johnnason avatar Jan 05 '18 16:01 johnnason