embulk-input-mongodb icon indicating copy to clipboard operation
embulk-input-mongodb copied to clipboard

Add support for multiple Mongo aggregation pipeline stages

Open emersonf opened this issue 4 years ago • 2 comments

The current implementation only allows you to specify a single Mongo pipeline stage, e.g.

in:
  type: mongodb
  aggregation: >
    { 
      $match: { int32_field: { $gte: 5 }}
    }

But the power of Mongo's aggregation pipeline comes from supporting multiple stages, e.g.

in:
  type: mongodb
  aggregation: >
    [
      { 
        $match: {
          int32_field: { $gte: 1 }
        }
      },
      {
        $sort: {
          int32_field: -1
        }
      }
    ]

This PR adds support for multiple stages, keeping backwards compatibility.

emersonf avatar Jan 14 '21 20:01 emersonf

@dmikurube 👋 I don't know how long it typically takes to review and approve PRs. If it can be approved and released quickly, it would avoid me having to set up a private Gem source.

emersonf avatar Jan 15 '21 09:01 emersonf

@emersonf Thanks for your contribution.

We, as the Embulk team, took over maintenance of this plugin from another individual developer very recently, and we have had little time to learn it so far, unfortunately. Honestly speaking, please expect a certain amount of time to review, merge, and release a new version.

dmikurube avatar Jan 15 '21 12:01 dmikurube