MongoDB - Multiple Collection Filters

[malcyL]

Feature Request

Currently, you can specify a CollectionFilter for a MongoDB source like this:

var source = mongodb({
  "uri": "mongodb://localhost:32769/local_database",
  "collection_filters": "{ \"collectionname\": {\"property\": \"test.68817\"}}",
})

The string value for the collection filter doesn't work for more complex filtering. For example, the following causes a json unmarshalling exception:

  "collection_filters": "{ \"collectionname\": "{\"$or\": [
      {\"property\":\"test.68817\"},
      {\"property\":\"test.88217\"}
  }]}"

It would be great if transporter supported more complex MongoDB filters within the CollectionFilter.

Proposal:

Alternatively, allowing the filtering of documents which match more than one value of a property might be a simpler step to start with.

I have forked transporter and created a branch which allows the following syntax for a collection filter:

"collection_filters": "{ \"collectionname\": [{\"property\": \"test.68817\"}, {\"property\":\"test.88217\"}]}",

The ability to transport documents matching multiple filters using the tail option to keep two databases in sync is important for us to be able to migrate data in a live system without taking the system down.

The fork and PR with this change are here: https://github.com/talis/transporter/pull/1

Do you have plans to add this feature? Would you accept a PR making the change I have spiked in our fork? Have I missed something - is it already possible to achieve what I'm doing without a code change?

[sashaaKr]

Is there any plans to add this to master? Encountered with the same problem, and it seems like very useful feature.