dynamodb-replicator
dynamodb-replicator copied to clipboard
mapbox
module for dynamodb multi-region replication
There was an authentication error when trying to run a backup using the `eu-central-1` region. According to [Stackoverflow](http://stackoverflow.com/a/26538266/518801) the default mechanism (`v2`) was never available in `eu-central-1`. Setting the authentication...
Nodejs 4.3 onwards streambot is deprecated and environment variables can be set directly. I have removed the streambot dependency from index.js and added extra logging to note the changes that...
As reported in #90 there is a bug with key ordering when computing S3 keys. The bug reporter submitted a PR #91 which was never accepted. Maybe the use of...
enable S3 ServerSideEncryption for all writes. S3 doesn't charge any extra for using encryption and there's no reason not to. AES256 using the S3 managed keys are used by default...
... so that it doesn't need to be named 'mapbox'. The bucket named by the ENV variable BackupBucket will be used. BackupBucket is set to 'mapbox' in test/env.js which is...
Some features I needed for my project - Would love to hear your thoughts and if you are interested in these features?
Hi there! First off, great library. It's super useful and a much better/simpler option (for me) than the whole EMR/Datapipeline situation. I have this simple lambda function that is subscribed...
`bin/incremental-snapshot.js s3://$BackupBucket/$BackupPrefix/$TABLE s3://$BackupBucket/${TABLE}-snapshot` sometimes this leaves empty lines in the snapshot output - for example: `aws s3 cp s3://$BackupBucket/${TABLE}-snapshot - | gzcat ` ```{"what":{"S":"bb"},"a":{"S":"bb"}} {"what":{"S":"new1"},"a":{"S":"b"}} {"what":{"S":"new2"},"a":{"S":"11"}} {"what":{"S":"a"},"b":{"S":"ccd"}} {"what":{"S":"test2"},"a":{"S":"asdf"}} {"what":{"S":"new10"},"a":{"S":"b"}} {"what":{"S":"sdfg"},"a":{"S":"asdf"}}...
`incremental-snapshot.js` doesnt seem to handle s3 timeouts very well - leaving a broken (partial, missing, or otherwise) snapshot in it's wake: ```bin/incremental-snapshot.js s3://$BackupBucket/$BackupPrefix/$TABLE s3://$BackupBucket/${TABLE}-snapshot``` ```create snapshot from bucket [Tue, 10...
simple test case with a small table: ``` ENV=dev TABLE=dsrtest2 . config.env.$ENV bin/incremental-backfill.js $AWS_REGION/$TABLE s3://$BackupBucket/$BackupPrefix bin/incremental-snapshot.js s3://$BackupBucket/$BackupPrefix/$TABLE s3://$BackupBucket/${TABLE}-snapshot s3print s3://$BackupBucket/${TABLE}-snapshot | dyno put $AWS_REGION/dsr-test-restore-$TABLE ``` ``` %% sh test-backup.sh 12...