aws-serverless-data-lake-framework
aws-serverless-data-lake-framework copied to clipboard
awslabs
Enterprise-grade, production-hardened, serverless data lake on AWS
**Describe the bug** I am trying to follow the tutorial for dataset dependencies at https://github.com/awslabs/aws-serverless-data-lake-framework/blob/master/sdlf-utils/pipeline-examples/dataset-dependency/README.md I can not make this work. Perhaps there is a step missing from the readme?...
*#69* *Correct and clean manifests and cloudfront examples* This pull request addresses some missing steps and correction in two examples of SDLF: * **Deequ and EMR steps using Step Functions**...
I was going through the "manifest" pipeline example (in ./sdlf-utils/pipeline-examples/manifests), and I noticed, one step is mission from installation instructions: - that is about the pipeline project, which is in...
Dear SDLF Team, I recently bumped in SDLF and I love the concept and the architecture. I have 3 items about the "Testing the Framework" section of the **Serverless Data...
*Description of changes:* This change enables sdlf-cicd-team pipeline to create IAM groups so when a new team is provisioned an analysts group is created. Members of this group will be...
*Issue #, if available:* *Description of changes:* By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
*Issue #, if available:* Regions file was not processed because it was a json lines object instead of a json array document. Fixed *Description of changes:* Changed regions.json file By...
**Describe the bug** We appear to use `sqs:SendMessageBatch` in IAM policies [here](https://github.com/awslabs/aws-serverless-data-lake-framework/blob/main/sdlf-team/template.yaml#L462) and [there](https://github.com/awslabs/aws-serverless-data-lake-framework/blob/main/sdlf-stageB/template.yaml#L233) (and other places). It doesn't appear to be a valid permission. Find out when it was...
Will there ever be a Terraform version of this framework? Cloudformation is a non-starter for us. Thanks!
