aws-serverless-data-lake-framework
aws-serverless-data-lake-framework copied to clipboard
Compatibility install option for SDLFv2
Having SDLFv1 and SDLFv2 side-by-side in the same accounts. [more details to follow]
- [ ] SSM parameters prefixed with
/SDLF2/
instead of/SDLF/
- [ ] KMS aliases starting with
sdlf2-
instead ofsdlf-
- [ ] CodeCommit repositories starting with
sdlf2-
instead ofsdlf-
- [ ] DynamoDB table names using a prefix distinct from
octagon-
(such assdlf2-
) - [ ] import team keys when deploying CloudFormation stacks
#290 helps handling the CodeCommit part of this issue.
#291 helps handling the DynamoDB part - but there is more work required.
#295 and #296 are also linked to this task.
There is a branch with the changes necessary to install SDLFv2 side-by-side with SDLFv1: https://github.com/awslabs/aws-serverless-data-lake-framework/tree/sdlf1-backcompatibility
Some of the changes have been incorporated into main
(see previous comments) but not all are relevant/desirable for all installations of SDLFv2. This is an upgrade path more than a good solution long-term.
Important caveats:
- when deploying foundations, do not use
datalake
as the domain name (we suggestlake
). - when deploying stageA and stageB in SDLFv2, do not use the names
a
andb
(lower-case) forpStageName
.
Other changes that may become part of main
:
removing FunctionName
for the replicate lambda in sdlf-foundations
https://github.com/awslabs/aws-serverless-data-lake-framework/commit/c968be52b3b3be3c4fedc02951ab5c8b719c2704
retrieving the Glue crawler name from SSM in sdlf-stageB fetch-metadata Lambda function: https://github.com/awslabs/aws-serverless-data-lake-framework/commit/612e15454100680edeff42db786d7febb5b03f1e