bcda-app
bcda-app copied to clipboard
BCDA-7895: Support CCLF file imports from S3
🎫 Ticket
https://jira.cms.gov/browse/BCDA-7895
🛠 Changes
- Update CCLF CLI to support S3 flags
- Encapsulate CCLF import functionality in CclfImporter interface, which can support Local and S3 imports
- Break out shared utilities into
parser.go
- Break down optout s3_file_handler into smaller utility functions for re-use in CCLF importer
ℹ️ Context for reviewers
This PR contains fairly significant changes to the CCLF import code to support importing zip archive files from S3.
In order to enable this functionality, the CCLF code utilizes S3 utilities broken down from the optout module. Since the optout module is the only shared Go module we currently have, this seemed to make the most sense; however, I recognize it feels a bit odd to reuse/wrap the optout module in this way. Ideally the reused S3 functionality would be repackaged in a separate module that both the optout and CCLF code code use.
✅ Acceptance Validation
- [x] Unit testing
- [x] Smoke test in dev for local files
🔒 Security Implications
- [ ] This PR adds a new software dependency or dependencies.
- [ ] This PR modifies or invalidates one or more of our security controls.
- [ ] This PR stores or transmits data that was not stored or transmitted before.
- [ ] This PR requires additional review of its security implications for other reasons.
If any security implications apply, add Jason Ashbaugh (GitHub username: StewGoin) as a reviewer and do not merge this PR without his approval.