earthdata-search
earthdata-search copied to clipboard
EDSC-3932: Integrate edsc-visualize as a lambda into EDSC remove calls to the browse-scaler API
Overview
What is the feature?
We want to remove the application browse-scaler
from CMR and achieve feature parity with that API as a lambda within EDSC. EDSC is the only client to that API and infrastructure, maintenance, and over complication of that API's artifacts can be significantly reduced this way.
What is the Solution?
We pulled in an existing application which was never deployed edsc-visaulize
(https://git.earthdata.nasa.gov/projects/EDSC/repos/edsc-visualize/browse/src/scaleImage) updated it to be able to integrate with EDSC and updated remaining EDSC modules to utilize this API in the from of a new AWS lambda. This also utilizes the infrastructure that browse-scaler previously used (a redis cache deployed as an elasticache) to cache images found to improve performance. Additionally due to limitations around API-gateway instances not allowing for binary media to be returned unless specific Accept
headers get passed image/png
in our case. Because we cannot predict the order or the type of Accept header that a browser is going to sent via the src
prop of an img
element. We must request the image ourselves while supplying that Accept header. As a result we needed to add modules and refactor state of the EDSCImage
component so that we could ensure that thumbnails are rendered correctly. One additional point of contention was around the granule browse imagery cells on the granule table view. That component is wrapping around the react-table
NPM component. Either that component (the NPM) or our implementation of the component has issues where table cells and columns are re-rendering often (this also occurs in prod but, is masked by browser-caching). We are significantly behind in versions and there was evidence that others experienced similar issues on their issues page. Memoization on those cells was not successful which appears to be because of the react-table
package. The v7
that we are using has reported issues where memoization was not possible for individual cells or columns. Due to this and the addition of the imagery modal popover which was added last year, and replaces much of that functionality, as well as the image size being small such that its use to discern granules is limited, we have elected to hide the cell for now and either delete it or restore it pending an update to the table component for its updated version.
What areas of the application does this impact?
Serverless, cloud-formation, adds a new lambda, and alters collection/granule request modules
Testing
Reproduction steps
- Any Environment but, Prod has by far the greatest variety and number of browsable-imagry
- Collection to test with:
-
From the top level
/search
page we can see the thumbnails for the various collections and these should fill in as we scroll down for collections which have images. The rest should get the no image found default image instead. -
Click into a collection with a thumbnail to see the granule images once the granule images load we can scroll down to browse all of the granule images within a collection
-
Click on a specific granule i.e. make it a focused granule to get the image viewer modal to pop up. For granules with multiple images we can see all the images on that specific granule.
-
View the cache keys to ensure that values are in fact getting cached. To do so locally you need to enable the
useCache
feature toggle and utilize some redis cache either the service itself or through a docker container
Attachments
Please include relevant screenshots or files that would be helpful in reviewing and verifying this change.
Checklist
- [x] I have added automated tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have updated other documentation pertaining to EDSC architecture to include the
redis
cache
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 92.00%. Comparing base (
16b266d
) to head (e5f8c47
). Report is 1 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #1716 +/- ##
==========================================
+ Coverage 91.95% 92.00% +0.04%
==========================================
Files 725 739 +14
Lines 19383 19638 +255
Branches 4585 4653 +68
==========================================
+ Hits 17824 18067 +243
- Misses 1423 1434 +11
- Partials 136 137 +1
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
This PR is still ongoing; the issue is that during some feature branch testing we came to find a dependency we have with our AWS infrastructure which requires NGAP to configure an additional API_GWY layer outside of our control to allow binary media types in the lambda responses. See ticket comments for accompanying NASD
ticket
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 84.13%. Comparing base (
b796613
) to head (4c6e09f
).
Additional details and impacted files
@@ Coverage Diff @@
## main #1716 +/- ##
==========================================
- Coverage 91.93% 84.13% -7.80%
==========================================
Files 730 744 +14
Lines 19568 17872 -1696
Branches 4647 4599 -48
==========================================
- Hits 17989 15036 -2953
- Misses 1441 2442 +1001
- Partials 138 394 +256
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.