marquez
marquez copied to clipboard
WIP: Elasticsearch indexing and queries
Very much a WIP
Problem
Our search right now does not enable nested queries on OpenLineage facets, code, linked entities, and ids. We want to enable our search to be the very best place to absorb and catalog OpenLineage based data.
Checklist
- [ ] You've signed-off your work
- [ ] Your changes are accompanied by tests (if relevant)
- [ ] Your change contains a small diff and is self-contained
- [ ] You've updated any relevant documentation (if relevant)
- [ ] You've included a one-line summary of your change for the
CHANGELOG.md(Depending on the change, this may not be necessary). - [ ] You've versioned your
.sqldatabase schema migration according to Flyway's naming convention (if relevant) - [ ] You've included a header in any source code files (if relevant)
Deploy Preview for peppy-sprite-186812 canceled.
| Name | Link |
|---|---|
| Latest commit | d0961387cb09577936d80aeabb1295d4b4a370e1 |
| Latest deploy log | https://app.netlify.com/sites/peppy-sprite-186812/deploys/66bfdfaf894d82000853712d |
Just a suggestion as I did not explore this too deeply. Wouldn't PSQL's text search be enough? I've seen some comparisons, e.g. here and it seems to be pretty viable option.
Codecov Report
Attention: Patch coverage is 22.36025% with 125 lines in your changes missing coverage. Please review.
Project coverage is 83.28%. Comparing base (
422fd43) to head (d096138). Report is 1 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #2834 +/- ##
============================================
- Coverage 84.77% 83.28% -1.49%
- Complexity 1470 1477 +7
============================================
Files 256 259 +3
Lines 6626 6785 +159
Branches 308 313 +5
============================================
+ Hits 5617 5651 +34
- Misses 856 977 +121
- Partials 153 157 +4
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.