jdaviz icon indicating copy to clipboard operation
jdaviz copied to clipboard

POC: catalog importer/parser

Open kecnry opened this issue 8 months ago • 1 comments

Description

This pull request implements parser support for astropy qtables and importer support for catalogs into the "Catalog Search" plugin, along with user-definable column selection.

This is currently an incomplete proof-of-concept so that we can decide if this is the direction we want to pursue for support for flexible catalog import.

https://github.com/user-attachments/assets/02a9d802-594e-43c5-8ec2-57d40d7c6e36

Once the user clicks "IMPORT" the entry, with the user-defined label, would show as an option in the "Catalog" select dropdown (perhaps with auto-selection?). The user would still need to click search to query that catalog for the current FOV and max sources for plotting.

Migrating this to a future-catalogs implementation with source catalogs as data entries would be very simple, as the label would just need to become data-label and the import method changed from calling the plugin method to adding to the data-collection.

Work that would need to be done to complete an MVP:

  • [ ] fix repopulating options on the importer when switching between two inputs that both support catalogs
  • [ ] update import_catalog to store an entry within the plugin which is then available for querying
  • [ ] validate all inputs (RA and DEC both assigned, valid units, not the same, etc) before allowing to click import
  • [ ] smart automatic selection of RA/DEC when available (otherwise blank)
  • [ ] decide whether importing should change catalog selection in plugin
  • [ ] support for sky_centroid (select for both RA/DEC and parse to split into decimal degrees)
  • [ ] support Table (in addition to QTable), with user assigned units for columns
  • [ ] deprecate existing file import in the dropdown
  • [ ] UI/layout improvements
  • [ ] update documentation
  • [ ] decide whether to deprecate import_catalog method directly and direct users through the loader, or only require it for the advanced use-cases here and keep import_catalog with its strict requirements

Potential future work:

  • [ ] allow renaming columns (during import or after)
  • [ ] changing units on data after imported (i.e. display units for table data)

Change log entry

  • [ ] Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts, list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

  • [ ] Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
  • [ ] Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
  • [ ] Do the proposed changes follow the STScI Style Guides?
  • [ ] Are tests added/updated as required? If so, do they follow the STScI Style Guides?
  • [ ] Are docs added/updated as required? If so, do they follow the STScI Style Guides?
  • [ ] Did the CI pass? If not, are the failures related?
  • [ ] Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone. Bugfix milestone also needs an accompanying backport label.
  • [ ] After merge, any internal documentations need updating (e.g., JIRA, Innerspace)?

kecnry avatar Mar 12 '25 13:03 kecnry

Codecov Report

Attention: Patch coverage is 85.00000% with 18 lines in your changes missing coverage. Please review.

Project coverage is 87.81%. Comparing base (c4034e6) to head (696b426). Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
jdaviz/core/loaders/importers/catalog/catalog.py 81.03% 11 Missing :warning:
jdaviz/core/template_mixin.py 83.33% 6 Missing :warning:
jdaviz/core/loaders/parsers/astropytable.py 94.73% 1 Missing :warning:

:x: Your patch check has failed because the patch coverage (85.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3501      +/-   ##
==========================================
- Coverage   87.82%   87.81%   -0.01%     
==========================================
  Files         156      159       +3     
  Lines       20909    20992      +83     
==========================================
+ Hits        18363    18434      +71     
- Misses       2546     2558      +12     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Mar 12 '25 14:03 codecov[bot]

Superseded by #3751

kecnry avatar Sep 03 '25 12:09 kecnry