rosettasciio icon indicating copy to clipboard operation
rosettasciio copied to clipboard

Custom Scan .mrc

Open CSSFrancis opened this issue 11 months ago • 1 comments

Description of the change

Direct Electron allows for custom scan patterns/ non-linear distributed or serpentine scan patterns to be loaded. This is a nice feature, but sometimes it can be difficult to account for when loading data. Ultimately, the data is still locked to a grid, but no every pixel has to be scanned and the data is often not in the correct order.

Along with each .mrc file a scan_coordinates.csv file is also saved which is simply two columns with x, y scan positions. This file is likely to be updated in future with additional columns (time stamp, in situ information etc) but for now it should be relatively static.

Loading this type of data is a little tricky, but the core concept is that a fully dense array is created. If a scan position isn't scanned, it is set to all zeros and should be easy to ignore, mask etc. and it should compress down to almost nothing. Otherwise I could set those missing arrays to all nan but that would require casting from int to float which I would rather not do.

Progress of the PR

  • [x] Add loading custom scans
  • [ ] Zeros where no scan is
  • [ ] update docstring (if appropriate),
  • [ ] update user guide (if appropriate),
  • [ ] add a changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
  • [ ] Check formatting of the changelog entry (and eventual user guide changes) in the docs/readthedocs.org:rosettasciio build of this PR (link in github checks)
  • [ ] add tests,
  • [ ] ready for review.

Minimal example of the bug fix or the new feature

s = hs.load("test_scan.mrc", scan_file="scan_coordinates.csv")

CSSFrancis avatar Jan 16 '25 17:01 CSSFrancis

Codecov Report

:x: Patch coverage is 90.78947% with 7 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 87.88%. Comparing base (1ef8be0) to head (d027972). :warning: Report is 134 commits behind head on main.

Files with missing lines Patch % Lines
rsciio/mrc/_api.py 91.30% 3 Missing and 1 partial :warning:
rsciio/utils/distributed.py 90.00% 1 Missing and 2 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #356      +/-   ##
==========================================
+ Coverage   87.86%   87.88%   +0.02%     
==========================================
  Files          87       87              
  Lines       11335    11393      +58     
  Branches     2095     2109      +14     
==========================================
+ Hits         9959    10013      +54     
- Misses        872      873       +1     
- Partials      504      507       +3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jan 16 '25 19:01 codecov[bot]

pre-commit.ci autofix

CSSFrancis avatar May 20 '25 14:05 CSSFrancis

pre-commit.ci autofix

CSSFrancis avatar May 21 '25 01:05 CSSFrancis

pre-commit.ci autofix

CSSFrancis avatar May 21 '25 14:05 CSSFrancis