project structure and outline

[scottstanie]

Creates the skeleton of the repo, including

• Setup and installation script (pyproject.toml, requirements files)
• workflow subpackage showing how modules will be called in main workflow
• config module for loading the Workflow, containing the configuration
• docs generated from mkdocs and reference pages from mkdocstrings. You can check what the docs currently look like here. They're built from my branch, so once this is merged we'll have them at https://opera-adt.github.io/dolphin/.
• Docker image for running the command line script

Some things to try out if you wanted to play around with it:

• Installation/"development setup" in the README.md/ mkdocs serve to see the documentation locally

After installing with the Getting Started instructions (mamba install -c conda-forge --file requirements.txt pip install -e .[docs] then you should be able to see

• dolphin config: create a workflow configuration file.
• dolphin run : run the workflow using this file.

The run command is mostly empty, outlining what steps will get called in the workflow, but the config results creates the YAML file resulting from config.Workflow.to_yaml.

You can run:

$ dolphin config --slc-files ../../hawaii/test-loading-gtiff-complex64-gzip/t*.tif
$ cat dolphin_config.yaml

workflow_name: stack
inputs:
  cslc_file_list:
  - /u/aurora-r0/staniewi/dev/hawaii/test-loading-gtiff-complex64-gzip/t087_185682_iw2_20180210_VV.slc.tif
  - /u/aurora-r0/staniewi/dev/hawaii/test-loading-gtiff-complex64-gzip/t087_185682_iw2_20180827_VV.slc.tif
  cslc_directory:
  cslc_file_ext: 
  cslc_date_fmt: '%Y%m%d'
  mask_files: []
outputs:
  output_format: NetCDF
  scratch_directory: /u/aurora-r0/staniewi/dev/dolphin-test/scratch
  output_directory: /u/aurora-r0/staniewi/dev/dolphin-test/output
  hdf5_creation_options:
    chunks: true
    compression: gzip
    compression_opts: 4
    shuffle: true
  gtiff_creation_options:
  - TILED=YES
  - COMPRESS=DEFLATE
  - ZLEVEL=5
ps_options:
  directory: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/PS
  output_file: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/PS/ps_pixels.tif
  amp_dispersion_file: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/PS/amp_dispersion.tif
  amp_mean_file: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/PS/amp_mean.tif
  amp_dispersion_threshold: 0.42
phase_linking:
  directory: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/linked_phase
  ministack_size: 15
  compressed_slc_file: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/linked_phase/compressed_slc.tif
  temp_coh_file: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/linked_phase/temp_coh.tif
  half_window:
    x: 11
    y: 5
interferogram_network:
  reference_idx: 0
  max_bandwidth:
  max_temporal_baseline:
  network_type: single-reference
unwrap_options:
  run_unwrap: false
  directory: /u/aurora-r0/staniewi/dev/dolphin-test/scratch/unwrap
  unwrap_method: snaphu
  tiles:
  - 1
  - 1
  init_method: mcf
worker_settings:
  gpu_enabled: true
  gpu_id: 0
  n_workers: 16
  max_ram_gb: 1.0
creation_time_utc: '2022-12-09T21:21:54.638896'
dolphin_version: 0.0.post1.dev34+g2ecd2fa

For @hfattahi , i'll provide other steps to run the full dev-version of the workflow on my fork here

[scottstanie]

Thank you for the very many bug fixes/improvement's @gmgunter ! I think I've got the comments addressed again.

[codecov-commenter]

Codecov Report

:exclamation: No coverage uploaded for pull request base (stable@f491105). Click here to learn what that means. The diff coverage is n/a.

@@            Coverage Diff            @@
##             stable       #2   +/-   ##
=========================================
  Coverage          ?   84.11%           
=========================================
  Files             ?       14           
  Lines             ?      510           
  Branches          ?        0           
=========================================
  Hits              ?      429           
  Misses            ?       81           
  Partials          ?        0           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.