pecan icon indicating copy to clipboard operation
pecan copied to clipboard

Fix "external pointer is not valid" error and improve output handling in soilgrids_soilC_extract()

Open divine7022 opened this issue 8 months ago • 1 comments

Description

This PR introduces multiple reliability improvements to the soilgrids_soilC_extract() function, focusing on handling edge cases, preventing errors, and ensuring robust execution across diverse geographic locations. These PR address issues, such as failures caused by external pointer is not valid issue and dimension errors during gamma fitting.

Fixes and Improvements:

  1. Fixed "external pointer is not valid" error:
  • Extracted coordinates (terra::crds()) for safe transfer in parallel execution.
  • Recreated SpatVector objects (terra::vect()) inside parallel workers to prevent invalid external pointer errors when working across processes.
  • Added fallback to sequential processing when parallel execution fails.
  1. Addressed Gamma Fitting Failures:
  • Fixed cases where missing or insufficient data caused dimension errors in bestPar[1, ].
  • Added explicit validation to prevent gamma fitting when data contains all NA values.
  • Ensured gamma fitting handles edge cases (e.g., sites with insufficient quantiles).
  1. Critical Validation Checks:
  • Early return implemented when no valid sites remain after preprocessing:
        if (nrow(internal_site_info) == 0) {
          warning("No valid sites with extractable data.")
          return(NULL)
        }
  • Skipped gamma fitting when less than two quantiles are available for a site:
      if (length(unique(ocd_df$Quantile)) < 2) {
             warning("Insufficient quantiles for gamma fitting.")
             return(NULL)
           }
  1. Added directory creation with recursive = TRUE to automatically create full directory paths when needed.

Motivation and Context

Review Time Estimate

  • [ ] Immediately
  • [ ] Within one week
  • [ ] When possible

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • [ ] My change requires a change to the documentation.
  • [ ] My name is in the list of CITATION.cff
  • [ ] I agree that PEcAn Project may distribute my contribution under any or all of
    • the same license as the existing code,
    • and/or the BSD 3-clause license.
  • [ ] I have updated the CHANGELOG.md.
  • [ ] I have updated the documentation accordingly.
  • [ ] I have read the CONTRIBUTING document.
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.

divine7022 avatar Apr 03 '25 09:04 divine7022

Hello sir, @dlebauer @infotroph Could you please review this PR when you have a moment, containing important error fixes and enhancements for the SoilGrids data extraction.

divine7022 avatar Apr 03 '25 09:04 divine7022

@dlebauer thanks for ur feedback! addressed the suggestion

divine7022 avatar Aug 07 '25 21:08 divine7022