pygmtsar
pygmtsar copied to clipboard
AlexeyPechnikov
[Help]: PS processing
I am having issues with sbas.ps_parallel, some indications on workflow would be helpful.
Is ps_parallel supposed to be run after intf_parallel and merge_parallel? Doing so causes {final date}_F{merged subswaths}.PRM missing file error.
FileNotFoundError: [Errno 2] No such file or directory: 'raw_asc/S1_20230530_ALL_F23.PRM'
Removing this file from dates and passing custom dates causes another error:
sbas.ps_parallel(dates=sbas.df.index[:-1])
Exception: "ValueError('mmap length is greater than file size')"
It is not ready for public usage. I share the examples on Patreon for early access for sponsors only. For now, PS-SBAS works but we need ability to export the original resolution (~15x4 meter) grids for PS validation and so on. Also, there are two PS-SBAS approaches already available in PyGMTSAR and a lot of testing required to provide the recommendations how and when use them.
@SteffanDavies See the example of PS+SBAS processing: https://www.linkedin.com/feed/update/urn:li:activity:7089507310389121024/
@SteffanDavies See the example of PS+SBAS processing: https://www.linkedin.com/feed/update/urn:li:activity:7089507310389121024/
I am currently testing this and in your example sbas.ps_parallel() is called before merging, however doing so will cause:
AssertionError: ERROR: multiple subswaths [2 3] found, merge them first using SBAS.merge_parallel()
So I suppose this only supports 1 subswath at the moment?
It should work for all the subswaths. Are you sure SBAS.ps_parallel() does not work for multiple subswaths?
It should work for all the subswaths. Are you sure SBAS.ps_parallel() does not work for multiple subswaths?
If I change to only 1 subswath it works fine, otherwise it gives the above error.
Also ps_parallel() leads to huge unmanaged memory used even after it has finished. My 64 GB RAM and 16GB Swap were full after it finished.
More issues, topo_ra error and empty interferograms:
Dem is fine:
topo_ra error seems dask related?
Empty intf and corr (only plot 1 but all the same):
For some reason, you have no active Dask client for the processing and most of PyGMTSAR operations are not available.
The Dask issue can be related to disconnected client or maybe Dask version change (when Dask or distributed libraries upgraded and inconsistent to the running cluster).
ps_parallel() leads to huge unmanaged memory used even after it has finished.
Please explain how much scenes do you have to analyse?
I reverted to PyGMTSAR pygmtsar-2023.5.3 and it is working. Something has broken in latest code.
The Dask issue can be related to disconnected client or maybe Dask version change (when Dask or distributed libraries upgraded and inconsistent to the running cluster).
ps_parallel() leads to huge unmanaged memory used even after it has finished.
Please explain how much scenes do you have to analyse?
181 scenes.
But this is the recent PyPI version for now. GitHub version is in the alpha stage and I change some function calls to allow a single-pixel accuracy for PS geocoding and more. For SBAS analysis, we don’t need ~4m output accuracy, but it’s critical to verify detected PS.
But this is the recent PyPI version for now. GitHub version is in the alpha stage and I change some function calls to allow a single-pixel accuracy for PS geocoding and more. For SBAS analysis, we don’t need ~4m output accuracy, but it’s critical to verify detected PS.
I think the problem is related to your new dask delayed code. I was testing the new PS-SBAS to compare differences over railway.
Yes, actually, I've fixed lots of issues in the github version last days. And you are right, ps_parallel does not work for multiple subswaths (it's easy to fix). About the memory consumption I need to test some ways — while it looks simple for sbas.ps_parallel() when the results should be stored on dist, it's not so trivial for sbas.ps_parallel(interactive=True) call when we expect to get the lazy result immediately.
Yes, actually, I've fixed lots of issues in the github version last days. And you are right, ps_parallel does not work for multiple subswaths (it's easy to fix). About the memory consumption I need to test some ways — while it looks simple for sbas.ps_parallel() when the results should be stored on dist, it's not so trivial for sbas.ps_parallel(interactive=True) call when we expect to get the lazy result immediately.
Glad I could help. Looking forward to new PS processing.
You'd try functions https://github.com/mobigroup/gmtsar/blob/pygmtsar/pygmtsar/pygmtsar/SBAS_ps.py on top of the same commit as the notebook above (the recent commits have changed SBAS and STL processing). It works well for subswaths while the old intf_parallel "weight" supports only a single subswath grid. 200 scenes ADI calculation requires ~30 minutes for a single subswath on Apple iMac M1 or Apple Air M2.
Some posts about known corner reflectors detection on Sentinel-1 amplitudes and InSAR coherence: https://www.linkedin.com/feed/update/urn:li:activity:7091290249380691968/ https://www.linkedin.com/feed/update/urn:li:share:7091295824273408001/
The normed dispersion is large for high amplitude pixels. The intensities-based Amplitude Dispersion Index (ADI) looks as
when the amplitude-based ADI is not usable:
The weighted interferometry results validation using known corner reflectors: https://www.linkedin.com/feed/update/urn:li:activity:7092021658172981248/
Checked and it works for the known corner reflector:
https://www.linkedin.com/feed/update/urn:li:share:7098256457971732480/
A set of screencasts available on the YouTube channel: https://m.youtube.com/channel/UCSEeXKAn9f_bDiTjT6l87Lg See the related example notebooks on Patreon https://www.patreon.com/pechnikov and on LinkedIn in PDF.
@SteffanDavies GMTSAR subswath merging is not single-pixel accurate, and it introduces inconsistencies in PSI processing. Although we may not detect the shift in output products, the azimuthal accuracy of ±15m makes it impossible to perform single-pixel time-tracing. For the common multi-looking output of 60m or 90m, this is not an issue.
@SteffanDavies GMTSAR subswath merging is not single-pixel accurate, and it introduces inconsistencies in PSI processing. Although we may not detect the shift in output products, the azimuthal accuracy of ±15m makes it impossible to perform single-pixel time-tracing. For the common multi-looking output of 60m or 90m, this is not an issue.
The offset seems semi-consistent, wouldn't it be possible to do some kind of pixel shift in azimuth?
GMTSAR calculates the offsets independently for every scene, and the accuracy is ±1 pixel for my test case. We cannot predict the shift, but it can be resolved using reference scene vertical shifts for all the scenes. For more details, please refer to my LinkedIn post: https://www.linkedin.com/feed/update/urn:li:activity:7110177621832843264/
@SteffanDavies You can check the recent commit 030325a1983fad9394d564a88da78e0f78918edc in PyGMTSAR2 for performance evaluation on your hardware using the attached notebook: phasediff.ipynb.zip Please share your processed notebook in PDF (printable) format for reviewing the progress bars.
It should be significantly faster when using a processing chunk size of 2048 and with NetCDF compression disabled. The old configuration was tuned for a chunk size of 512 or 1024, and NetCDF compression was always enabled. Potentially, using Dask chunks that are four times larger allows Dask to work effectively on hosts with 4x4 times more cores compared to my 8-core development host, making it effective on 128+ core hosts.
P.S. The DEM is intentionally incomplete (check the upper left corner on the plot below) to verify that PyGMTSAR2 has no issues in this case, unlike GMTSAR where a single NODATA pixel on DEM crashes the processing completely.
@SteffanDavies You can check the recent commit 030325a in PyGMTSAR2 for performance evaluation on your hardware using the attached notebook: phasediff.ipynb.zip Please share your processed notebook in PDF (printable) format for reviewing the progress bars.
It should be significantly faster when using a processing chunk size of 2048 and with NetCDF compression disabled. The old configuration was tuned for a chunk size of 512 or 1024, and NetCDF compression was always enabled. Potentially, using Dask chunks that are four times larger allows Dask to work effectively on hosts with 4x4 times more cores compared to my 8-core development host, making it effective on 128+ core hosts.
P.S. The DEM is intentionally incomplete (check the upper left corner on the plot below) to verify that PyGMTSAR2 has no issues in this case, unlike GMTSAR where a single NODATA pixel on DEM crashes the processing completely.
Is there a specific list of scenes you would like me to test on?
@SteffanDavies Maybe do you have some results? I’ve shared my timings for GMTSAR example: https://www.linkedin.com/feed/update/urn:li:activity:7117094805393858560/
New PyGMTSAR provides processing that is 10 times faster than GMTSAR on the same case, S1A_Stack_CPGF_T173. This notebook is already available on Google Colab: https://www.linkedin.com/feed/update/urn:li:activity:7117369530749730816/
I was having some issues setting up but I noticed you no longer import SBAS from pygmtsar
Help command for S1.scan_slc() function listing "basedir" as argument despite being unrecognized.