Geo-SAM2: An Extension of Geo-SAM Built upon SAM2

[wenhwu]

@coolzhao , @Fanchengyan , We sincerely appreciate your team’s outstanding contributions. Building upon your Geo-SAM plugin, we have developed Geo-SAM2 by integrating the SAM2 architecture.

Key Differences Between Geo-SAM2 and Geo-SAM

Feature	Geo-SAM	Geo-SAM2
UI Design	Integrated into the QGIS top toolbar; some functions require switching between toolbars to access.	Integrated into the QGIS sidebar; all controls are immediately visible with an optimized, user-friendly layout.
Cropping Strategy	Utilizes an Encoder Copilot module to assist in selecting cropping strategies.	Adopts a non-overlapping cropping strategy; users can visualize crop boundaries in real time to facilitate interactive refinement.
Underlying Model	Segment Anything Model (SAM) with a Vision Transformer (ViT) backbone.	Segment Anything Model 2 (SAM2) with a Hiera backbone.
Sampling Strategy	Leverages torchgeo samplers with configurable overlapping strides.	Employs a non-overlapping, edge-adaptive, GDAL-based sliding window approach (fixed 1024×1024 patches).

Challenges Encountered with Geo-SAM and Our Solutions

• Performance Bottleneck in Image Encoding:
  When processing large-scale satellite imagery (e.g., width = 30,680, height = 24,186), the Image Encoder computation in Geo-SAM becomes extremely slow. We identified that the underlying torchgeo and rasterio operations block the QGIS user interface. To address this, we completely replaced torchgeo with a lightweight, GDAL-based sliding window implementation for efficient and responsive processing.
• Poor Segmentation at Feature Patch Boundaries:
  Segmentation quality degrades significantly at the boundaries between adjacent Image Encoder feature patches. To mitigate this, we implemented real-time visualization of cropping boundaries during mask decoder processing, enabling users to interactively refine results based on visible patch seams.

We hope these findings prove valuable to your ongoing work. We welcome your feedback, discussion, and suggestions. thank you

[coolzhao]

Hi @wenhwu, thanks for your contribution to the community and for recognizing our work. I will look over your project and give you feedback if applicable.