microsoft
Sentinel-5 P data usage
Hi! I've been trying to use sentinel-5 data to map CH4 emissions over my area of interest.
I replicated pretty much the code that i previously had for sentinel 1 and sentinel 2 processing:
sentinel_search_url = "https://planetarycomputer.microsoft.com/api/stac/v1"
sentinel_stac_client = pystac_client.Client.open(sentinel_search_url)
items = sentinel_stac_client.search(
bbox=bounds,
collections=["sentinel-5p-l2-netcdf"],
datetime=f"{start_date}/{end_date}",
query={"s5p:processing_mode": {"eq": "OFFL"}, "s5p:product_name": {"eq": "ch4"}},
).item_collection()
print(f"Total number of items found: {len(items)}")
scene_dates = sorted([item.datetime for item in items])
selected_scenes = [scene_dates[0]]
for i in range(1, len(scene_dates)):
if (scene_dates[i] - selected_scenes[-1]).days >= min_time_step:
selected_scenes.append(scene_dates[i])
items = [
item for item in items if item.datetime in selected_scenes
]
print(f"Number of items after minimum time step filtering: {len(items)}")
print("\nSelected scenes after filtering:")
for item in items:
print(f"Scene date: {item.datetime}")
(i had to provide a proj:epsg field as this data doesn't have it)
for item in items:
# Access the asset metadata
asset = item.assets['ch4']
# Add proj:epsg metadata if it doesn't exist
if 'proj:epsg' not in asset.extra_fields:
asset.extra_fields['proj:epsg'] = 4326
sentinel_stack = stackstac.stack(
items,
assets=["ch4"],
gdal_env=stackstac.DEFAULT_GDAL_ENV.updated({
'GDAL_HTTP_MAX_RETRY': 3,
'GDAL_HTTP_RETRY_DELAY': 5,
'AWS_NO_SIGN_REQUEST': 'YES',
}),
epsg=4326,
resolution=0.0629,
chunksize=(1, 1, 1000, 1000),
).to_dataset(dim='band')
# This length number represents the number of assets (bands) that are to be extracted
# len(sentinel_stack)
sentinel_stack
# Remove attributes that are not time, y or x
sentinel_stack = sentinel_stack.drop_vars([c for c in sentinel_stack.coords if not (c in ['time', 'y', 'x',])])
sentinel_stack
sentinel_stack_float32 = sentinel_stack.astype('float32')
sentinel_stack_float32
crs = "EPSG:4326"
crs_number = crs[5:]
sentinel_stack_float32 = sentinel_stack_float32.rio.write_crs(fr"{crs}", inplace=True)
sentinel_stack_float32
sentinel_stack_clipped = sentinel_stack_float32.rio.clip(aoi.geometry, aoi.crs, all_touched=True, drop=True)
with ProgressBar():
sentinel_stack_clipped = sentinel_stack_clipped.persist()
Here, i get:
############ ] | 32% Completed | 1.75 sms c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/01/S5P_OFFL_L2__CH4____20240101T092629_20240101T110759_32220_03_020600_20240103T013302/S5P_OFFL_L2__CH4____20240101T092629_20240101T110759_32220_03_020600_20240103T013302.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/06/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/03/17/S5P_OFFL_L2__CH4____20240317T104914_20240317T123044_33299_03_020600_20240319T160340/S5P_OFFL_L2__CH4____20240317T104914_20240317T123044_33299_03_020600_20240319T160340.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/04/19/S5P_OFFL_L2__CH4____20240419T103414_20240419T121544_33767_03_020600_20240421T231632/S5P_OFFL_L2__CH4____20240419T103414_20240419T121544_33767_03_020600_20240421T231632.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/03/27/S5P_OFFL_L2__CH4____20240327T110315_20240327T124445_33441_03_020600_20240329T115341/S5P_OFFL_L2__CH4____20240327T110315_20240327T124445_33441_03_020600_20240329T115341.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/03/22/S5P_OFFL_L2__CH4____20240322T105615_20240322T123745_33370_03_020600_20240324T111434/S5P_OFFL_L2__CH4____20240322T105615_20240322T123745_33370_03_020600_20240324T111434.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/02/26/S5P_OFFL_L2__CH4____20240226T102115_20240226T120245_33015_03_020600_20240228T022449/S5P_OFFL_L2__CH4____20240226T102115_20240226T120245_33015_03_020600_20240228T022449.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/04/01/S5P_OFFL_L2__CH4____20240401T111013_20240401T125144_33512_03_020600_20240403T032212/S5P_OFFL_L2__CH4____20240401T111013_20240401T125144_33512_03_020600_20240403T032212.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/02/21/S5P_OFFL_L2__CH4____20240221T101417_20240221T115547_32944_03_020600_20240223T023451/S5P_OFFL_L2__CH4____20240221T101417_20240221T115547_32944_03_020600_20240223T023451.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/11/S5P_OFFL_L2__CH4____20240111T093933_20240111T112103_32362_03_020600_20240113T020813/S5P_OFFL_L2__CH4____20240111T093933_20240111T112103_32362_03_020600_20240113T020813.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/03/07/S5P_OFFL_L2__CH4____20240307T103513_20240307T121644_33157_03_020600_20240309T030532/S5P_OFFL_L2__CH4____20240307T103513_20240307T121644_33157_03_020600_20240309T030532.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/03/02/S5P_OFFL_L2__CH4____20240302T102814_20240302T120944_33086_03_020600_20240304T025010/S5P_OFFL_L2__CH4____20240302T102814_20240302T120944_33086_03_020600_20240304T025010.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/04/08/S5P_OFFL_L2__CH4____20240408T103921_20240408T122051_33611_03_020600_20240410T030005/S5P_OFFL_L2__CH4____20240408T103921_20240408T122051_33611_03_020600_20240410T030005.nc': RasterioIOError('HTTP response code: 404') ... c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/21/S5P_OFFL_L2__CH4____20240121T095247_20240121T113418_32504_03_020600_20240123T022011/S5P_OFFL_L2__CH4____20240121T095247_20240121T113418_32504_03_020600_20240123T022011.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/07/24/S5P_OFFL_L2__CH4____20240724T103609_20240724T121739_35129_03_020600_20240726T072607/S5P_OFFL_L2__CH4____20240724T103609_20240724T121739_35129_03_020600_20240726T072607.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings... [################################## ] | 85% Completed | 1.96 s c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/07/29/S5P_OFFL_L2__CH4____20240729T104210_20240729T122340_35200_03_020600_20240804T092141/S5P_OFFL_L2__CH4____20240729T104210_20240729T122340_35200_03_020600_20240804T092141.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/03/S5P_OFFL_L2__CH4____20240803T104809_20240803T122938_35271_03_020600_20240809T130432/S5P_OFFL_L2__CH4____20240803T104809_20240803T122938_35271_03_020600_20240809T130432.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/08/S5P_OFFL_L2__CH4____20240808T105404_20240808T123534_35342_03_020600_20240812T214910/S5P_OFFL_L2__CH4____20240808T105404_20240808T123534_35342_03_020600_20240812T214910.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/18/S5P_OFFL_L2__CH4____20240818T110547_20240818T124717_35484_03_020600_20240820T031012/S5P_OFFL_L2__CH4____20240818T110547_20240818T124717_35484_03_020600_20240820T031012.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/23/S5P_OFFL_L2__CH4____20240823T111134_20240823T125304_35555_03_020600_20240825T033549/S5P_OFFL_L2__CH4____20240823T111134_20240823T125304_35555_03_020600_20240825T033549.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/28/S5P_OFFL_L2__CH4____20240828T111719_20240828T125848_35626_03_020600_20240830T032246/S5P_OFFL_L2__CH4____20240828T111719_20240828T125848_35626_03_020600_20240830T032246.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/09/02/S5P_OFFL_L2__CH4____20240902T112301_20240902T130431_35697_03_020600_20240904T034141/S5P_OFFL_L2__CH4____20240902T112301_20240902T130431_35697_03_020600_20240904T034141.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/09/07/S5P_OFFL_L2__CH4____20240907T112851_20240907T131021_35768_03_020600_20240909T040338/S5P_OFFL_L2__CH4____20240907T112851_20240907T131021_35768_03_020600_20240909T040338.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/09/17/S5P_OFFL_L2__CH4____20240917T114040_20240917T132210_35910_03_020701_20240919T063955/S5P_OFFL_L2__CH4____20240917T114040_20240917T132210_35910_03_020701_20240919T063955.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/08/13/S5P_OFFL_L2__CH4____20240813T105958_20240813T124127_35413_03_020600_20240815T215037/S5P_OFFL_L2__CH4____20240813T105958_20240813T124127_35413_03_020600_20240815T215037.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/09/22/S5P_OFFL_L2__CH4____20240922T114632_20240922T132801_35981_03_020701_20240924T071754/S5P_OFFL_L2__CH4____20240922T114632_20240922T132801_35981_03_020701_20240924T071754.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/09/12/S5P_OFFL_L2__CH4____20240912T113447_20240912T131617_35839_03_020701_20240914T201419/S5P_OFFL_L2__CH4____20240912T113447_20240912T131617_35839_03_020701_20240914T201419.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/26/S5P_OFFL_L2__CH4____20240126T114058_20240126T132228_32576_03_020600_20240128T051841/S5P_OFFL_L2__CH4____20240126T114058_20240126T132228_32576_03_020600_20240128T051841.nc': RasterioIOError('HTTP response code: 404') ... c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/11/12/S5P_OFFL_L2__CH4____20241112T104321_20241112T122450_36704_03_020701_20241115T183324/S5P_OFFL_L2__CH4____20241112T104321_20241112T122450_36704_03_020701_20241115T183324.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/12/17/S5P_OFFL_L2__CH4____20241217T094534_20241217T112704_37200_03_020800_20241219T020255/S5P_OFFL_L2__CH4____20241217T094534_20241217T112704_37200_03_020800_20241219T020255.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings... [########################################] | 100% Completed | 2.06 s c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/02/05/S5P_OFFL_L2__CH4____20240205T115424_20240205T133554_32718_03_020600_20240207T042306/S5P_OFFL_L2__CH4____20240205T115424_20240205T133554_32718_03_020600_20240207T042306.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/02/10/S5P_OFFL_L2__CH4____20240210T120110_20240210T134240_32789_03_020600_20240212T042117/S5P_OFFL_L2__CH4____20240210T120110_20240210T134240_32789_03_020600_20240212T042117.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg) c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:329: UserWarning: Error opening 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/02/16/S5P_OFFL_L2__CH4____20240216T100729_20240216T114900_32873_03_020600_20240218T023150/S5P_OFFL_L2__CH4____20240216T100729_20240216T114900_32873_03_020600_20240218T023150.nc': RasterioIOError('HTTP response code: 404') warnings.warn(msg)
And when i try to save the datarray by computing the mean over this clipped area, i get a tiff file with no results
update:
I forget to sign in to get access to STAC items
I added:
for item in items:
for asset_key, asset in item.assets.items():
asset.href = pc_sign(asset.href)
but now i got this error:
RuntimeError: Assets must have exactly 1 band, but file 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/06/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844.nc?st=2025-03-19T16%3A57%3A48Z&se=2025-03-20T17%3A42%3A48Z&sp=rl&sv=2024-05-04&sr=c&skoid=9c8ff44a-6a2c-4dfb-b298-1c9212f64d9a&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-03-17T09%3A00%3A07Z&ske=2025-03-24T09%3A00%3A07Z&sks=b&skv=2024-05-04&sig=twF3L4zjzRxfoNWUJPXqZWXHLL4w%2BLeor3fYlDkWQ2E%3D' has 0. We can't currently handle multi-band rasters (each band has to be a separate STAC asset), so you'll need to exclude this asset from your analysis.
start_date = "2024-01-01"
end_date = "2024-12-30"
time_range = f"{start_date}/{end_date}"
aoi_path = "aoi_1_4326_part_1.geojson"
aoi = gpd.read_file(aoi_path)
min_time_step = 30
output_directory = r"C:\Users\fuji_\methane-mapping"
bounds = aoi.total_bounds
geometry = box(bounds[0], bounds[1], bounds[2], bounds[3])
wkt_aoi = geometry.wkt
geojson_aoi = mapping(geometry)
print(wkt_aoi)
# Search for Sentinel-5p L2 data using the Planetary Computer STAC API
sentinel_search_url = "https://planetarycomputer.microsoft.com/api/stac/v1"
sentinel_stac_client = pystac_client.Client.open(sentinel_search_url)
items = sentinel_stac_client.search(
bbox=bounds,
collections=["sentinel-5p-l2-netcdf"],
datetime=f"{start_date}/{end_date}",
query={"s5p:processing_mode": {"eq": "OFFL"}, "s5p:product_name": {"eq": "ch4"}},
).item_collection()
print(f"Total number of items found: {len(items)}")
scene_dates = sorted([item.datetime for item in items])
selected_scenes = [scene_dates[0]]
for i in range(1, len(scene_dates)):
if (scene_dates[i] - selected_scenes[-1]).days >= min_time_step:
selected_scenes.append(scene_dates[i])
items = [
item for item in items if item.datetime in selected_scenes
]
print(f"Number of items after minimum time step filtering: {len(items)}")
print("\nSelected scenes after filtering:")
for item in items:
print(f"Scene date: {item.datetime}")
# Print available assets in the first item
first_item = items[0]
# Check available assets
print("Available assets:", list(first_item.assets.keys()))
## Must sign to the PC STAC assets to get access
for item in items:
for asset_key, asset in item.assets.items():
asset.href = pc_sign(asset.href)
sentinel_stack = stackstac.stack(
items,
assets=["ch4"],
gdal_env=stackstac.DEFAULT_GDAL_ENV.updated({
'GDAL_HTTP_MAX_RETRY': 3,
'GDAL_HTTP_RETRY_DELAY': 5,
'AWS_NO_SIGN_REQUEST': 'YES',
}),
epsg=4326,
resolution=0.0629,
chunksize=(1, 1, 1000, 1000),
).to_dataset(dim='band')
# This length number represents the number of assets (bands) that are to be extracted
# len(sentinel_stack)
# Remove attributes that are not time, y or x
sentinel_stack = sentinel_stack.drop_vars([c for c in sentinel_stack.coords if not (c in ['time', 'y', 'x',])])
sentinel_stack_float32 = sentinel_stack.astype('float32')
crs = "EPSG:4326"
crs_number = crs[5:]
sentinel_stack_float32 = sentinel_stack_float32.rio.write_crs(fr"{crs}", inplace=True)
sentinel_stack_clipped = sentinel_stack_float32.rio.clip(aoi.geometry, aoi.crs, all_touched=True, drop=True)
with ProgressBar():
sentinel_stack_clipped = sentinel_stack_clipped.persist()
[# ] | 4% Completed | 16.44 sms
[c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:325](file:///C:/Users/fuji_/anaconda3/envs/geo/Lib/site-packages/stackstac/rio_reader.py:325): NotGeoreferencedWarning: Dataset has no geotransform, gcps, or rpcs. The identity matrix will be returned.
ds = SelfCleaningDatasetReader(self.url, sharing=False)
[# ] | 4% Completed | 16.60 s
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[25], line 3
1 sentinel_stack_clipped = sentinel_stack_float32.rio.clip(aoi.geometry, aoi.crs, all_touched=True, drop=True)
2 with ProgressBar():
----> 3 sentinel_stack_clipped = sentinel_stack_clipped.persist()
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\xarray\core\dataset.py:1117, in Dataset.persist(self, **kwargs)
1093 """Trigger computation, keeping data as chunked arrays.
1094
1095 This operation can be used to trigger computation on underlying dask
(...)
1114 dask.persist
1115 """
1116 new = self.copy(deep=False)
-> 1117 return new._persist_inplace(**kwargs)
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\xarray\core\dataset.py:1085, in Dataset._persist_inplace(self, **kwargs)
1082 chunkmanager = get_chunked_array_type(*lazy_data.values())
1084 # evaluate all the dask arrays simultaneously
-> 1085 evaluated_data = chunkmanager.persist(*lazy_data.values(), **kwargs)
1087 for k, data in zip(lazy_data, evaluated_data, strict=False):
1088 self.variables[k].data = data
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\xarray\namedarray\daskmanager.py:90, in DaskManager.persist(self, *data, **kwargs)
87 def persist(self, *data: Any, **kwargs: Any) -> tuple[DaskArray | Any, ...]:
88 from dask import persist
---> 90 return persist(*data, **kwargs)
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\dask\base.py:1001, in persist(traverse, optimize_graph, scheduler, *args, **kwargs)
998 postpersists.append((rebuild, a_keys, state))
1000 with shorten_traceback():
-> 1001 results = schedule(dsk, keys, **kwargs)
1003 d = dict(zip(keys, results))
1004 results2 = [r({k: d[k] for k in ks}, *s) for r, ks, s in postpersists]
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\to_dask.py:189, in fetch_raster_window(reader_table, slices, dtype, fill_value)
182 # Only read if the window we're fetching actually overlaps with the asset
183 if windows.intersect(current_window, asset_window):
184 # NOTE: when there are multiple assets, we _could_ parallelize these reads with our own threadpool.
185 # However, that would probably increase memory usage, since the internal, thread-local GDAL datasets
186 # would end up copied to even more threads.
187
188 # TODO when the Reader won't be rescaling, support passing `output` to avoid the copy?
--> 189 data = reader.read(current_window)
191 if all_empty:
192 # Turn `output` from a broadcast-trick array to a real array, so it's writeable
193 if (
194 np.isnan(data)
195 if np.isnan(fill_value)
196 else np.equal(data, fill_value)
197 ).all():
198 # Unless the data we just read is all empty anyway
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:383, in AutoParallelRioReader.read(self, window, **kwargs)
382 def read(self, window: Window, **kwargs) -> np.ndarray:
--> 383 reader = self.dataset
384 try:
385 result = reader.read(
386 window=window,
387 out_dtype=self.dtype,
(...)
391 **kwargs,
392 )
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:379, in AutoParallelRioReader.dataset(self)
377 with self._dataset_lock:
378 if self._dataset is None:
--> 379 self._dataset = self._open()
380 return self._dataset
File c:\Users\fuji_\anaconda3\envs\geo\Lib\site-packages\stackstac\rio_reader.py:338, in AutoParallelRioReader._open(self)
336 nr_of_bands = ds.count
337 ds.close()
--> 338 raise RuntimeError(
339 f"Assets must have exactly 1 band, but file {self.url!r} has {nr_of_bands}. "
340 "We can't currently handle multi-band rasters (each band has to be "
341 "a separate STAC asset), so you'll need to exclude this asset from your analysis."
342 )
344 # Only make a VRT if the dataset doesn't match the spatial spec we want
345 if self.spec.vrt_params != {
346 "crs": ds.crs.to_epsg(),
347 "transform": ds.transform,
348 "height": ds.height,
349 "width": ds.width,
350 }:
RuntimeError: Assets must have exactly 1 band, but file 'https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__HCHO__/2024/03/01/S5P_OFFL_L2__HCHO___20240301T104708_20240301T122838_33072_03_020601_20240303T031309/S5P_OFFL_L2__HCHO___20240301T104708_20240301T122838_33072_03_020601_20240303T031309.nc?st=2025-03-19T17%3A09%3A51Z&se=2025-03-20T17%3A54%3A51Z&sp=rl&sv=2024-05-04&sr=c&skoid=9c8ff44a-6a2c-4dfb-b298-1c9212f64d9a&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-03-20T00%3A01%3A16Z&ske=2025-03-27T00%3A01%3A16Z&sks=b&skv=2024-05-04&sig=Lffz9KetmMI4xjoy%2BB7hhsD9wfIxOn5PVS/BYm72G4Q%3D' has 0. We can't currently handle multi-band rasters (each band has to be a separate STAC asset), so you'll need to exclude this asset from your analysis.
Maybe stackstac doesnt work with these kind of files, im trying with this:
datasets = []
for item in items:
f = fsspec.open(item.assets["ch4"].href).open()
ds = xr.open_dataset(f, group="PRODUCT", engine="h5netcdf")
datasets.append(ds)
# Merge datasets along the time dimension
ds_combined = xr.concat(datasets, dim="time")
# Extract methane data
ch4_data = ds_combined["methane_mixing_ratio"]
ch4_data
which is an adaptation of the code provided in the Example notebook, however it takes a long time to load a dataset (40 mins running with 12 items in the search list)
One reason it might be taking so long is because Sentinel-5 is in the form of NetCDF files which are not cloud optimized, so xarray is downloading the whole thing, regardless of your area of interest, and they are 50MB each for CH4. As far as the stacstack error, you're probably best opening an issue here https://github.com/gjoseph92/stackstac and reference one of the .nc files from MPC you're having trouble with so folks can easily download it. So for example, file
https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/06/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844.nc
Can be downloaded easily by going to the url https://planetarycomputer.microsoft.com/api/sas/v1/sign?href=https://sentinel5euwest.blob.core.windows.net/sentinel-5p/TROPOMI/L2__CH4___/2024/01/06/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844/S5P_OFFL_L2__CH4____20240106T093259_20240106T111429_32291_03_020600_20240108T014844.nc
then copy the part in "href" and paste in browser to download the file