blackmarblepy
blackmarblepy copied to clipboard
Issues running bm_raster on Anaconda
Bug Report
Description
The issue arises when replicating the daily data example. There are no problems up to gdf.explore(). But the bm_raster() function fails to process the files.
Reproducibility
This is the code:
gdf = geopandas.read_file(
"https://geodata.ucdavis.edu/gadm/gadm4.1/json/gadm41_BHS_1.json.zip"
)
gdf.explore()
bearer = os.getenv(BLACKMARBLE_TOKEN)
r_20210205 = bm_raster(
gdf, product_id="VNP46A2", date_range="2021-02-05", bearer=BLACKMARBLE_TOKEN
)
The output:
GETTING MANIFEST...: 100%
2/2 [00:00<00:00, 2.77it/s]
QUEUEING TASKS | Downloading...: 100%
2/2 [00:00<00:00, 331.33it/s]
PROCESSING TASKS | Downloading...: 100%
2/2 [00:35<00:00, 15.54s/it]
[2024-05-17 16:14:14 - backoff:105 - INFO] Backing off _download_file(...) for 0.7s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:14 - backoff:105 - INFO] Backing off _download_file(...) for 0.4s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:19 - backoff:105 - INFO] Backing off _download_file(...) for 0.1s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:19 - backoff:105 - INFO] Backing off _download_file(...) for 0.1s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:25 - backoff:105 - INFO] Backing off _download_file(...) for 0.8s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:25 - backoff:105 - INFO] Backing off _download_file(...) for 1.5s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:31 - backoff:105 - INFO] Backing off _download_file(...) for 5.3s (httpx.ReadTimeout: The read operation timed out)
[2024-05-17 16:14:31 - backoff:105 - INFO] Backing off _download_file(...) for 7.0s (httpx.ReadTimeout: The read operation timed out)
COLLECTING RESULTS | Downloading...: 100%
2/2 [00:00<00:00, 399.69it/s]
COLLATING RESULTS | Processing...: 0%
0/1 [00:00<?, ?it/s]
---------------------------------------------------------------------------
CPLE_BaseError Traceback (most recent call last)
File rasterio\\crs.pyx:775, in rasterio.crs.CRS.from_user_input()
File rasterio\\_err.pyx:209, in rasterio._err.exc_wrap_ogrerr()
CPLE_BaseError: OGR Error code 6
During handling of the above exception, another exception occurred:
CRSError Traceback (most recent call last)
Cell In[9], line 4
1 BLACKMARBLE_TOKEN = "[SUPRESSED]"
2 bearer = os.getenv(BLACKMARBLE_TOKEN)
----> 4 r_20210205 = bm_raster(
5 gdf, product_id="VNP46A2", date_range="2021-02-05", bearer=BLACKMARBLE_TOKEN
6 )
File ~\anaconda3\Lib\site-packages\pydantic\validate_call_decorator.py:59, in validate_call.<locals>.validate.<locals>.wrapper_function(*args, **kwargs)
57 @functools.wraps(function)
58 def wrapper_function(*args, **kwargs):
---> 59 return validate_call_wrapper(*args, **kwargs)
File ~\anaconda3\Lib\site-packages\pydantic\_internal\_validate_call.py:81, in ValidateCallWrapper.__call__(self, *args, **kwargs)
80 def __call__(self, *args: Any, **kwargs: Any) -> Any:
---> 81 res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
82 if self.__return_pydantic_validator__:
83 return self.__return_pydantic_validator__(res)
File ~\anaconda3\Lib\site-packages\blackmarble\raster.py:283, in bm_raster(gdf, product_id, date_range, bearer, variable, quality_flag_rm, check_all_tiles_exist, file_directory, file_prefix, file_skip_if_exists)
279 filenames = _pivot_paths_by_date(pathnames).get(date)
281 try:
282 # Open each GeoTIFF file as a DataArray and store in a list
--> 283 da = [
284 rioxarray.open_rasterio(
285 h5_to_geotiff(
286 f,
287 variable=variable,
288 quality_flag_rm=quality_flag_rm,
289 output_prefix=file_prefix,
290 output_directory=d,
291 ),
292 )
293 for f in filenames
294 ]
295 ds = merge_arrays(da)
296 ds = ds.rio.clip(gdf.geometry.apply(mapping), gdf.crs, drop=True)
File ~\anaconda3\Lib\site-packages\blackmarble\raster.py:285, in <listcomp>(.0)
279 filenames = _pivot_paths_by_date(pathnames).get(date)
281 try:
282 # Open each GeoTIFF file as a DataArray and store in a list
283 da = [
284 rioxarray.open_rasterio(
--> 285 h5_to_geotiff(
286 f,
287 variable=variable,
288 quality_flag_rm=quality_flag_rm,
289 output_prefix=file_prefix,
290 output_directory=d,
291 ),
292 )
293 for f in filenames
294 ]
295 ds = merge_arrays(da)
296 ds = ds.rio.clip(gdf.geometry.apply(mapping), gdf.crs, drop=True)
File ~\anaconda3\Lib\site-packages\blackmarble\raster.py:129, in h5_to_geotiff(f, variable, quality_flag_rm, output_directory, output_prefix)
121 height, width = data.shape
122 transform = from_origin(
123 left,
124 top,
125 (right - left) / width,
126 (top - bottom) / height,
127 )
--> 129 with rasterio.open(
130 output_path,
131 "w",
132 driver="GTiff",
133 height=height,
134 width=width,
135 count=1,
136 dtype=data.dtype,
137 crs="EPSG:4326",
138 transform=transform,
139 ) as dst:
140 dst.write(data, 1)
141 dst.update_tags(**attrs)
File ~\anaconda3\Lib\site-packages\rasterio\env.py:451, in ensure_env_with_credentials.<locals>.wrapper(*args, **kwds)
448 session = DummySession()
450 with env_ctor(session=session):
--> 451 return f(*args, **kwds)
File ~\anaconda3\Lib\site-packages\rasterio\__init__.py:327, in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
325 writer = get_writer_for_driver(driver)
326 if writer is not None:
--> 327 dataset = writer(
328 path,
329 mode,
330 driver=driver,
331 width=width,
332 height=height,
333 count=count,
334 crs=crs,
335 transform=transform,
336 dtype=dtype,
337 nodata=nodata,
338 sharing=sharing,
339 **kwargs
340 )
341 else:
342 raise DriverCapabilityError(
343 "Writer does not exist for driver: %s" % str(driver)
344 )
File rasterio\\_io.pyx:1563, in rasterio._io.DatasetWriterBase.__init__()
File rasterio\\_io.pyx:1592, in rasterio._io.DatasetWriterBase._set_crs()
File rasterio\\crs.pyx:777, in rasterio.crs.CRS.from_user_input()
CRSError: The WKT could not be parsed. OGR Error code 6
Environment
- Operating System: Windows 11
- Additional Environment Details: Jupyter Lab, Anaconda3,
I would appreciate any advice.