oggm
oggm copied to clipboard
OGGM
RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f
I followed the tutorial and got an error when running the code.
for gdir in gdirs:
try:
spinup_start_yr = 1979
# save the original melt_f
melt_f_original = gdir.read_json('mb_calib')['melt_f']
# save some reference values
area_reference = gdir.rgi_area_m2
volume_reference = tasks.get_inversion_volume(gdir)
ref_period = cfg.PARAMS['geodetic_mb_period']
df_ref_dmdtda = utils.get_geodetic_mb_dataframe().loc[gdir.rgi_id]
df_ref_dmdtda = df_ref_dmdtda.loc[df_ref_dmdtda['period'] == ref_period]
dmdtda_reference = df_ref_dmdtda['dmdtda'].values[0] * 1000
dmdtda_reference_error = df_ref_dmdtda['err_dmdtda'].values[0] * 1000
# dynamic melt_f calibration
tasks.run_dynamic_melt_f_calibration(gdir,
ys=spinup_start_yr,
ye=2020,
output_filesuffix='_dynamic_melt_f')
with xr.open_dataset(gdir.get_filepath('model_diagnostics', filesuffix='_dynamic_melt_f')) as ds:
ds_dynamic_melt_f = ds.load()
except Exception as e:
print(f"An error occurred while processing glacier {gdir.rgi_id}: {e}")
continue
<oggm.core.flowline.SemiImplicitModel object at 0x7f3efa003850> 2024-02-28 10:39:14: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_dynamic_melt_f". The error message of the dynamic spinup is: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187% 2024-02-28 10:39:14: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f on RGI60-13.44307: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187% 2024-02-28 10:39:14: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_dynamic_melt_f on RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%) An error occurred while processing glacier RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%) <oggm.core.flowline.SemiImplicitModel object at 0x7f3efa006650> <oggm.core.flowline.SemiImplicitModel object at 0x7f3efa05dd90> <oggm.core.flowline.SemiImplicitModel object at 0x7f3efdab8550> 2024-02-28 10:39:30: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_dynamic_melt_f". The error message of the dynamic spinup is: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121% 2024-02-28 10:39:30: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f on RGI60-13.44312: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121% 2024-02-28 10:39:30: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_dynamic_melt_f on RGI60-13.44312: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%) An error occurred while processing glacier RGI60-13.44312: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%) <oggm.core.flowline.SemiImplicitModel object at 0x7f3efd9197d0> <oggm.core.flowline.SemiImplicitModel object at 0x7f3efa039110> 2024-02-28 10:40:02: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_dynamic_melt_f". The error message of the dynamic spinup is: Not able to minimise without exceeding the domain! Best mismatch 56.458864375867265% 2024-02-28 10:40:02: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f on RGI60-13.44318: Not able to minimise without exceeding the domain! Best mismatch 56.458864375867265% 2024-02-28 10:40:02: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_dynamic_melt_f on RGI60-13.44318: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 56.458864375867265%) An error occurred while processing glacier RGI60-13.44318: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 56.458864375867265%) ...
Hi, yes, I know where the error is coming from. The short answer: the dynamic spin-up is not working for all glaciers globally, but it works for most. And I cannot give you any advice without knowing what you aim to achieve. What is your goal with OGGM? There are options for not using the dynamic spin-up for glaciers where it is not working, but again, it depends on what you want.
Again, these kinds of questions are perfectly suited for our Slack channel, as it is not a problem in the code but rather a challenge in finding the right workflow for your problem. On Slack, you will find a friendly community, old questions dealing with similar problems that you can look through, and you can ask a larger community for help :). Here is the blog post for Slack again: https://oggm.org/2022/10/11/Welcome-to-the-OGGM-Slack/ (you only need to write a short email briefly stating who you are and why you would like to join our Slack).
When I run this code, the dynamic spin-up does not work for parts of the glacier, but the level 5 pre-processed directories contain spin-up files for each glacier, such as 'model_geometry_spinup_historical.nc'.
Does this mean that I can only use this file to run it and not do my own dynamic spin-up?
Hi, yes, I know where the error is coming from. The short answer: the dynamic spin-up is not working for all glaciers globally, but it works for most. And I cannot give you any advice without knowing what you aim to achieve. What is your goal with OGGM? There are options for not using the dynamic spin-up for glaciers where it is not working, but again, it depends on what you want.
Again, these kinds of questions are perfectly suited for our Slack channel, as it is not a problem in the code but rather a challenge in finding the right workflow for your problem. On Slack, you will find a friendly community, old questions dealing with similar problems that you can look through, and you can ask a larger community for help :). Here is the blog post for Slack again: https://oggm.org/2022/10/11/Welcome-to-the-OGGM-Slack/ (you only need to write a short email briefly stating who you are and why you would like to join our Slack).
If the dynamic spinup is not working for a glacier we use the 'old' default fixed_geometry_spinup ('old' means before version 1.6'). When you open 'model_geometry_spinup_historical.nc' you will find a variable called is_fixed_geometry_spinup, which indicates which spinup was used.
Thanks for your reply, but the 'is_fixed_geometry_spinup' variable does not seem to specifically indicate which spinup was used.
<xarray.DataArray 'is_fixed_geometry_spinup' (time: 42)> [42 values with dtype=bool] Coordinates:
time (time) float64 1.979e+03 1.98e+03 ... 2.019e+03 2.02e+03 calendar_year (time) int64 ... calendar_month (time) int64 ... hydro_year (time) int64 ... hydro_month (time) int64 ... Attributes: description: Part of the series which are spinup unit: -
If the dynamic spinup is not working for a glacier we use the 'old' default fixed_geometry_spinup ('old' means before version 1.6'). When you open 'model_geometry_spinup_historical.nc' you will find a variable called is_fixed_geometry_spinup, which indicates which spinup was used.
if it is True it is the fixed_geometry_spinup, if it is False it is the dynamic spinup
As you said, the 'is_fixed_geometry_spinup' variable is 'False', but for the same glacier, such as RGI60-13.44309, the 'log.txt' file in the default level 5 preprocessing directory shows that 'run_dynamic_melt_f_calibration' runs successfully, but I get an error when I run it myself.
2023-08-15T22:58:15;run_dynamic_melt_f_calibration_spinup_historical;time:3.643380641937256;SUCCESS
2024-02-28T10:39:14;run_dynamic_melt_f_calibration_spinup_historical;RuntimeError: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%)
I cannot say why this is happening without knowing your entire workflow, the OGGM version you are using, and your reasons for wanting to redo this step. What is your goal?
You will find more resources for help when you join our Slack channel. GitHub is the place for issues related to the code, while Slack is more for discussing 'how to use OGGM for my problem'.
@Nicole1150 please share the entire code / workflow leading to this error. Please try to make it as small as possible, as minimal reproducible example
Code:
import matplotlib.pyplot as plt
import xarray as xr
import numpy as np
import pandas as pd
import seaborn as sns
import geopandas as gpd
import oggm.cfg as cfg
from oggm import cfg, utils, workflow, tasks, graphics
cfg.initialize(logging_level='WARNING')
cfg.PATHS['working_dir'] = utils.gettempdir(dirname='OGGM_dynamic_spinup3', reset=True)
fr = utils.get_rgi_region_file(13, version='62')
gdf = gpd.read_file(fr)
path = '/root/kaidu_up2.shp'
basin = gpd.read_file(path)
gdf_sel = gdf[gdf.geometry.within(basin.unary_union)]
base_url = ('https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/'
'L3-L5_files/2023.3/elev_bands/W5E5/')
gdirs = workflow.init_glacier_directories(gdf_sel, from_prepro_level=3, prepro_border=160, prepro_base_url=base_url)
for gdir in gdirs:
try:
spinup_start_yr = 1979
# save the original melt_f
melt_f_original = gdir.read_json('mb_calib')['melt_f']
# save some reference values
area_reference = gdir.rgi_area_m2
volume_reference = tasks.get_inversion_volume(gdir)
ref_period = cfg.PARAMS['geodetic_mb_period']
df_ref_dmdtda = utils.get_geodetic_mb_dataframe().loc[gdir.rgi_id]
df_ref_dmdtda = df_ref_dmdtda.loc[df_ref_dmdtda['period'] == ref_period]
dmdtda_reference = df_ref_dmdtda['dmdtda'].values[0] * 1000
dmdtda_reference_error = df_ref_dmdtda['err_dmdtda'].values[0] * 1000
# dynamic melt_f calibration
tasks.run_dynamic_melt_f_calibration(gdir,
ys=spinup_start_yr,
ye=2020,
output_filesuffix='_spinup_historical')
with xr.open_dataset(gdir.get_filepath('model_diagnostics', filesuffix='_dynamic_melt_f')) as ds:
ds_dynamic_melt_f = ds.load()
except Exception as e:
print(f"An error occurred while processing glacier {gdir.rgi_id}: {e}")
continue
ERROR:
<oggm.core.flowline.SemiImplicitModel object at 0x7f3efb1d3fd0>
2024-02-29 15:42:59: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_spinup_historical". The error message of the dynamic spinup is: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%
2024-02-29 15:42:59: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_spinup_historical on RGI60-13.44307: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%
2024-02-29 15:42:59: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_spinup_historical on RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%)
An error occurred while processing glacier RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%)
<oggm.core.flowline.SemiImplicitModel object at 0x7f3ef99b92d0>
<oggm.core.flowline.SemiImplicitModel object at 0x7f3efeb4fad0>
<oggm.core.flowline.SemiImplicitModel object at 0x7f3ef8acac90>
2024-02-29 15:43:24: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_spinup_historical". The error message of the dynamic spinup is: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%
2024-02-29 15:43:24: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_spinup_historical on RGI60-13.44312: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%
...
I tested your code with rgi_ids = ['RGI60-13.44307', 'RGI60-13.44309'] and the newest oggm version. There were no errors for me. (The only thing you should change output_filesuffix='_spinup_historical' to output_filesuffix='_dynamic_melt_f' as you try to open this file below.)
Which oggm version you are using?
Thank you for your reply. The version I am using is 1.6.1. I tried to change my code to continue running, but an error still occurred. I will show you the entire process.
>>> import matplotlib.pyplot as plt
>>> import xarray as xr
>>> import numpy as np
>>> import pandas as pd
>>> import seaborn as sns
>>> import geopandas as gpd
>>> import oggm.cfg as cfg
>>> from oggm import cfg, utils, workflow, tasks, graphics
>>> cfg.initialize(logging_level='WARNING')
2024-02-29 20:08:16: oggm.cfg: Reading default parameters from the OGGM `params.cfg` configuration file.
2024-02-29 20:08:16: oggm.cfg: Multiprocessing switched OFF according to the parameter file.
2024-02-29 20:08:16: oggm.cfg: Multiprocessing: using all available processors (N=2)
>>> cfg.PATHS['working_dir'] = utils.gettempdir(dirname='OGGM_dynamic_spinup3', reset=True)
>>> fr = utils.get_rgi_region_file(13, version='62')
>>> gdf = gpd.read_file(fr)
>>> path = '/root/kaidu_up2.shp'
>>> basin = gpd.read_file(path)
>>> gdf_sel = gdf[gdf.geometry.within(basin.unary_union)]
>>> rgi_ids = gdf_sel['RGIId'].tolist()
>>> base_url = ('https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/'
... 'L3-L5_files/2023.3/elev_bands/W5E5/')
>>> gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=3, prepro_border=160, prepro_base_url=base_url)
2024-02-29 20:09:13: oggm.workflow: init_glacier_directories from prepro level 3 on 479 glaciers.
2024-02-29 20:09:13: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 479 glaciers
2024-02-29 20:09:13: oggm.workflow: WARNING: you are trying to run an entity task on 479 glaciers with multiprocessing turned off. OGGM will run faster with multiprocessing turned on.
>>> for gdir in gdirs:
... try:
... spinup_start_yr = 1979
... # save the original melt_f
... melt_f_original = gdir.read_json('mb_calib')['melt_f']
... # save some reference values
... area_reference = gdir.rgi_area_m2
... volume_reference = tasks.get_inversion_volume(gdir)
... ref_period = cfg.PARAMS['geodetic_mb_period']
... df_ref_dmdtda = utils.get_geodetic_mb_dataframe().loc[gdir.rgi_id]
... df_ref_dmdtda = df_ref_dmdtda.loc[df_ref_dmdtda['period'] == ref_period]
... dmdtda_reference = df_ref_dmdtda['dmdtda'].values[0] * 1000
... dmdtda_reference_error = df_ref_dmdtda['err_dmdtda'].values[0] * 1000
... # dynamic melt_f calibration
... tasks.run_dynamic_melt_f_calibration(gdir,
... ys=spinup_start_yr,
... ye=2020,
... output_filesuffix='_dynamic_melt_f')
... with xr.open_dataset(gdir.get_filepath('model_diagnostics', filesuffix='_dynamic_melt_f')) as ds:
... ds_dynamic_melt_f = ds.load()
... except Exception as e:
... print(f"An error occurred while processing glacier {gdir.rgi_id}: {e}")
... continue
<oggm.core.flowline.SemiImplicitModel object at 0x7f3c245a7d10>
2024-02-29 20:15:55: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_dynamic_melt_f". The error message of the dynamic spinup is: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%
2024-02-29 20:15:55: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f on RGI60-13.44307: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%
2024-02-29 20:15:55: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_dynamic_melt_f on RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%)
An error occurred while processing glacier RGI60-13.44307: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise! Problem is unknown, need to check by hand! Best mismatch 4.032392532218187%)
<oggm.core.flowline.SemiImplicitModel object at 0x7f3c245aabd0>
<oggm.core.flowline.SemiImplicitModel object at 0x7f3c245ae410>
<oggm.core.flowline.SemiImplicitModel object at 0x7f3c28a35610>
2024-02-29 20:16:12: oggm.core.dynamic_spinup: No dynamic spinup could be conducted and the original model with no spinup is saved using the provided output_filesuffix "_dynamic_melt_f". The error message of the dynamic spinup is: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%
2024-02-29 20:16:12: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_spinup_dynamic_melt_f on RGI60-13.44312: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%
2024-02-29 20:16:12: oggm.core.dynamic_spinup: RuntimeError occurred during task run_dynamic_melt_f_calibration_dynamic_melt_f on RGI60-13.44312: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%)
An error occurred while processing glacier RGI60-13.44312: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Not able to minimise without exceeding the domain! Best mismatch 48.67667783553121%)
...
...
Is there a difference between the code you posted previously and this code?
You can try updating oggm to the development version and see if the error persists, following 'install the dev version' in the documentation https://docs.oggm.org/en/stable/installing-oggm.html#install-oggm-itself
This code only modifies the problem that you mentioned, nothing else has changed at all.
Hi, I've installed the development version and tried to run it again, the number of glaciers with errors has decreased, but I still get the following glacier error message:
rgi_ids = ['RGI60-13.44312', 'RGI60-13.44315', 'RGI60-13.44332', 'RGI60-13.44354', 'RGI60-13.44405', 'RGI60-13.44406', 'RGI60-13.44425', 'RGI60-13.44480', 'RGI60-13.44497', 'RGI60-13.44522', 'RGI60-13.44569', 'RGI60-13.44594', 'RGI60-13.44602', 'RGI60-13.44605', 'RGI60-13.44644', 'RGI60-13.44682', 'RGI60-13.44710', 'RGI60-13.44712', 'RGI60-13.44721', 'RGI60-13.44741', 'RGI60-13.44742', 'RGI60-13.44751', 'RGI60-13.44762', 'RGI60-13.44786', 'RGI60-13.44789', 'RGI60-13.44816', 'RGI60-13.44823', 'RGI60-13.44829', 'RGI60-13.44872', 'RGI60-13.44876']
Have you checked if all these glaciers are working in the preprocessed directories?
Yes, all these glaciers are in the preprocessed directories. The preprocessed directories contains a total of 479 glaciers, and only these 30 glaciers have errors during operation.
Do this glaciers work with the dynamic spinup in the preprocessed directories?
And only errors for 30 out of 479 is not bad. The dynamic spinup is not working for all glaciers but for most.
Thanks for your help! I have a curious question. For the same glacier, I get an error when I run ‘run_dynamic_melt_f_calibration_dynamic_melt_f’, but the 'log.txt' file in the level 5 preprocessing directory records that 'run_dynamic_melt_f_calibration_spinup_historical' runs successfully. What is the reason for this difference?
Error:
2024-02-29T22:46:50;run_dynamic_melt_f_calibration_dynamic_melt_f;RuntimeError: Dynamic melt_f calibration was not successful! Error Message: Dynamic calibration is not working with first guess! Error message: Dynamic spinup raised error! (Message: Could not find mismatch smaller 1% (only 1.1617287540430243%) in 30Iterations!)
Success:
2023-08-15T22:58:30;run_dynamic_melt_f_calibration_spinup_historical;time:16.12317728996277;SUCCESS
Which rgi_id?
Note that just because the log file indicates "SUCCESS," it does not necessarily mean that the dynamic spinup was used. By default, we attempt the dynamic spinup, and if it fails, we resort to using the fixed geometry spinup. Only if the fixed geometry spinup also fails will this be reflected in the log file. To determine which option was actually used, you should check gdir.get_diagnostics()['used_spinup_option'].
If you want the same you should set ignore_errors=True (as explained in the tutorials https://oggm.org/tutorials/stable/notebooks/advanced/dynamical_spinup.html).
Thank you for your reply and help. I will carefully inspect the problematic glaciers.