ismn icon indicating copy to clipboard operation
ismn copied to clipboard

No 'variable' in station '3.09' and min_depth/max_depth don't work

Open xushanthu-2014 opened this issue 1 year ago • 14 comments

I am trying to extract data from station 3.09, about the variable 'soil_moisture' from depth 0.01 to 0.04. By default I should write command 1 like this:

min_depth,max_depth=0.01, 0.04
ids = ismn_data.get_dataset_ids(variable='soil_moisture',
                                        min_depth=min_depth,
                                        max_depth=max_depth,
                                        filter_meta_dict={'station': '3.09',
                                                          'lc_2000': [10,11,12,20,60,130],
                                                          'lc_2005': [10,11,12,20,60,130],
                                                          'lc_2010': [10,11,12,20,60,130],})

But this yields no element in ids. So I tried to print the metadata for 3.09 by ismn_data.read(1098, return_meta=True), and found there is indeed soil moisture in the metadata, but no values in variable:

ismn_data.read(1098, return_meta=True)
Out[117]: 
(                     soil_moisture soil_moisture_flag soil_moisture_orig_flag
 date_time                                                                    
 2017-01-01 00:00:00          0.192                  G                       M
                           ...                ...                     ...
 2019-02-22 09:00:00          0.155                  G                       M
 
 [12745 rows x 3 columns],
 variable        key       
 clay_fraction   val                           5.2
                 depth_from                    0.0
                 depth_to                     0.05
 climate_KG      val                           Dfb
 climate_insitu  val                       unknown
 elevation       val                         104.0
 instrument      val                 Decagon-5TE-B
                 depth_from                    0.0
                 depth_to                     0.05
 latitude        val                       55.8609
 lc_2000         val                            10
 lc_2005         val                            10
 lc_2010         val                            10
 lc_insitu       val                          None
 longitude       val                        9.2945
 network         val                          HOBE
 organic_carbon  val                           0.5
                 depth_from                    0.0
                 depth_to                      0.3
 sand_fraction   val                          85.1
                 depth_from                    0.0
                 depth_to                     0.05
 saturation      val                          0.41
                 depth_from                    0.0
                 depth_to                      0.3
 silt_fraction   val                           5.7
                 depth_from                    0.0
                 depth_to                     0.05
 station         val                          3.09
 timerange_from  val           2017-01-01 00:00:00
 timerange_to    val           2019-02-22 09:00:00
 variable        val                 soil_moisture
                 depth_from                    0.0
                 depth_to                     0.05
 Name: data, dtype: object)

You can see, right above clay_fraction, there is no value of key variable. So I have to use command 2

ids = ismn_data.get_dataset_ids(variable=None,min_depth=min_depth,
                                        max_depth=max_depth,
                                        filter_meta_dict={'station': '3.09',
                                                          'variable'='soil_moisture',
                                                          'lc_2000': [10,11,12,20,60,130],
                                                          'lc_2005': [10,11,12,20,60,130],
                                                          'lc_2010': [10,11,12,20,60,130],})

but still get nothing in ids. I found that's because I set min_depth and max_depth. If I delete min_depth and max_depth in command 2, I can get ids as [1098, 1104]. But I do want to extract values between 0.01 and 0.04. So is there anything wrong in the data on 3.09? And I am confused what's the difference bewteen command 1 and command 2?

xushanthu-2014 avatar Sep 11 '22 18:09 xushanthu-2014