datahub icon indicating copy to clipboard operation
datahub copied to clipboard

Automation of CRS derived data series

Open Duncan-Knox opened this issue 7 years ago • 6 comments

Requires further look-in to determine scale.

Contains the following:

donor_profile.recipient_bundle_2015 donor_profile.loans_grants_2015 donor_profile.sectors_over_time_2015 donor_profile.donor_bundle_2015

Duncan-Knox avatar Aug 09 '18 15:08 Duncan-Knox

I am not sure if this needs to be a new issue or if it is correct to note here. @akmiller01 we need to find the dimension tables that were used in Jan or Feb. @Duncan-Knox can give more info.

k8hughes avatar Aug 16 '18 13:08 k8hughes

Here's a full list of all the existing dimension tables in the DDW:

> dims = schemaListTables("dimension")
> dims[order(dims)]
 [1] "deflator_metadata"                                             
 [2] "di_id"                                                         
 [3] "di_id_to_iso_3166_1_map"                                       
 [4] "di_itep_channel"                                               
 [5] "di_itep_sector"                                                
 [6] "imf_weo_country"                                               
 [7] "imf_weo_country_to_di_id_map"                                  
 [8] "imf_weo_ncu_deflator_2014_2016_apr_pivoted"                    
 [9] "imf_weo_ncu_deflator_2014_2016_oct_pivoted"                    
[10] "imf_weo_ncu_deflator_2015_2017_apr_pivoted"                    
[11] "imf_weo_usd_deflator_2014_2016_apr_pivoted"                    
[12] "imf_weo_usd_deflator_2014_2016_oct_pivoted"                    
[13] "imf_weo_usd_deflator_2015_2017_apr_pivoted"                    
[14] "iso_3166_1"                                                    
[15] "master_oecd_deflator"                                          
[16] "master_oecd_deflator_2016_12_20"                               
[17] "master_oecd_deflator_pivoted"                                  
[18] "master_oecd_deflator_pivoted_2016_12_20"                       
[19] "oda_bundle"                                                    
[20] "oecd_crs_channel_code_5_digit"                                 
[21] "oecd_crs_channel_code_5_digit_to_di_itep_channel_map"          
[22] "oecd_crs_channel_code_5_digit_to_itep_channel_web_id_map"      
[23] "oecd_crs_donor"                                                
[24] "oecd_crs_purpose_code_5_digit"                                 
[25] "oecd_crs_purpose_code_5_digit_to_di_itep_sector_map"           
[26] "oecd_crs_purpose_code_5_digit_to_itep_sector_web_id_map"       
[27] "oecd_crs_recipient"                                            
[28] "oecd_crs_sector_code_3_digit"                                  
[29] "oecd_crs_sector_code_3_digit_to_di_itep_sector_map"            
[30] "oecd_crs_sector_code_3_digit_to_itep_sector_web_id_map"        
[31] "oecd_dac_1_donor"                                              
[32] "oecd_dac_2a_donor"                                             
[33] "oecd_dac_2a_recipient"                                         
[34] "oecd_dac_2b_donor"                                             
[35] "oecd_dac_2b_recipient"                                         
[36] "oecd_deflator_lookup"                                          
[37] "oecd_deflator_pivoted_dac_2016_01_20_non_dac_2016_01_20_dac_1" 
[38] "oecd_deflator_pivoted_dac_2016_01_20_non_dac_2016_01_20_dac_2a"
[39] "oecd_deflator_pivoted_dac_2016_04_20_non_dac_2016_04_20_dac_1" 
[40] "oecd_deflator_pivoted_dac_2016_04_20_non_dac_2016_04_20_dac_2a"
[41] "oecd_deflator_pivoted_dac_2016_07_20_non_dac_2016_07_20_dac_1" 
[42] "oecd_deflator_pivoted_dac_2016_07_20_non_dac_2016_07_20_dac_2a"
[43] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_crs"   
[44] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_1" 
[45] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_2a"
[46] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_10_20_dac_2b"
[47] "oecd_deflator_pivoted_dac_2016_10_20_non_dac_2016_11_21_dac_5" 
[48] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_crs"   
[49] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_1" 
[50] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_2a"
[51] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_2b"
[52] "oecd_deflator_pivoted_dac_2016_12_20_non_dac_2016_12_20_dac_5" 
[53] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_12_crs"   
[54] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_1" 
[55] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_2a"
[56] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_2b"
[57] "oecd_deflator_pivoted_dac_2017_04_13_non_dac_2017_04_13_dac_5" 
[58] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_crs"   
[59] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_1" 
[60] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_2a"
[61] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_2b"
[62] "oecd_deflator_pivoted_dac_2017_04_25_non_dac_2017_04_25_dac_5" 
[63] "oecd_donor"                                                    
[64] "oecd_donor_to_di_id_map"                                       
[65] "oecd_donor_to_iso_3166_1_map"                                  
[66] "oecd_donor_type"                                               
[67] "oecd_loc_donor"                                                
[68] "oecd_loc_recipient"                                            
[69] "oecd_recipient"                                                
[70] "oecd_recipient_income_group"                                   
[71] "oecd_recipient_to_di_id_map"                                   
[72] "oecd_recipient_to_iso_3166_1_map"                              
[73] "oof_bundle"                                                    
[74] "time"                                                          
[75] "wb_wdi_country"                                                
[76] "wb_wdi_country_to_di_id_map"                                   
[77] "wb_wdi_country_to_imf_weo_country_map"                         
[78] "wb_wdi_usd_deflator_2013_2016_01_20_pivoted"                   
[79] "wb_wdi_usd_deflator_2014_2016_04_20_pivoted"                   
[80] "wb_wdi_usd_deflator_2014_2016_07_20_pivoted"                   
[81] "wb_wdi_usd_deflator_2014_2016_10_20_pivoted"                   
[82] "wb_wdi_usd_deflator_2014_2016_12_20_pivoted"                   
[83] "wb_wdi_usd_deflator_2015_2016_12_20_pivoted"                   
[84] "wb_wdi_usd_deflator_2015_2017_04_25_pivoted"

akmiller01 avatar Aug 16 '18 14:08 akmiller01

Hi @akmiller01 thanks for sharing. The ones I was raising which had me confused around versions were the following:

Channels:

[4] "di_itep_channel" [20] "oecd_crs_channel_code_5_digit"
[21] "oecd_crs_channel_code_5_digit_to_di_itep_channel_map"
[22] "oecd_crs_channel_code_5_digit_to_itep_channel_web_id_map"

Sectors and purpose codes:

[5] "di_itep_sector"
[24] "oecd_crs_purpose_code_5_digit"
[25] "oecd_crs_purpose_code_5_digit_to_di_itep_sector_map"
[26] "oecd_crs_purpose_code_5_digit_to_itep_sector_web_id_map"
[28] "oecd_crs_sector_code_3_digit"
[29] "oecd_crs_sector_code_3_digit_to_di_itep_sector_map"
[30] "oecd_crs_sector_code_3_digit_to_itep_sector_web_id_map"

These had been updated more recently, however it looked like the ones coming through, especially for channels were older versions. Any help here in recovering or finding the newer version would be very helpful. Please let me know if this makes sense as a query.

Duncan-Knox avatar Aug 16 '18 14:08 Duncan-Knox

@Duncan-Knox , attached is the current state of all of the dimensions listed above. If these dimension tables are up to date, but the code is still not creating the correct output, we'll need to adjust the code rather than the dimension tables. dims.xlsx

akmiller01 avatar Aug 16 '18 15:08 akmiller01

@akmiller01 thank you. The dimension tables attached are not up to date.

Duncan-Knox avatar Aug 16 '18 15:08 Duncan-Knox

Complete for digital profiles. Amendments required for any PDF profile data. Put on hold given PDF prioritisation has been decreased in favour of digital profiles.

Duncan-Knox avatar Sep 25 '18 15:09 Duncan-Knox