gcam-core
gcam-core copied to clipboard
JGCRI
Mismatches between configuration_ref.xml and distribution tree incl. missing files
I need to preface this issue by saying that I'm trying to set up CGAM for multiuser operation; the intent of the design seems to be for people to run GCAM out of their own filespace, specifically out of e.g. ~/gcam-core-gcam-v7.0/exe. That's no good for a multiuser installation but configuration_ref.xml is hardcoded for users to operate that way e.g. :
<Value name = "ind_water">../input/gcamdata/xml/water_demand_industry.xml</Value>
So I've sedded myself a new configuration.xml so that instead that line would read
<Value name = "ind_water">/opt/gcam/input/gcamdata/xml/water_demand_industry.xml</Value>
and that worked well enough (although using variables in those paths will not work). But there are things that are awry with the content of configuration_ref.xml that aren't just part and parcel of the app being designed to be single-user.
For instance, we have this (I'm going to refer to my modded version henceforth):
<Value name="xmlInputFileName">/opt/gcam/input/gcamdata/xml/modeltime.xml</Value>
This is a problem because there is no /opt/gcam/input/gcamdata/xml. There is a modeltime.xml, but it's in .../input/gcamdata/tests/testthat/test-data/. I "fixed" that via mkdir and ln -s, but upon the next run attempt it can't find .../input/gcamdata/xml/socioeconomics_macro.xml; because I created .../input/gcamdata/xml as I already mentioned that's not the issue but there's no socioeconomics_macro.xml in there and in fact, there's not even one in the distribution.
Same story with hector.xml.
It gets worse:
$ **for k in `grep "opt/gcam/input" jlh_configuration.xml | sed -e 's/<[^>]*>//' | sed -e 's/<\/Value>//'`; do if [ -f "$k" ]; then echo "$k"" exists" ; else echo "$k"" does not exist"; fi; done**
/opt/gcam/input/gcamdata/xml/modeltime.xml exists
/opt/gcam/input/policy/forcing_target_4p5.xml exists
/opt/gcam/input/magicc/inputs/input_gases.emk exists
/opt/gcam/input/gcamdata/xml/hector.xml does not exist
/opt/gcam/input/gcamdata/xml/socioeconomics_gSSP2.xml does not exist
/opt/gcam/input/gcamdata/xml/resources.xml does not exist
/opt/gcam/input/gcamdata/xml/en_supply.xml does not exist
/opt/gcam/input/gcamdata/xml/en_transformation.xml does not exist
/opt/gcam/input/gcamdata/xml/electricity.xml</Value--> does not exist
/opt/gcam/input/gcamdata/xml/electricity_water.xml does not exist
/opt/gcam/input/gcamdata/xml/heat.xml does not exist
/opt/gcam/input/gcamdata/xml/hydrogen.xml does not exist
/opt/gcam/input/gcamdata/xml/en_distribution.xml does not exist
/opt/gcam/input/gcamdata/xml/other_industry.xml does not exist
/opt/gcam/input/gcamdata/xml/other_industry_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/iron_steel.xml does not exist
/opt/gcam/input/gcamdata/xml/iron_steel_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/iron_steel_trade.xml does not exist
/opt/gcam/input/gcamdata/xml/Off_road.xml does not exist
/opt/gcam/input/gcamdata/xml/Off_road_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/chemical.xml does not exist
/opt/gcam/input/gcamdata/xml/chemical_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/aluminum.xml does not exist
/opt/gcam/input/gcamdata/xml/aluminum_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/cement.xml does not exist
/opt/gcam/input/gcamdata/xml/cement_incelas_gssp2.xml does not exist
/opt/gcam/input/gcamdata/xml/en_Fert.xml does not exist
/opt/gcam/input/gcamdata/xml/HDDCDD_constdd_no_GCM.xml does not exist
/opt/gcam/input/gcamdata/xml/building_det.xml does not exist
/opt/gcam/input/gcamdata/xml/transportation_UCD_CORE.xml does not exist
/opt/gcam/input/gcamdata/xml/Ccoef.xml does not exist
/opt/gcam/input/gcamdata/xml/Cstorage.xml does not exist
/opt/gcam/input/gcamdata/xml/onshore_wind.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_For_Past_bio_base_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_cost_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_prodchange_ref_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/resbio_input_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/an_input.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_Fert_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/land_input_1.xml does not exist
/opt/gcam/input/gcamdata/xml/land_input_2.xml does not exist
/opt/gcam/input/gcamdata/xml/land_input_3_IRR.xml does not exist
/opt/gcam/input/gcamdata/xml/land_input_4_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/land_input_5_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/protected_land_input_2.xml does not exist
/opt/gcam/input/gcamdata/xml/protected_land_input_3.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_an_demand_input.xml does not exist
/opt/gcam/input/gcamdata/xml/bio_trade.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_trade.xml does not exist
/opt/gcam/input/gcamdata/xml/unlimited_water_supply.xml does not exist
/opt/gcam/input/gcamdata/xml/water_supply_constrained.xml does not exist
/opt/gcam/input/gcamdata/xml/desalination.xml does not exist
/opt/gcam/input/gcamdata/xml/water_td.xml does not exist
/opt/gcam/input/gcamdata/xml/EFW_input_coefs.xml does not exist
/opt/gcam/input/gcamdata/xml/EFW_irrigation.xml does not exist
/opt/gcam/input/gcamdata/xml/EFW_manufacturing.xml does not exist
/opt/gcam/input/gcamdata/xml/EFW_municipal.xml does not exist
/opt/gcam/input/gcamdata/xml/ag_water_input_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/electricity_water_coefs.xml does not exist
/opt/gcam/input/gcamdata/xml/water_demand_industry.xml does not exist
/opt/gcam/input/gcamdata/xml/water_demand_livestock.xml does not exist
/opt/gcam/input/gcamdata/xml/water_demand_municipal.xml does not exist
/opt/gcam/input/gcamdata/xml/water_demand_primary.xml does not exist
/opt/gcam/input/gcamdata/xml/ind_urb_processing_sectors.xml does not exist
/opt/gcam/input/gcamdata/xml/all_energy_emissions.xml does not exist
/opt/gcam/input/gcamdata/xml/all_fgas_emissions.xml does not exist
/opt/gcam/input/gcamdata/xml/all_unmgd_emissions.xml does not exist
/opt/gcam/input/gcamdata/xml/all_aglu_emissions_IRR_MGMT.xml does not exist
/opt/gcam/input/gcamdata/xml/all_protected_unmgd_emissions.xml does not exist
/opt/gcam/input/gcamdata/xml/emission_factor_controls.xml does not exist
/opt/gcam/input/gcamdata/xml/all_energy_emissions_MAC.xml does not exist
/opt/gcam/input/gcamdata/xml/all_fgas_emissions_MAC.xml does not exist
/opt/gcam/input/gcamdata/xml/all_aglu_emissions_IRR_MGMT_MAC.xml does not exist
/opt/gcam/input/gcamdata/xml/ind_urb_processing_sectors_MAC.xml does not exist
/opt/gcam/input/gcamdata/xml/socioeconomics_macro.xml does not exist
/opt/gcam/input/gcamdata/xml/liquids_limits.xml does not exist
/opt/gcam/input/gcamdata/xml/water_elec_liquids_limits.xml does not exist
/opt/gcam/input/gcamdata/xml/bio_externality.xml does not exist
/opt/gcam/input/gcamdata/xml/gas_trade.xml does not exist
/opt/gcam/input/gcamdata/xml/prune_empty_ag.xml does not exist
/opt/gcam/input/solution/cal_broyden_config.xml exists
/opt/gcam/input/magicc/inputs does not exist
I ran that again with an invocation of locate interposed to see if files by those names even existed in the distribution; they don't. BTW it looks like /opt/gcam/input/gcamdata/xml/electricity.xml up there got commented out; okay.
It looks like there's not a lot of point in my going farther with GCAM 7.0 until it goes through another round of QA but poking around the 6.0 distribution tree, it's the same story there.
Is there some other place from which input files are supposed to come, because from the looks of things there are over six dozen input files that are flat-out missing from the distribution.
Since I have been simultaneously trying to get a second instance of GCAM to run under Windows, I looked at the GCAM distribution over there and it does have a populated .../input/gcamdata/xml directory so I tarballed that up and copied it to the Gentoo Linux machine where I was able to get a successful run (although the GCAM build there is without Java). It ran the reference configuration in 8m19s.
So it looks like the source distributions for GCAM 7.0 and 6.0 are missing .../input/gcamdata/xml and about 4.2GiB of XML content therein.
There is a problem for multiuser operation in that an exception was thrown via ini_to_core_reader.cpp because it was looking for ../input/climate/hector-gcam.ini. I was not able to easily determine where/how that path was hardcoded so I developed a workaround by creating a symlink called "input" beside my regular-user GCAM running directory that points to /opt/gcam/input. That path shouldn't be hardcoded. Also, GCAM wants to write into ./restart and can't absent the ./restart directory so I wasn't able to get it to run until I did "mkdir restart" in my running directory.
On Linux, you need to run the data system to generate the XML files, the prerequisite for which is installing all the needed R packages. If you're going to modify any input CSV files (on Mac and Windows as well), you need to be able to run the data system to generate the corresponding XML.
Guidance to that effect probably ought to be inserted somewhere in part 4.1 of http://jgcri.github.io/gcam-doc/gcam-build.html because that document leaves one with the impression that once built, GCAM is ready to run.
In any case, I'll see if I can get the data system set up and working; my clients are probably going to have an easier time of modifying CSV than XML.