countrycode
countrycode copied to clipboard
vincentarelbundock
panel issues
I remember looking into codelist_panel for peacesciencer, but I saw a few issues in it. E.g. Germany should be gwcode == 255 before 1945. Korea should not have gwcode == 732 before the civil/inter-state war. I could be mistaken on how much of an issue those would be.
pinging @svmiller because they originally reported the issue here
I can confirm these issues with the GW numeric code (currently gwn not gwcode) for Korea and Germany...
relevant lines from GW source
| _ | _ | _ | _ | _ |
|---|---|---|---|---|
| 255 | GMY | Germany (Prussia) | 01:01:1816 | 07:05:1945 |
| 260 | GFR | German Federal Republic | 21:09:1949 | 31:12:2017 |
| 265 | GDR | German Democratic Republic | 05:10:1949 | 02:10:1990 |
| 730 | KOR | Korea | 01:01:1816 | 22:08:1910 |
| 731 | PRK | Korea, People's Republic of | 09:09:1948 | 31:12:2017 |
| 732 | ROK | Korea, Republic of | 15:08:1948 | 31:12:2017 |
current results from countrycode
library(dplyr)
library(countrycode)
codelist_panel %>%
filter(country.name.en == "Germany") %>%
filter(year < 1945) %>%
select(country.name.en, year, gwc, gwn)
#> # A tibble: 156 x 4
#> country.name.en year gwc gwn
#> <chr> <dbl> <chr> <dbl>
#> 1 Germany 1789 GFR 260
#> 2 Germany 1790 GFR 260
#> 3 Germany 1791 GFR 260
#> 4 Germany 1792 GFR 260
#> 5 Germany 1793 GFR 260
#> 6 Germany 1794 GFR 260
#> 7 Germany 1795 GFR 260
#> 8 Germany 1796 GFR 260
#> 9 Germany 1797 GFR 260
#> 10 Germany 1798 GFR 260
#> # … with 146 more rows
codelist_panel %>%
filter(gwn == 732) %>%
select(country.name.en, year, gwc, gwn)
#> # A tibble: 232 x 4
#> country.name.en year gwc gwn
#> <chr> <dbl> <chr> <dbl>
#> 1 South Korea 1789 ROK 732
#> 2 South Korea 1790 ROK 732
#> 3 South Korea 1791 ROK 732
#> 4 South Korea 1792 ROK 732
#> 5 South Korea 1793 ROK 732
#> 6 South Korea 1794 ROK 732
#> 7 South Korea 1795 ROK 732
#> 8 South Korea 1796 ROK 732
#> 9 South Korea 1797 ROK 732
#> 10 South Korea 1798 ROK 732
#> # … with 222 more rows
presumably this is caused by the process that creates the panel dataset here https://github.com/vincentarelbundock/countrycode/blob/a3732f916bdf7973eeeaf3cecec34618151eb302/dictionary/build.R#L66-L121
In regards to the other specifically noted problem in @svmiller's blog post...
For example, Serbia (SRB) always throws countrycode() for a loop, but Serbia has always been 345 in the CoW system. You can fix that yourself with an addendum to the mutate() wrapper. Something like ccode = ifelse(Alpha_3 == "SRB", 345, ccode) will work.
CoW assigns the code 345 to Yugoslavia according to the source we use: http://www.correlatesofwar.org/data-sets/cow-country-codes/cow-country-codes
Whether or not that should be considered the same as Serbia is probably contextual to one's data and situation. I'm not sure there's an always right or wrong answer to that, unless I don't understand it completely (which is totally possible). Note that this "official" source has duplicate rows for YUG,345,Yugoslavia (as well as a bunch of other countries), so there could be something wrong with the original source data. Maybe we should contact them and ask?
As far as dealing with the situation as it is in code, my preference is to use the custom_match argument of countrycode()...
library(countrycode)
countrycode("Serbia", "country.name", "cown")
#> Warning in countrycode("Serbia", "country.name", "cown"): Some values were not matched unambiguously: Serbia
#> [1] NA
countrycode("Serbia", "country.name", "cown", custom_match = c(Serbia = 345))
#> [1] 345
I worry my language may be misinterpreted as petty or critical in that context (re: Serbia). Please know I didn't mean anything harsh by it! Honestly.
But yeah, @cjyetman is right to note that this issue is heavily dependent on context, and mine happens to be the CoW system. It does mean that Serbia is often not matched via countrycode() even as it's a simple fix after the fact.
Not harsh at all!
I just really appreciate it when people take the time to look at things carefully and report their thoughts. I don't often work with CoW myself, so I was vaguely aware of this issue but it didn't strike me as an irritant.
Good to have an issue on Github so we can think of solutions in the future.
Thanks again for the great blog post!
yeah, no offense taken @svmiller.... if you have other specific examples of strange things related to countrycode, I'd be happy to look in to them
I finally fixed the codelist_panel problem with Gleditsch-Ward, Germany, and Korea. The issue is that we did not treat these data as panel at all, and instead took the last year only, and merged that back into the panel data.
With proper pre-processing as panel, we now get the good codes for early period Germany and Korea.
With respect to Serbia, I just checked and it doesn't seem to appear at all in the CoW codes (version 2016). This means, of course, that it doesn't show up in codelist_panel at all.
However, the "SER" code appears in codelist_panel$gwc for the following years:
1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
And "YUG" appears in codelist_panel$gwc for these years:
1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006
I am closing this issue now to clean up the repo, but I would be very eager to know if you spot an error, or if there is anything else that you think should be fixed in the panel data.
The ideal workflow, from my perspective, would be for people to just left_join() all their country-year data.