Alex Miller

With my knowledge of SQL and how it's programmed, I've tried my best to pen-test it, and it's not vulnerable to the common attacks. But testing it myself is a...

Oh, but it's built on a read only user. So the worst vulnerability possible would be accessing data they're not supposed to (which should be explicitly blacklisted).

Yep, that makes sense to validate the file format and maybe make a new error code for unknown formats. And I would agree with you on SSL. It's a "nice-to-have"...

Code for upload stored here: https://github.com/devinit/ddh_donata_scripts/blob/6e264331d2440edfd43c2834e40c1571cc8c9fcd/data_extraction/di_data_hub_mirror/CREATE_TABLES_data_series.sql#L1133-L1190 Will probably need updates to `reference.di_flow_type` and `reference.di_flow_name`

Database name is `in-ha` and here's the code to upload: https://github.com/devinit/ddh_donata_scripts/blob/6e264331d2440edfd43c2834e40c1571cc8c9fcd/data_extraction/di_data_hub_mirror/gha_updates_additions/CREATE_TABLE_in_ha.sql

Once null characters are removed from raw CRS, it can be read into R without error like so: ``` r = readBin(txt, raw(), file.info(txt)$size) r[r==as.raw(0)] = as.raw(0x20) ## replace with...

Here's a full list of all the existing `dimension` tables in the DDW: ``` > dims = schemaListTables("dimension") > dims[order(dims)] [1] "deflator_metadata" [2] "di_id" [3] "di_id_to_iso_3166_1_map" [4] "di_itep_channel" [5] "di_itep_sector"...

@Duncan-Knox , attached is the current state of all of the dimensions listed above. If these dimension tables are up to date, but the code is still not creating the...

Again, previous code seems to rely on pre-deflated usd_disbursement values being constant 2015; https://github.com/devinit/ddh_donata_scripts/blob/6e264331d2440edfd43c2834e40c1571cc8c9fcd/data_etl/dh/fact_oecd/CREATE_TABLE_oda.sql#L344-L345 So if a new scrape from the CRS will yield constant 2016, no changes are required...

I think this is actually an issue in the Schema text: https://github.com/IATI/IATI-Schemas/blob/version-2.03/iati-activities-schema.xsd#L273 Moving this issue there.