sruti

@vituslehner thanks for pointing this out! It looks like there are some instances where `value_time_inserted` is even before `value_time_start` 🤦‍♀ Highlighted in green is the data that's being captured, yellow...

Just wanted to add that I found the [documentation](http://dd.eionet.europa.eu/vocabulary/aq/observationverification) and the `value_verification flag` is defined as follows: 1 - _Data has gone through full quality assurance and quality control by...

FYI - that source only has data from Abu Dhabi. According to this [UAE Air Quality Index site](http://uae-air-quality-index-fcsa.hub.arcgis.com/), there are ~70 stations in the UAE total, but it seems to...

I think we have everything we need for this source! Note: - the website seems to be about 12hrs behind the table, which is up to date. - the averaging...

It would be good to look into this source and see if it still exists/meets the following requirements: - [ ] Data is ‘raw’ and reported in physical concentrations not...

About the unauthorized requests, I think it's probably fine. @jflasher may have thoughts

Added some color-coding to more easily compare between different format and compression options: - Because the cost is calculated by amount of data read ($5/1TB), I think it makes sense...

#### INSERT_INTO investigation 1. Created an s3 bucket with a single realtime data gzipped file (original size ~6MB, gzipped ~200KB) 2. Ran CTAS query on it, _1.1 seconds, 0KB_: ```...

As for how to switch the data to the new format, here's a proposed plan that won't significantly change our current processes:  - A new lambda function to...

Because this is not a governmental source, we reached out to them to 1) understand what sensors they're using and 2) if they would be comfortable with sharing data on...