atsaf icon indicating copy to clipboard operation
atsaf copied to clipboard

Create a data package

Open RamiKrispin opened this issue 2 years ago • 11 comments

Data requirements Frequency:

  • [x] Hourly

  • [x] Daily

  • [x] Monthly

  • [ ] Quarterly
    Structure:

  • [x] Seasonal

  • [x] Multiple seasonal

  • [x] Linear trend

  • [ ] Non-linear trend

Resources

RamiKrispin avatar Oct 01 '22 18:10 RamiKrispin

US hourly demand for electricity by subregion

API: EIA

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/region-sub-ba-data

API URL: https://api.eia.gov/v2/electricity/rto/region-sub-ba-data/data/

METHOD: GET

SERIES DESCRIPTION: Hourly demand by balancing authority subregion. Source: Form EIA-930 Product: Hourly Electric Grid Monitor

Header:

"X-Params": {
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {},
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "parent",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

RamiKrispin avatar Oct 02 '22 04:10 RamiKrispin

US hourly demand and net generation of electricity by subregion

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/region-data

API URL: https://api.eia.gov/v2/electricity/rto/region-data/data/

METHOD: GET

SERIES DESCRIPTION: Hourly demand, day-ahead demand forecast, net generation, and interchange by balancing authority. Source: Form EIA-930 Product: Hourly Electric Grid Monitor

Header:

"X-Params": {
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {
        "type": [
            "D",
            "NG"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

RamiKrispin avatar Oct 02 '22 04:10 RamiKrispin

Monthly demand for natural gas

Dashboard: https://www.eia.gov/opendata/browser/natural-gas/sum/lsum

API URL: https://api.eia.gov/v2/natural-gas/sum/lsum/data/

METHOD: GET

Header:

"X-Params": {
    "frequency": "monthly",
    "data": [
        "value"
    ],
    "facets": {
        "series": [
            "N3010NY2"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

RamiKrispin avatar Oct 02 '22 05:10 RamiKrispin

Vehicle Miles Traveled

Source: FRED

Link: https://fred.stlouisfed.org/series/TRFVOLUSM227NFWA

RamiKrispin avatar Oct 02 '22 05:10 RamiKrispin

Total Vehicle Sales

Source: FRED

Link: https://fred.stlouisfed.org/series/TOTALNSA

RamiKrispin avatar Oct 02 '22 05:10 RamiKrispin

Total Construction Spending: Total Construction in the United States

Source: FRED

Link: https://fred.stlouisfed.org/series/TTLCON

RamiKrispin avatar Oct 02 '22 05:10 RamiKrispin

Air Traffic Passenger Statistics

Source: SF Data Portal

Link: https://data.sfgov.org/Transportation/Air-Traffic-Passenger-Statistics/rkru-6vcg

RamiKrispin avatar Oct 02 '22 06:10 RamiKrispin

Air Traffic Landings Statistics

Source: SF Data Portal

Link: https://data.sfgov.org/Transportation/Air-Traffic-Landings-Statistics/fpux-q53t

RamiKrispin avatar Oct 02 '22 06:10 RamiKrispin

Recommendation from LinkedIn post:

  • https://db.nomics.world/
  • https://datasetsearch.research.google.com/
  • Eurostat Tourism - https://ec.europa.eu/eurostat/databrowser/view/TOUR_OCC_NIM__custom_1357282/default/table?lang=en
  • Income and Wealth Inequality in America, 1949–2016 - https://www.journals.uchicago.edu/doi/10.1086/708815
  • https://zenodo.org/record/2582968
  • Open sky - https://opensky-network.org/
  • California ISO - http://www.caiso.com/todaysoutlook/pages/default.aspx
  • https://physionet.org/about/database/
  • https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177312
  • https://journals.humankinetics.com/view/journals/jab/33/6/article-p464.xml
  • https://github.com/TenanATC/EMG

RamiKrispin avatar Oct 03 '22 17:10 RamiKrispin

Recommendation from Twitter post:

  • TFL data - https://t.co/sdzmokWiqW
  • Road crash data - stats19 https://docs.ropensci.org/stats19/
  • IPEADATA API https://github.com/gomesleduardo/ipeadatar
  • Envirnmental data - https://twitter.com/USGS_DataSci/status/1384541933222178816
  • Water data - https://waterdata.usgs.gov/blog/tags/dataretrieval/

RamiKrispin avatar Oct 05 '22 00:10 RamiKrispin

Daily demand, day-ahead demand forecast, net generation, and interchange by balancing authority

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/daily-region-data

API URL: https://api.eia.gov/v2/electricity/rto/daily-region-data/data/

METHOD: GET

Header:

"X-Params": {
    "frequency": "daily",
    "data": [
        "value"
    ],
    "facets": {
        "timezone": [
            "Pacific"
        ],
        "type": [
            "D"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

RamiKrispin avatar Oct 09 '22 18:10 RamiKrispin