SQL data loaders

[mbostock]

It’d be nice to have foo.csv.sql and somehow automatically run against a registered database, and the CLI would convert the output to CSV automatically. I don’t know how we’d register the databases, though.

[mythmon]

YAML frontmatter or comments? Locality of configuration is nice for data loaders.

[mbostock]

We added client-side SQL code blocks using a sql front matter option. We could have .sql data loaders if we registered a database in the project config file.

[jaanli]

This would be fantastic. Right now I’m using dbt to version control the SQL data loaders for Observable Framework:

example - https://github.com/jaanli/exploring_american_community_survey_data/blob/main/american_community_survey/models/public_use_microdata_sample/figures/income-histogram-with-sector-historical-inflation-adjusted-industry-mapped.sql

corresponds to this observable framework plot: https://jaanli.github.io/american-community-survey/income