data-making-guidelines icon indicating copy to clipboard operation
data-making-guidelines copied to clipboard

Add patterns for making databases

Open hancush opened this issue 6 years ago • 3 comments

We've got great documentation for generating files, but we could use more detail about a concise and Make-ish approach for building databases, e.g., writing SQL-heavy Makefiles. A really great example lives over in datamade/school-report-cards.

Most notably, it involves this handy lil' Make function for checking if a table exists before trying to create it that allows you to include database-level recipes in your overall flow without using dummy .table files or having things break all the time.

Let's add a Makefile 301 (since database-building seems to be a non-standard application of Make), or a database-specific appendix, documenting some good patterns. (I'm happy to take the lead!)

(We should link to the school-report-cards repo in the code examples, too!)

hancush avatar Jan 05 '18 16:01 hancush

I've taken this approach before. I'm curious why you prefer this approach to dummy files.

ghing avatar Aug 08 '18 11:08 ghing

@ghing dummy files are sometimes created even if your database operation fails, which creates an incorrect picture of dependency existence. they also clutter up the working directory. both are hacky, i.e., i wouldn’t call one more “correct,” but as a matter of which has been less annoying to contend with for me, i’ll go with checking the db (almost) every time. have you had issues with this approach?

hancush avatar Aug 08 '18 12:08 hancush

@hancush That makes sense. I didn't have any issues with the approach, I think it just felt dirty to hit the db so frequently, and to re-run recipes for targets, even though they existed. For a recent project, I went the dummy file approach, and I'll see if that ends up feeling bad also.

ghing avatar Aug 08 '18 14:08 ghing