[WIP] Version 2

[pjbull]

We've seen a lot of potential features for this where we need to handle forking paths gracefully. By default cookiecutter can't do this (see https://github.com/audreyr/cookiecutter/pull/848). It's been years, so we can't reasonably expect this to change upstream...

This implements a monkey-patching workaround to enable this behavior. It introduces a couple of major changes, so here are my recommendations.

Here are the big differences for a user:

• we now need to run ccds <path to repo> instead of cookiecutter <path to repo>
• the options and their defaults are changing
• as a consequence of supporting more environments/dependency managers at setup time, we stop supporting them at execution time. this means that teams will have to pick one of each and stick to it across developers. I think this is pretty widespread already. (i.e. make create_environment will only support one of the options rather than multiple like it does now)

Beta Testing v2

V2 is available to beta testers! Expect some bugs and changes as you go. If you encounter issues, you can file an issue and use the v2-bug label.

To beta test v2, you can install the ccds package with pip:

• pip install git+https://github.com/drivendata/cookiecutter-data-science@v2

And then to make a new project with the v2 template, you can run:

• ccds https://github.com/drivendata/cookiecutter-data-science.git --checkout v2

Note: because this is a separate package, you should be able to run it alongside your normal cookiecutter installation as well and still use v1 when you choose to. Also, always answer "yes" to re-cloning the project so that you get the latest and so v1 continues to work as well.

High-level feedback on choices and design is welcome!

Tracking issues for release

Done:

• [x] we now support multiple storage providers (#120)
• [x] we allow user to choose package manager and some default packages (#5)
• [x] we allow user to choose a dependency format (#118)
• [x] src is now {{ cookiecutter.module_name }} (#140)
• [x] remove test_environment.py
• [x] tag current master as v1 so anyone relying on the current flow/structure can continue to use it easily (#240)
• [x] implement the rails for the high priority items
• [x] make comprehensive tests run on CI/CD to de-risk
• [x] add support for azure/google cloud as storage backends (#8)
• [x] add ccds command and make cookiecutter-data-science a proper package
• [x] add support for differenct virtual env providers (#6)
• [x] add lint command to Makefile
• [x] formatting commands / configuration (e.g., black)
• [x] Configuration file format defaults (setup.cfg/pyproject.toml)
• [x] Default to no license (#241)

---

Remaining items

Cookiecutter default structure

• [ ] support user supplied config files (#325)
• [x] revise generated python package boilerplate (make optional) (#49, #326)
• [x] mkdocs in place of sphinx (#327)

Cookiecutter options

• [ ] add deon (#154)
• [ ] add nbautoexport (#324)
• [ ] test suite configuration (e.g., pytest install and make commands) #195

Infrastructure

• [x] tests passing on windows https://github.com/drivendata/cookiecutter-data-science/issues/316
• [ ] Makefiile working on Windows (and instructions) #221, #68
• [ ] release command and make PyPI release https://github.com/drivendata/cookiecutter-data-science/issues/317
• [ ] Transfer ownership to drivendataorg organization rather than drivendata user https://github.com/drivendata/cookiecutter-data-science/issues/315
• [x] Add python 3.12 support (#328)

Fixes

• [x] #248
• [x] #142
• [x] Don't show deprecation message when running v2 #281

Docs

• [x] add v2 beta testing documentation
• [x] #318
• [x] update screencast and add screenshots of the new flow (#314)
• [ ] Add links to real projects and also a few simple examples #238, #38, #48, #59
• [x] #19
• [ ] #109
• [ ] #136

---

Under consideration

• [ ] Add a default (optional) Dockerfile (#13)
• [ ] git init as an option #56
• [ ] Wrap make commands in a check that the right environment is active #83
• [ ] Add github actions boilerplate (optional)
• [ ] pre-commit hooks
• [ ] add option to render a completed demo project
• [ ] #276
• [ ] #274
• [ ] Use pytest-cookies for testing CCDS #166
• [ ] #188 Version with a make register_hooks is probably reasonable
• [ ] #282 Validation of inputs (at least that dir doesn't already exist)

Scoped out

• [ ] #9 All of the current options are still worse than make 😿
• [ ] #226 Not generic enough of a need

[shortcipher3]

Is this active? It looks like there hasn't been any changes for a while, but I would love to know when it is expected to be released.

[pjbull]

@shortcipher3 Yes, we're hoping to finish the work-in-progress here this summer. Thanks!

[raphaelreimann]

Hey, any updates on when this will be released? :)

[pjbull]

Thanks for the interest @raphaelreimann, work is progressing. Please see the updated beta testing v2 section at the start of this thread. It would be great to get feedback from interested folks that start using the latest version!