[WIP] Major Refactor to enable python api access and packaging

[ktehranchi]

This commit enables users to pip install pypsa-usa and run workflows from a python api as described in the new quick start guide below.

None of the python files were changed nor the snakemake rules besides the paths.... I will request reviews from folks soon

Installation

Option 1: PyPI Installation (Recommended)

pip install pypsa-usa

Option 2: Development Installation

git clone https://github.com/PyPSA/pypsa-usa.git
cd pypsa-usa
uv venv && source .venv/bin/activate
uv pip install -e .

Your First Workflow

Using the Python API

from pypsa_usa.api import run_workflow, set_default_workspace
from pathlib import Path

# Set your default workspace (only needed once)
set_default_workspace("/path/to/my/project/workspace")

# Run a simple workflow with default settings
success = run_workflow(cores=4)

if success:
    print("Workflow completed successfully!")
    print("Results are in your workspace results/ directory")
else:
    print("Workflow failed. Check logs/ for details.")

Using Snakemake Directly (Developmer Installation)

# Navigate to workflow directory
cd workflow

# Run with default configuration
uv run snakemake -j4 --configfile config/config.default.yaml

[ktehranchi]

@WeiAi-Energy @trevorb1 @mchjones TBH I want to hear from you guys if you think it's worth it to support this functionality. I'm not sure how often you think you will use it in your own research workflows. I was thinking of using this for my next project which will use a completely different repository but I wanted to make it easy to integrate different pypsa-usa network builds into a different RA workflow rather than expanding the snakemake to do Resource Adequacy assessments.

I imagine this being useful for classes that are using pypsa-usa as a teaching tool, making it easier for students to load networks. There would need to be some effort put into improving the API to actually make it easier to use than snakemake.

Let me know your thoughts...

[mchjones]

I personally probably wouldn't use it but I think it's a good functionality to develop and support to make the tool more accessible and practical for use in an educational setting. Up to you, though, to decide whether it is worth the amount of effort you'd have to put into it to make it useful.

[trevorb1]

Alternatively, we could also look at containerizing this via docker. This might be the best of both worlds. It is flexible enough so that both developers and general users would install it (as it allows for pure snakemake calls still). Plus, that way if any commands come up that are needed (ie. --unlock or --rerun-incomplete), they are easy enough to append to the docker call. I believe pypsa-eur has containerized their workflow here.

Downsides are it might be a bit of a pain to setup and manage. Ive never done it before for a real project, so idk the types of issues that are likely to come up. May be worth checking in with the pypsa-eur crew if we want to explore this more to get their thoughts. Cause it may also just be more hassle then its worth, even for developers to use.