Note If this is your first time checking out bqup, you might want to try out Dataform or dbt instead!

bqup

bqup is a backup tool for BigQuery projects. It can export a BigQuery project's structure and source code while mimicking the hierarchy of datasets and tables.

How bqup
works

For the full story of why we made bqup, check out our blog post!

Installation

bqup can be installed using pip.

pip install bqup

Alternatively, you can also clone the repository then run install.

git clone https://github.com/thinkingmachines/bqup.git
cd bqup
python3 setup.py install

Usage

Command line options

You can list down the options by running bqup --help.

bqup [-p PROJECT_ID] [-d TARGET_DIR] [-fvxr] [-e REGEX]

Options:
  -p PROJECT_ID, --project PROJECT_ID  Project ID to load. If unspecified,
                                       defaults to current project in
                                       configuration.
  -d TARGET_DIR, --dir TARGET_DIR      The target directory where the project
                                       will be written. Defaults to current
                                       timestamp.
  -f --force                           Overwrite target directory if it exists.
  -v --verbose                         Print a summary of the loaded project.
  -x --schema                          Export table schemata as json.
  -r --routine                         Include routines in export.
  -e REGEX, --regex REGEX              Regex pattern to filter datasets to be exported.

Development

Set up gcloud to run with your personal account (aka run with scissors).
Set up application-default.
```
gcloud auth application-default login
```
Install wheel.
```
pip3 install wheel
```
Install bqup.
```
pip3 install -e .
```
Alternatively, you can also install it using:
```
python3 setup.py develop
```
Run bqup (see Usage).