foil
foil copied to clipboard
Utilities for data cleaning and ETL processing
foil
foil contains for data cleaning and ETL processing.
Design Conventions
- Use iterators and generators to build data processing pipelines.
- eliminate IF statements
- descriptive variable names
- reliance on the standard library
- priority for test driven development
Contributions are welcome.
Getting Started
These instructions assume Python 3.5. It is recommended that you use conda or a virtualenv. foil is fairly lightweight, but has some dependencies.
For conda install follow:
Download the conda installer. And follow setup instructions.
Conda Environment
conda create --name <environment_name> python=3.5
activate <environment_name>
conda install --file requirements.txt
python setup.py install bdist_wheel
debian installation
Follow the instructions in the link provided. DO NOT SUDO PIP INSTALL. Alias the preferred Python installation by adding, for example:
alias python='/usr/bin/python3.5'
When using Pip
pip install --upgrade pip
pip install wheel
pip install -r requirements.txt
python setup.py install bdist_wheel
Running the Tests
py.test
Running Coverage Report
py.test --cov