serverless-for-data-scientists
serverless-for-data-scientists copied to clipboard
mikepqr
Code and notebooks for a talk given at PyBay, 2018-08-19
Serverless for data scientists
Code and notebooks for a talk given at PyBay, 2018-08-19.
Contents
- Deployment of a basic flask application with zappa
- Pywren demo
- Hyperparameter optimization with pywren
- Model deployment on AWS Lambda with zappa
Projects and resources mentioned in the talk
- zappa -- tool for AWS Lambda deployment of Python applications
- dwx -- Twitter bot that runs on AWS Labmda.
- Serving 39 Million Requests for $370/Month -- AWS Lambda migration example
- Occupy the Cloud: Distributed Computing for the 99% -- the pywren paper
- pywren -- tool for embarrassingly parallel computation using AWS Lambda as the backend
- riscamp 2017 pywren tutorial -- more detailed look at pywren than my talk
- pywren web scraping -- blog post that inspired the webscraping example in the talk
- 305 Million Solutions to The Black-Scholes Equation in 16 Minutes with AWS Lambda -- large scale embarrassingly parallel numerical computation example
- nips.json -- code to scrape the NIPS website using pywren (and the data scraped)
- Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads -- an example of AWS Lambda as part of a long-lived data processing pipeline (doesn't use pywren)
mikepqr