open list

[MikeBadescu]

Open list / wishes / references. Once fully defined, move them to separate issues

[MikeBadescu]

AWS lambda and R ref links

• Analyzing Genomics Data at Scale using R, AWS Lambda, and Amazon API Gateway
• Extracting Video Metadata using Lambda and Mediainfo
• Run ML predictions with R on AWS Lambda
• Lambda R Survival Statistics
• Serverless-Data-Product
• list of AWS actions

[MikeBadescu]

ideas:

logging

• ~~use python logging: http://docs.aws.amazon.com/lambda/latest/dg/python-logging.html~~
• ~~write input_json, output_json ?~~
• ~~log other errors (e.g. R failures) ?~~
• per 12 factors, keep it simple, let R code write to stderr stdout then collect and analyze (if needed)

CI

• TravisCI ?

[MikeBadescu]

hack to increase the deployment size:

• check that R is installed from source
• install R in the normal place, but install lib in temp, e.g., /tmp/lib
• create two zips, one for /tmp/lib, save both to S3
• in Python, connect to lib zip (but do not save to disk), unzip in memory, write file in /tmp; then call R
  • speedup: check if already present in /tmp (hash?)
  • check /tmp is empty
• pro: max usage of /tmp 512MB unzipped for R lib; con: slower starts
  • add option to save R lib to /tmp (let user decide)