Zappa
Zappa copied to clipboard
Miserlou
Conda Virtual Environments?
Great project, btw! I see myself making liberal use of this.
I just wanted to check and see if there are any issues with using conda virtual environments? I successfully create a conda virtual environment (just flask and its dependencies), activate the new environment, and then maneuver to my project directory and then "zappa deploy project" where I have the zappa_settings.json configured correctly the best I can tell.
conda create -n enviro python=2.7 flask
source activate enviro
zappa deploy project
but then...
Packaging project as zip...
Zappa requires an active virtual environment.
Also, unrelated, when specifying a domain should it be the same as the hostname in app.py? For example "www.abc.com". I have my domain up on Route 53.
There is an unmerged pull-request that adds Conda support. I've neglected it for too long already, so this is a good opportunity to re-address it: https://github.com/Miserlou/Zappa/pull/108
The main thing is that I personally just don't know enough about Conda to evaluate this PR. It also needs tests. Do you think you can take a look at it and let me know what you think?
Sure, I'm glad to take a look. As an anaconda user, I'm partial to conda for convenience. I'll look for the right dev branch and see if I can do some functionality testing at least.
This is in the pull-request branch, not dev: https://github.com/mathieu1/Zappa/tree/conda_support
Would be really interesting to know! I used this to install: https://anaconda.org/mathieu/zappa so there is some progress, but after deploy I ran into a bug very similar to this: https://github.com/Miserlou/Zappa/issues/260 Does anyone have a clue about possible fixes?
One caveat: I did not use the latest version, so I did
conda install -c mathieu zappa
and not
conda install -c mathieu zappa=0.39.1.dev
(dependecies misssing...)
Update: upgraded to 0.39.1.dev, and can confirm issue https://github.com/Miserlou/Zappa/issues/260 Sadly. :-(
And to be more precise: "module initialization error: Attempted relative import in non-package"
I use Flask 0.12 istalled via Conda main channel.
Sorry, any update on using zappa with anaconda? Looks like there are still issues with this. Thanks.
Same here, conda environment is useful for windows users. It would be great to not have this issue.
upvoting. zappa/miniconda compatibility on, like any OS would be a huge help.
With the addition of Conda Pack it seems like this might be an option to get conda environment up? I suppose there will be complications if the conda environment was created on Windows and then tried to be unpacked into the Lambda env.
It seems like this "hacky" solution works: (just manually set the VIRTUAL_ENV environment variable) https://github.com/Miserlou/Zappa/issues/1030
I just tried using a conda environment to install both conda and pip packages. When doing a zappa update with a slim handler, the tar.gz file ends up being more than 3x the size for some reason. It also takes much, much longer to process. In addition there is something going on because I keep getting Module not found errors for jinja2 (installed and working in the conda env) coming from an import flask statement when deployed. Everything is working locally though.
Has there been any update on fully adding support for conda environments? This would be very helpful to become a priority because it seems like many newer packages are opting to only release via conda channels and not pip.
A question for conda users: Is this in the context of serving machine learning models on AWS Lambda?
If so, I wrote a library called Thampi which uploads your model and your conda environment to AWS Lambda and abstracts away the DevOps part of model serving. The caveat is that your conda requirements file has to be manually written.
Conceptually, you can work on Mac(Windows not tested, but seems possible) and Thampi will use docker to load your requirements file and zappa deploy for you.
This would not be possible without the amazing work of zappa. So thank you!
Hi there it seems to be possible to use Zappa with Conda by following these instructions : https://github.com/Miserlou/Zappa/issues/1030#issuecomment-319907354
Apologies if I am missing the point of this issue.
@Miserlou could you please give your opinion on whether or not this workaround is wise?
I'll add in a request for support in 2019. Not having conda support makes the case for supporting zappa difficult :(
As @johntiger1 says, its easy to fix. I just had to run the following command
conda create -n zzz python=3.7
conda activate zzz
# Install required libraries, test your application
(zzz) ➜ which python
/home/lokesh/miniconda3/envs/zzz/bin/python
# Remove the /bin/python and copy the rest
export VIRUTAL_ENV=/home/lokesh/miniconda3/envs/zzz
and it worked !
I'm sorry, I never used conda, so I'll need a bit of guidance on this one if we're going to get the support in. Can someone point me to a tutorial and either describe me the issues or point me to the comments on this thread that identify the still existing ones?
@lokeshh if you can install zappa from master, can you check if the newly merged --no_venv option removes the need to export the VIRTUAL_ENV?
@jneves No, it doesn't work. I ran the latest zappa from master. Here's the error message I got https://gist.github.com/lokeshh/ba6eac37748bff04b5f3bdd1b8d0779f
@lokeshh if you can install zappa from master, can you check if the newly merged --no_venv option removes the need to export the VIRTUAL_ENV?
can you specify the command? I would like to try it out! FYI for me the option exporting the virtual environment worked!
And for windows users
conda create -n ds python=3.7 conda activate ds # Install required libraries, test your application (ds) ➜ where python C:\Users\user_name\Anaconda3\envs\ds\python.exe set VIRUTAL_ENV=C:\Users\user_name\Anaconda3\envs\dsand it worked!
