Getting GeoDjango Working

[bahoo]

Update September 2019: This issue + comment originally laid out a proposal to make project directory a configurable setting in Zappa, as a way of getting GeoDjango to work more seamlessly in Zappa across different projects. I've found a better solution to my original problem, laid out in a comment below, but I'm going to leave this issue as-is in case it offers guidance for someone else with a similar issue. Mods / maintainer folks, if you've got better solutions for next steps / what to do with this ticket, I'm all ears. 👍

---

A suggestion: make the project directory a configurable option within a stage.

Context

Following davepretty's comment on Slack for getting GeoDjango working, I hardcoded the rpath in one of my libraries I needed.

To make this approach more re-usable for future GeoDjango / Zappa projects, I thought that having the option for a consistent path for the project directory would be helpful. Then, future projects can use the same library files, specify the same path (something generic like /tmp/code/ in my case) and Bob's your uncle.

Currently, Zappa uses /tmp/{0!s}'.format(self.settings.PROJECT_NAME) which a good and fine default for most cases, but being able to use the same directory across different projects would be more portable.

Possible Fix

I've made this change in a branch, here: https://github.com/bahoo/Zappa/commits/specify_project_directory

Your Environment

• Zappa version used: 0.42.0
• Operating System and Python version: macOS Sierra, Python 2.7.12
• The output of pip freeze:

appdirs==1.4.3
argcomplete==1.8.2
base58==0.2.4
boto3==1.4.4
botocore==1.5.40
click==6.7
dj-database-url==0.4.2
Django==1.10.6
django-bootstrap3==8.2.1
django-dotenv==1.4.1
docutils==0.13.1
durationpy==0.4
EasyProcess==0.2.3
future==0.16.0
futures==3.1.1
googlemaps==2.4.6
gunicorn==19.7.1
hjson==2.0.7
jmespath==0.9.3
kappa==0.6.0
lambda-packages==0.16.1
packaging==16.8
placebo==0.8.1
psycopg2==2.7.1
pyparsing==2.2.0
python-dateutil==2.6.0
python-slugify==1.2.4
pyunpack==0.1.2
PyYAML==3.12
requests==2.11.1
s3transfer==0.1.10
six==1.10.0
toml==0.9.2
tqdm==4.14.0
troposphere==1.9.4
Unidecode==0.4.21
Werkzeug==0.12
whitenoise==3.3.0
wsgi-request-logger==0.4.6
zappa==0.42.0

• Your zappa_settings.py:

{
    "production": {
        "aws_region": "us-west-2", 
        "django_settings": "culverstack.settings_zappa", 
        "profile_name": "default",
        "project_directory": "/tmp/code",
        "s3_bucket": "culverstack",
        "slim_handler": true,
        "remote_env": "[ ... snipped ... ]"
    }
}

[bahoo]

Someone requested the libraries; if it helps anyone else, they're here (with rpath set to /tmp/code/lib): www.jonculver.com/zappa-geo-django-libs.zip

[Miserlou]

@bahoo - would you be interested in adding those to https://github.com/Miserlou/lambda-packages ?

[bahoo]

Ooh, yes. 👍 Just now seeing that project, perfect. Might take me a bit to package up a PR but that seems like a much better bet.

[bahoo]

So this might be an elementary set of questions... one interesting aspect of adding these libs is that it's part of Django's included contrib package. The Python libs are something you include / configure rather than pip install ... as a separate package.

But, to make use of them relies on installing some other libraries that get installed through non-pip means.

So a few questions I have:

• is that going to conflict with lambda-package's basic premise / contract?
• if not, once installed, where would we want these files to go?
• do you all think I need any of the rpath trickery I mentioned in my original issue?

This is not my wheelhouse in terms of skillset :D but, happy to take a crack at it if someone can help me have an "a-ha" moment. Thank you!

[grischa]

I think this would also solve my issue, where I have all Python/Django code in $project_dir/src Currently, I cannot see a way to use it as is with Zappa.

[LinuxChristian]

Is there any news on getting this working? I cannot seam to get @bahoo fix to work. I keep getting

django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0", "gdal1.10.0", "gdal1.9.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.

In my zappa tail.

Even after making the changes to my local zappa and setting the GDAL_LIBRARY_PATH='/tmp/code/libs/' in my lambda env variables. I have added @bahoo files to my local repo in a libs folder.

Any suggestions?

[bahoo]

I set these environment variables:

    "LD_LIBRARY_PATH": "/tmp/code/lib/",
    "PROJ_LIB": "/tmp/code/lib/proj4/",

and in my (Django) app's settings file, I set:

GDAL_LIBRARY_PATH = "/tmp/code/lib/libgdal.so.20.1.3"
GEOS_LIBRARY_PATH = "/tmp/code/lib/libgeos_c.so.1"

May be redundant, but that combo works for me. Hope it helps you!

[LinuxChristian]

Thanks for the suggestions. I have tried a couple of things. It got me a bit further to set LD_LIBRARY_PATH=/tmp/code/libs and I also had to create two symlinks, in the libs folder: libgdal.so.20 and libgdal.so. Now I get an error about libgdal.so.20 is missing despite my symlink.

So still not quite there yet but I think I am close. Where in the app do you set the PATH variables for GDAL and GEOS? In your view or model file?

[bahoo]

I add those (the second set of variables) in the Django settings file. I'm not sure where the expected spot would be for e.g. Flask or others.

Don't know much else to offer on your problems unfortunately, but if you have any other specifics I can answer, feel free to drop me an email: jon.c.culver AT gmail

[LinuxChristian]

Thanks! I will post any solutions I find.

[giovannicimolin]

Any news on this? I'm also trying to use GDAL, however @bahoo fix didn't work for me.

[bahoo]

@giovannicimolin Are you using my branch I linked to in my issue? It is probably horrendously out of date at this point, but it did the trick for me.

[giovannicimolin]

@bahoo No, I'm using zappa 0.45.1 master branch. I'll look into your work... I'm also talking to the guys in slack and trying to come up with a more permanent solution (making a lambda package). But I still didn't even manage to run GDAL on the cloud, so it's going to take a while.

[bahoo]

👍 I'm all for more permanent solutions! I'm admittedly a bit out of my depth (my comment above offers my understanding of the problem) but if I can lend a hand, more than happy to help. Would love to have a less janky solution. :)

[samirtendulkar]

I am trying to deploy my Geo-Django app to Zappa 1st I got django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0", "gdal1.10.0", "gdal1.9.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.

I set these environment variables:

"LD_LIBRARY_PATH": "/tmp/code/lib/",
"PROJ_LIB": "/tmp/code/lib/proj4/",

and in my (Django) app's settings file, I set:

GDAL_LIBRARY_PATH = "/tmp/code/lib/libgdal.so.20.1.3" GEOS_LIBRARY_PATH = "/tmp/code/lib/libgeos_c.so.1"

Now I am getting the error OSError: /tmp/code/lib/libgdal.so.20.1.3: cannot open shared object file: No such file or directory

Just a bit of FYI. When I deploy on EC2 on my Ubuntu instance I used in my server_setup.sh file. I have terminated that instance apt-get install -y python3-dev python3-venv python-pip supervisor nginx git python-gdal postgresql postgresql-contrib postgis python3-distutils

Below is my pip freeze

argcomplete==1.9.3
boto3==1.9.82
botocore==1.12.82
certifi==2018.10.15
cfn-flip==1.1.0.post1
chardet==3.0.4
Click==7.0
defusedxml==0.5.0
Django==1.11.15
django-bootstrap-datepicker-plus==3.0.5
django-bootstrap3==11.0.0
django-braces==1.13.0
django-cleanup==2.1.0
django-debug-toolbar==1.10.1
docutils==0.14
durationpy==0.5
future==0.16.0
hjson==3.0.1
idna==2.7
jmespath==0.9.3
kappa==0.6.0
lambda-packages==0.20.0
oauthlib==2.1.0
Pillow==5.3.0
placebo==0.8.2
psycopg2==2.7.5
psycopg2-binary==2.7.5
PyJWT==1.6.4
python-dateutil==2.6.1
python-slugify==1.2.4
python3-openid==3.1.0
pytz==2018.5
PyYAML==3.13
requests==2.20.0
requests-oauthlib==1.0.0
s3transfer==0.1.13
six==1.11.0
social-auth-app-django==2.1.0
social-auth-core==1.7.0
sqlparse==0.2.4
stripe==2.10.1
toml==0.10.0
tqdm==4.19.1
troposphere==2.4.1
Unidecode==1.0.23
urllib3==1.24
uWSGI==2.0.15
Werkzeug==0.14.1
wsgi-request-logger==0.4.6
zappa==0.47.1

[bahoo]

@samirtendulkar and you're using my branch, right? Those code paths wouldn't work without it, I don't think. See the first comment under "Possible Fix"

[samirtendulkar]

Hi bahoo, I am sorry if I sound naive or even a idiot. But could you dumb it down a bit for me. How do I use your branch . I have done what your branch "bahoo/Zappa" did with with my previous steps too

$ pip install zappa
$ zappa init
$ zappa deploy prod

Below is my zappa_settings.json

{
    "prod": {
        "aws_region": "us-east-1",
        "django_settings": "Cool.settings",
        "profile_name": "default",
        "project_name": "cool",
        "runtime": "python3.6",
        "s3_bucket": "coolplaces-t47c5adgt",
        "extra_permissions": [{
            "Effect": "Allow",
            "Action": ["rekognition:*"],
            "Resource": "*"
        }]
    }
}

Should I add "project_directory": "/tmp/code", to the above file. Also which requirements do I need to add to my requirements. python-gdal was previously added to my ubuntu instance. How do I add that in Zappa. Do I need to? along with postgresql postgresql-contrib postgis python3-distutils

Should I do $ zappa undeploy prod then fix the zappa_settings.json and do
$ zappa deploy prod again

Do I need to add pip install lambda-packages ?

Trust me if I get it. You code becomes Fool Proof :)

[bahoo]

@samirtendulkar so it'd be a few steps:

1. Install zappa from my fork; so using pipenv

    pipenv install -e [email protected]:bahoo/zappa.git#egg=zappa
   

2. Add to your zappa_settings.json:

    project_directory": "/tmp/code",
   

3. Download my libraries, and drop them into ./lib/ (beneath your project's root directory)

4. Add these environment variables:

        "LD_LIBRARY_PATH": "/tmp/code/lib/",
        "PROJ_LIB": "/tmp/code/lib/proj4/",
   

5. Finally, add these to your Django settings.py:

    GDAL_LIBRARY_PATH = "/tmp/code/lib/libgdal.so.20.1.3"
    GEOS_LIBRARY_PATH = "/tmp/code/lib/libgeos_c.so.1"
   

Hope that helps!

[samirtendulkar]

Ok I think I almost got it This is what I did zappa undeploy prod pip uninstall zappa delete the zappa_settings.json file

Step 1) $ pip install git+git://github.com/bahoo/Zappa.git#egg=zappa #Since pipenv was another mountain to climb

Step 2) then type zappa init then you will see it automatically creates a file called zappa_settings.json Add to your zappa_settings.json: "project_directory": "/tmp/code", (Copy this as is no "," if this is the last statement) "slim_handler": true (Use this if it gives you a error saying. Your file is too big. Which I am sure it will as the lib file size is 107.1 MB. Also no "," since this was my last statement in my zappa_settings.json no quotes for true)

Step 3) made a directory called lib in my root directory and copied the files to it. See images below https://imgur.com/yyd0ixn (Download the files from step 3 in bahoo's link above)

Step4) (Sorry do this last ) In your AWS lambda console.

 "LD_LIBRARY_PATH": "/tmp/code/lib/",
 "PROJ_LIB": "/tmp/code/lib/proj4/",

Remember do not replace code keep it as is https://imgur.com/a/UZIz65B

Step5) add these to your Django settings.py: (Do not replace code with your path keep it as is)

GDAL_LIBRARY_PATH = "/tmp/code/lib/libgdal.so.20.1.3"
 GEOS_LIBRARY_PATH = "/tmp/code/lib/libgeos_c.so.1"

Step 6) Finally, zappa deploy dev or zappa deploy prod whatever stage you want

Step 7) If it gives you errors do zappa tail it will give you all logs and tell you what the error is fix them and do zappa update

It gave me a error saying bad request told me to add a long amazon link to my allowed host. Did that. Now the next error was to add my data_base. I am doing that. But I feel I got it

This was successful. Thank you bahoo so much for your help and taking the time to dumb it down for me. Also Thank you so much for making geodjango work on zappa

[bahoo]

Great to hear it! Happy to help (as I also had a lot of help getting that far. :D)

[aralmpa]

Hello guys,

I guess I am a bit late (compared to the fix @bahoo proposed and @samirtendulkar tried succesfully) but I have been trying this (external libs for django project when using zappa - geodjango / geospatial libs) again and again and wanted to share the results and concerns and if possible to hear your feedback.

I followed the steps suggested above and I have been succesful in deploying the geodjango app using zappa (bahoo's branch) - Btw I did not set the LD_LIBRARY_PATH and PROJ_LIB and worked ok. It was only when I tried using GeoJSON serilizing that the libs would thow exceptions 'OGR Failure'.

When I tried setting the LD and PROJ env variables then it did not work giving me: File "/var/lang/lib/python3.6/ctypes/__init__.py", line 348, in __init__ self._handle = _dlopen(self._name, mode) OSError: libgeos_c.so.1: cannot open shared object file: No such file or directory Note this is a 500 code (not a 502) and the last error that is returned from zappa tail is: 'NoneType' object is not callable

Finally I noticed that only bahoo's libraries worked and that any other compilation of the libs I tried to make would fail when deployed with zappa giving OSErrors that it was unable to find the libgeos_c.so.1 file although the rest of the settings were identical (GDAL and GEOS path etc) and the fact that the file existed in the same place (inside lib folder). For making the .so files I followed miserlou's guide to django setting up environment using the lambda docker container.

Is it posisble for anybody to give me some info if I am doing something clearly wrong? Is this is a problem with the "making" of the libraries?

Thank you in advance!

[bahoo]

I've found a bit more elegant solution to the GeoDjango problem. Good news: it doesn't rely on the configurable directory hackish-ness, but still (as of September 19th, 2019) relies on a separate fork of Zappa.

TL/DR

Lambda added layers for exactly this sort of reason. Support for it is not in Zappa's master yet, but there's an excellent fork from @jedestep, and pull request, #1842. I've also got a fork that is more up to date with master.

Instructions:

1. Use my fork of zappa:

   pipenv install git+https://github.com/bahoo/Zappa.git#egg=zappa
   

2. Create a layer in the Lambda console, using this zip: http://www.jonculver.com/geodjango-libraries-with-data.zip

3. Add the ARN of that layer to your zappa_settings.json:

   "layers": ["arn:aws:lambda:us-east-1:1234567890:layer:geodjango-libraries-with-data:1"],
   

4. Add this environment variable to your environment:

   "GDAL_DATA": "/opt/lib/gdal_data"
   

5. Add these to your Django settings:

   GDAL_LIBRARY_PATH = "/opt/lib/libgdal.so.20.1.3"
   GEOS_LIBRARY_PATH = "/opt/lib/libgeos_c.so.1"
   

6. Boom. 💥

[aralmpa]

@bahoo excellent suggestion man! I myself was investigating the leverage of Lambda layers in having custom libraries available to your django project. During the search I bumped to your comment!

I did a succesful deployment and Geodjango works fine! Even the GeoJSON serilizing now works OK. Btw, I also tried using my own compiled GDAL, GEOS and PROJ libraries and those worked as well! The deployment package is now light, responsive and much less likely to fail (no requests to S3).

Again thanks man! One final question - not sure why you have added to your geospatial libraries libpq and older versions of the gdal and geos - I did not find those in the instruction of installing geospatial libraries for Geodjango as well as the supporting folders (gdal_data, proj4 etc).

[bahoo]

I think I needed either gdal_data or proj4 to handle transformations between different projections. But those libraries are from a few years ago ( when I first opened the ticket circa summer 2017 ) and it's a bit outside of my technical wheelhouse, so I haven't updated them in a while.

But if you've got a leaner build that doesn't rely on S3 / slim_handler, I'm interested to hear more. I've noticed some intermittent failures with that approach myself.

[aralmpa]

OK I thought so! I have also little familiarity with geospatial libraries - I utilize them in a current project of mine and hope to dive a little deeper. As of now I follow striclty installation instructions from GeoDjango.

With leaner I meant: Since you upload the geospatial binaries in a layer and assuming that your core django project is not that "heavy" you should be well under 50 MB meaning that you can deploy without slim_handler.

If I find something more interesting on my way I will share! Thanks once more!

[jyr]

Jon,

Thanks a lot.

@bahoo

[Alex-Mackie]

Another option (not sure if it is better or worse than the one suggested above) is to install rasterio using pip, then in zappa-settings.json:

        "environment_variables": {
            "GDAL_LIBRARY_PATH": "/var/task/rasterio/.libs/libgdal-7fd1dafb.so.20.5.3",
            "GEOS_LIBRARY_PATH": "/var/task/rasterio/.libs/libgeos_c-1aedf783.so.1.10.2",
            "GDAL_DATA": "/var/task/rasterio/gdal_data"
        }

To find where these lived, I ran:

    # zappa invoke dev "import subprocess; print(subprocess.getoutput(['find / -name *libgdal*']))" --raw
    # zappa invoke dev "import subprocess; print(subprocess.getoutput(['find / -name *libgeos*']))" --raw
    # zappa invoke dev "import subprocess; print(subprocess.getoutput(['find / -name *gcs.csv*']))" --raw

And finally in settings.py:

    GDAL_LIBRARY_PATH = os.environ['GDAL_LIBRARY_PATH']
    GEOS_LIBRARY_PATH = os.environ['GEOS_LIBRARY_PATH']

Not particularly elegant but it works. I saw this solution in someone's blog but now I can't find it. I seem to have full GeoDjango functionality (using RDS Postgres + Postgis as backend).

Django==3.0.2
rasterio==1.1.1
zappa==0.48.2

[shenmander]

@bahoo i am getting this following error when trying to run: pip install git+https://github.com/bahoo/Zappa.git#egg=zappa

ERROR: Command errored out with exit status 1:
     command: 'c:\users\shen2\documents\xx\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shen2\\AppData\\Local\\Temp\\pip-install-927y44ut\\zappa\\setup.py'"'"'; __file__='"'"'C:\\Users\\shen2\\AppData\\Local\\Temp\\pip-install-927y44ut\\zappa\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\shen2\AppData\Local\Temp\pip-install-927y44ut\zappa\pip-egg-info'
         cwd: C:\Users\shen2\AppData\Local\Temp\pip-install-927y44ut\zappa\
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\shen2\AppData\Local\Temp\pip-install-927y44ut\zappa\setup.py", line 8, in <module>
        long_description = readme_file.read()
      File "C:\Users\shen2\Anaconda3\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 75234: character maps to <undefined>
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Does anyone have any idea how to fix this error? p.s. i'm noob

[aralmpa]

Hello @shenmander ,

I cannot help you with this particular issue (as Iit does not ring any bells to me) but I can point to you to the master branch of the Zappa repo (the latest release) includes everything that bahoo's branch has plus fixes, features etc.

I had my Geodjango project ran from the standard repo. Hopefully this will bypass your issue.

[bahoo]

My notes are a bit out of date, yes. Layers are now in master so you should be able to skip my fork, and everything should stick. I don't have any immediate ideas on your issue though, @shenmander, sorry!

[shenmander]

thanks for the replies @aralmpa and @bahoo ! do you guys mind sharing what Python version, django versions you are using? I recently was able to get my project to deploy with Zappa however it required a bunch of de-bugging. Also I can't get it to deploy when slim_handler is true.

edit: I forgot to mention I am using a postgresql db

[aralmpa]

Hello,

My outfit: Python 3.6.8 Django 3.0.4

I had issues with slim_handler (when set to true) as well - and I do not like the idea of making a request to S3 for getting additional code etc so I tried to bypass slim_handler using Lambda layers.

I had a simple geodjango app (I am also using PostgreSQL - should not matter at all btw) with some static files (a lightweight react app) and all this was well under 25MB deployment package. Normally if you do not have big static files you should not be in need of slim_handler (that is my opinion).

[bahoo]

I too prefer to avoid slim_handler if I can help it — though it is a glorious hack! — but just an FYI for folks who may read this in the future: the size of your layer files still contributes to the overall limit. So a 25MB upload + 100MB layer might skirt past Zappa's upload based, built-in warning system to suggest switching to slim_handler, but you might still need to employ it on your project.

[bahoo]

@shenmander I've got a bunch of projects running on Python 3.6 and 3.7; and Django 2.x and 3.0, and I think even some old 1.11.

[harbi]

For those who are still looking for a clean way to enable GeoDjango with Zappa, here's how:

1. Pick a layer that matches your requirements and your AWS region from GeoLambda.
2. In zappa_settings.json add the following:

{
    ...
    "layers": ["arn-for-the-layer"],
    "environment_variables": {
        ...
        "GDAL_LIBRARY_PATH": "/opt/lib/libgdal.so",
        "GDAL_DATA": "/opt/share/gdal",
        "PROJ_LIB": "/opt/share/proj",
    }
    ...
}

3. In your settings.py add the following: GDAL_LIBRARY_PATH = os.environ.get('GDAL_LIBRARY_PATH')
4. zappa update dev 🚀

[bahoo]

Just want to bookmark, looks like Python 3.8's environment bumps up to Amazon Linux 2 ( and they don't even specify the image, which is weak sauce ). I tested the waters on an upgrade and that turned into a big nope. My libraries are out of date ( gdal relies on older versions other libraries, and seems like those went missing in Amazon Linux 2 ). I'll wrestle with it to get 3.8 + Amazon Linux 2 happy at some point, but just flagging for folks who might be curious — I would hold off a while longer. And I'm also taking some mental notes as this ticket / effort project has recently completed its third trip around the sun. 👨‍🚀

[kulor]

For those who are still looking for a clean way to enable GeoDjango with Zappa, here's how:

1. Pick a layer that matches your requirements and your AWS region from GeoLambda.
2. In zappa_settings.json add the following:

{
    ...
    "layers": ["arn-for-the-layer"],
    "environment_variables": {
        ...
        "GDAL_LIBRARY_PATH": "/opt/lib/libgdal.so",
        "GDAL_DATA": "/opt/share/gdal",
        "PROJ_LIB": "/opt/share/proj",
    }
    ...
}

1. In your settings.py add the following: GDAL_LIBRARY_PATH = os.environ.get('GDAL_LIBRARY_PATH')
2. zappa update dev 🚀

@harbi You saved my sanity with this, thanks. Some follow-ups for internet stragglers like me. I got stuck with various options so I ended up using one of the images found in https://github.com/lambgeo/docker-lambda (specifically arn:aws:lambda:eu-west-2:524387336408:layer:gdal24-al2:3 (ensure you change your region)). I then tweaked my config to:

...
"environment_variables": {
      "GDAL_LIBRARY_PATH": "/opt/lib/libgdal.so",
      "GEOS_LIBRARY_PATH": "/opt/lib/libgeos_c.so",
      "GDAL_DATA": "/opt/share/gdal",
    }
...

and my settings.py had:

GDAL_LIBRARY_PATH = os.environ.get('GDAL_LIBRARY_PATH')
GEOS_LIBRARY_PATH = os.environ.get('GEOS_LIBRARY_PATH')

[HaithamKhedrSalem]

@kulor @harbi Hello, sorry I am new at django and zappa. I did all the above steps that you both mentioned with both GeoLambda and https://github.com/lambgeo/docker-lambda and I considered the region for the lambda layer but still getting this error [ERROR] ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.1.0", "gdal3.0.0", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.

I am using [email protected]/[email protected]/[email protected] . Any suggestions please!!! I am going to lose my mind!!!

[kulor]

@HaithamKhedrSalem When I run into issues with Lambda binaries, I tend to do some directory listing to debug what's available e.g ls -la /opt/share/.

Is it possible to share your zappa config to see if I can help?

[HaithamKhedrSalem]

@kulor Thanks for ur reply. I have question what do u mean by doing some directory listing. Where should I run this ls -la command? Here is a sample of my zappa_settings.json

 {
 "dev": {
   "aws_region": "us-east-1",
   "django_settings": "main.settings",
   "profile_name": "default",
    "project_name": "project-api",
   "runtime": "python3.8",
   "s3_bucket": "payloads",
   "layers": [
     "arn:aws:lambda:us-east-1:524387336408:layer:gdal24-al2:3"
    ],
   "environment_variables": {
     "APP_ENV": "dev",
     "ALLOWED_HOSTS": "localhost",
     "DEBUG": "False",
      "GDAL_LIBRARY_PATH": "/opt/lib/libgdal.so",
      "GEOS_LIBRARY_PATH": "/opt/lib/libgeos_c.so",
      "GDAL_DATA": "/opt/share/gdal",
     "PROJ_LIB": "/opt/share/proj"
   }
 }
}

[kulor]

@kulor Thanks for ur reply. I have question what do u mean by doing some directory listing. Where should I run this ls -la command? Here is a sample of my zappa_settings.json

 {
 "dev": {
   "aws_region": "us-east-1",
   "django_settings": "main.settings",
   "profile_name": "default",
    "project_name": "project-api",
   "runtime": "python3.8",
   "s3_bucket": "payloads",
   "layers": [
     "arn:aws:lambda:us-east-1:524387336408:layer:gdal24-al2:3"
    ],
   "environment_variables": {
     "APP_ENV": "dev",
     "ALLOWED_HOSTS": "localhost",
     "DEBUG": "False",
      "GDAL_LIBRARY_PATH": "/opt/lib/libgdal.so",
      "GEOS_LIBRARY_PATH": "/opt/lib/libgeos_c.so",
      "GDAL_DATA": "/opt/share/gdal",
     "PROJ_LIB": "/opt/share/proj"
   }
 }
}

This all looks fine. One thing that I've spotted, though is that you mentioned using [email protected] but the layer specified is for [email protected] (gdal24). I'd suggest trying downgrading to 3.3 and using something like arn:aws:lambda:us-east-1:524387336408:layer:gdal33-al2:2 as your layer.