docker-lambda missing `liblzma.so` in nodejs environment

I am using the ARN for a lambda layer (arn:aws:lambda:us-west-2:524387336408:layer:gdal32:3) and adding it to my lambda function, with my lambda function in nodejs. I am trying to use ogr2ogr in the function but I'm getting the following error that is most likely related to the GDAL:

ogr2ogr: error while loading shared libraries: libpcre.so.0: cannot open shared object file: No such file or directory\n

I tried to set GDAL_DATA and PROJ_LIB as environmental variables on my lambda function (I just went to Environment Variables on the UI of the lambda function and added the two as environmental variables), but I am still getting the same error. Am I doing something wrong?

Feb 24 '21 18:02 ferily7

@ferily7 thanks for your issue.

What version of nodejs are you using?

Feb 24 '21 18:02 vincentsarago

Note, the latest version in us-west-2 are arn:aws:lambda:us-west-2:524387336408:layer:gdal32-al2:4 or arn:aws:lambda:us-west-2:524387336408:layer:gdal32:4

https://github.com/lambgeo/docker-lambda/blob/master/layer.json#L557

Feb 24 '21 19:02 vincentsarago

I'm using Node.js 14.x for the lambda function! I also changed the version to the one you listed, still getting the same error though

Feb 24 '21 19:02 ferily7

for Node.js 14 you have to use arn:aws:lambda:us-west-2:524387336408:layer:gdal32-al2:4

Feb 24 '21 19:02 vincentsarago

Gotcha! I have my environmental variables set up like this in the lambda,

variables

is this the right way to add the environmental variables?

Feb 24 '21 19:02 ferily7

Actually after I changed the version, I get a different error now Error: ogr2ogr: error while loading shared libraries: liblzma.so.5: cannot open shared object file: No such file or directory

Feb 24 '21 19:02 ferily7

yes I'm seeing this too! I never tried nodejs but seem that there is a difference between the docker image used for the build and the one running the lambda 🤷‍♂️

The nodejs image from aws seems to have liblzma at the right place tho 🤷‍♂️

docker run --rm -it lambgeo/lambda-gdal:3.2-al2 bash
bash-4.2# ldd /opt/bin/ogr2ogr | grep "liblzma"
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007ff1d104b000)

docker run --rm -it --entrypoint "" amazon/aws-lambda-nodejs:14 bash
...
Status: Downloaded newer image for amazon/aws-lambda-nodejs:14
bash-4.2# ls -la /lib64/ | grep "liblzma"
lrwxrwxrwx  1 root root       16 Feb 23 11:02 liblzma.so.5 -> liblzma.so.5.2.2
-rwxr-xr-x  1 root root   157400 Jul 26  2018 liblzma.so.5.2.2

Feb 24 '21 20:02 vincentsarago

Hmm so does that mean that the library is there then?

Feb 24 '21 20:02 ferily7

Hmm so does that mean that the library is there then?

the library is in the official AWS Lambda NodeJS 14x image .... but maybe not in the lambda runtime which is really weird!

Feb 24 '21 20:02 vincentsarago

Strange... Is there any way I could resolve this issue?

Feb 24 '21 20:02 ferily7

if you could list all the files found in /lib64 and /lib we could compare with what we have in our building env.

It's always hard to keep a track of what is shipped in the AWS runtime images 😭

Feb 24 '21 21:02 vincentsarago

How would I find the list of all the files in /lib64 and /lib?

Feb 24 '21 21:02 ferily7

create a lambda function that return the content of /lib64!

Feb 24 '21 21:02 vincentsarago

I'm not too sure how to do that... Is that the only way to get the files in /lib64 or /lib?

Feb 24 '21 21:02 ferily7

Is that the only way to get the files in /lib64 or /lib?

Here is how I do in it python

import json
from pathlib import Path

def lambda_handler(event, context):

    p = Path('/lib64').glob('**/*.so')
    files = [str(x) for x in p]
    print(files)
    return {
        'statusCode': 200,
        'body': json.dumps(files)
    }

/lib64/libnss_files-2.26.so
/lib64/libSegFault.so
/lib64/libmemusage.so
/lib64/libfreeblpriv3.so
/lib64/libpcprofile.so
/lib64/libnspr4.so
/lib64/librt-2.26.so
/lib64/libanl-2.26.so
/lib64/p11-kit-trust.so
/lib64/libfreebl3.so
/lib64/libnssckbi.so
/lib64/libBrokenLocale-2.26.so
/lib64/libthread_db-1.0.so
/lib64/libc-2.26.so
/lib64/libcidn-2.26.so
/lib64/libnss_compat-2.26.so
/lib64/libnsl-2.26.so
/lib64/libresolv-2.26.so
/lib64/libnssutil3.so
/lib64/libplc4.so
/lib64/ld-2.26.so
/lib64/p11-kit-proxy.so
/lib64/libplds4.so
/lib64/libnss_dns-2.26.so
/lib64/libutil-2.26.so
/lib64/libdl-2.26.so
/lib64/libm-2.26.so
/lib64/libpthread-2.26.so
/lib64/libmvec-2.26.so
/lib64/gconv/IBM904.so
/lib64/gconv/ANSI_X3.110.so
/lib64/gconv/IBM864.so
/lib64/gconv/IBM4971.so
/lib64/gconv/HP-TURKISH8.so
/lib64/gconv/IBM1157.so
/lib64/gconv/TCVN5712-1.so
/lib64/gconv/ISO_5427.so
/lib64/gconv/IBM905.so
/lib64/gconv/ISO8859-15.so
/lib64/gconv/GBK.so
/lib64/gconv/IBM285.so
/lib64/gconv/ISO-2022-KR.so
/lib64/gconv/MAC-IS.so
/lib64/gconv/IBM423.so
/lib64/gconv/EBCDIC-ES-S.so
/lib64/gconv/IBM290.so
/lib64/gconv/CP1258.so
/lib64/gconv/HP-GREEK8.so
/lib64/gconv/IBM4899.so
/lib64/gconv/ISO-IR-197.so
/lib64/gconv/IBM852.so
/lib64/gconv/IBM1025.so
/lib64/gconv/IBM281.so
/lib64/gconv/ISO8859-6.so
/lib64/gconv/ISIRI-3342.so
/lib64/gconv/ISO8859-10.so
/lib64/gconv/IBM1137.so
/lib64/gconv/IBM1167.so
/lib64/gconv/IBM1122.so
/lib64/gconv/DEC-MCS.so
/lib64/gconv/IBM855.so
/lib64/gconv/GB18030.so
/lib64/gconv/EUC-KR.so
/lib64/gconv/CP1255.so
/lib64/gconv/ISO8859-5.so
/lib64/gconv/EUC-CN.so
/lib64/gconv/EBCDIC-ES-A.so
/lib64/gconv/IBM865.so
/lib64/gconv/IBM868.so
/lib64/gconv/CP770.so
/lib64/gconv/PT154.so
/lib64/gconv/IBM918.so
/lib64/gconv/IBM1153.so
/lib64/gconv/IBM1371.so
/lib64/gconv/EBCDIC-CA-FR.so
/lib64/gconv/IBM1148.so
/lib64/gconv/ISO8859-8.so
/lib64/gconv/CP774.so
/lib64/gconv/INIS-8.so
/lib64/gconv/IBM856.so
/lib64/gconv/SAMI-WS2.so
/lib64/gconv/ISO-2022-JP-3.so
/lib64/gconv/MIK.so
/lib64/gconv/IBM273.so
/lib64/gconv/UNICODE.so
/lib64/gconv/IBM857.so
/lib64/gconv/IBM1123.so
/lib64/gconv/CP773.so
/lib64/gconv/ISO_5428.so
/lib64/gconv/UHC.so
/lib64/gconv/IBM870.so
/lib64/gconv/IBM1004.so
/lib64/gconv/MAC-SAMI.so
/lib64/gconv/IBM16804.so
/lib64/gconv/IBM274.so
/lib64/gconv/SJIS.so
/lib64/gconv/IBM901.so
/lib64/gconv/HP-THAI8.so
/lib64/gconv/TIS-620.so
/lib64/gconv/IBM1155.so
/lib64/gconv/IBM297.so
/lib64/gconv/IBM1149.so
/lib64/gconv/IBM803.so
/lib64/gconv/EUC-JISX0213.so
/lib64/gconv/IBM12712.so
/lib64/gconv/VISCII.so
/lib64/gconv/IBM1146.so
/lib64/gconv/GOST_19768-74.so
/lib64/gconv/UTF-7.so
/lib64/gconv/UTF-32.so
/lib64/gconv/BIG5HKSCS.so
/lib64/gconv/JOHAB.so
/lib64/gconv/IBM277.so
/lib64/gconv/SHIFT_JISX0213.so
/lib64/gconv/ISO8859-4.so
/lib64/gconv/IBM1008.so
/lib64/gconv/CP1253.so
/lib64/gconv/CP1257.so
/lib64/gconv/IBM256.so
/lib64/gconv/KOI-8.so
/lib64/gconv/KOI8-U.so
/lib64/gconv/libJIS.so
/lib64/gconv/CP775.so
/lib64/gconv/T.61.so
/lib64/gconv/IBM9030.so
/lib64/gconv/EBCDIC-IT.so
/lib64/gconv/IBM1141.so
/lib64/gconv/IBM1144.so
/lib64/gconv/IBM875.so
/lib64/gconv/GEORGIAN-ACADEMY.so
/lib64/gconv/ARMSCII-8.so
/lib64/gconv/IBM935.so
/lib64/gconv/IBM1133.so
/lib64/gconv/HP-ROMAN8.so
/lib64/gconv/CP737.so
/lib64/gconv/ISO_6937.so
/lib64/gconv/IBM874.so
/lib64/gconv/ISO8859-9.so
/lib64/gconv/IBM850.so
/lib64/gconv/EBCDIC-FR.so
/lib64/gconv/ISO8859-11.so
/lib64/gconv/GREEK7-OLD.so
/lib64/gconv/IBM1364.so
/lib64/gconv/EBCDIC-FI-SE.so
/lib64/gconv/IBM278.so
/lib64/gconv/IBM275.so
/lib64/gconv/IBM1163.so
/lib64/gconv/GREEK-CCITT.so
/lib64/gconv/CP1250.so
/lib64/gconv/GEORGIAN-PS.so
/lib64/gconv/IBM863.so
/lib64/gconv/ISO8859-13.so
/lib64/gconv/CP772.so
/lib64/gconv/ISO-2022-JP.so
/lib64/gconv/RK1048.so
/lib64/gconv/CP1254.so
/lib64/gconv/ISO_5427-EXT.so
/lib64/gconv/IBM1161.so
/lib64/gconv/IBM1008_420.so
/lib64/gconv/ISO_6937-2.so
/lib64/gconv/CP10007.so
/lib64/gconv/MACINTOSH.so
/lib64/gconv/IBM1046.so
/lib64/gconv/EBCDIC-US.so
/lib64/gconv/KOI8-R.so
/lib64/gconv/CP932.so
/lib64/gconv/MAC-CENTRALEUROPE.so
/lib64/gconv/IBM1156.so
/lib64/gconv/EBCDIC-FI-SE-A.so
/lib64/gconv/KOI8-T.so
/lib64/gconv/IBM038.so
/lib64/gconv/ISO8859-16.so
/lib64/gconv/IBM1132.so
/lib64/gconv/EUC-JP-MS.so
/lib64/gconv/IBM1142.so
/lib64/gconv/CP1251.so
/lib64/gconv/IBM1164.so
/lib64/gconv/IBM1140.so
/lib64/gconv/IBM939.so
/lib64/gconv/libISOIR165.so
/lib64/gconv/IBM1147.so
/lib64/gconv/CSN_369103.so
/lib64/gconv/ISO-2022-CN.so
/lib64/gconv/IBM858.so
/lib64/gconv/BRF.so
/lib64/gconv/IBM921.so
/lib64/gconv/IBM943.so
/lib64/gconv/IBM284.so
/lib64/gconv/CP1252.so
/lib64/gconv/libKSC.so
/lib64/gconv/EBCDIC-AT-DE-A.so
/lib64/gconv/IBM851.so
/lib64/gconv/CP1256.so
/lib64/gconv/IBM902.so
/lib64/gconv/IBM1112.so
/lib64/gconv/IBM930.so
/lib64/gconv/UTF-16.so
/lib64/gconv/HP-ROMAN9.so
/lib64/gconv/IBM1145.so
/lib64/gconv/IBM866.so
/lib64/gconv/GBBIG5.so
/lib64/gconv/IBM4517.so
/lib64/gconv/IBM9066.so
/lib64/gconv/IBM871.so
/lib64/gconv/ISO8859-2.so
/lib64/gconv/ISO-IR-209.so
/lib64/gconv/IBM1399.so
/lib64/gconv/IBM861.so
/lib64/gconv/LATIN-GREEK-1.so
/lib64/gconv/IBM932.so
/lib64/gconv/IBM1124.so
/lib64/gconv/IEC_P27-1.so
/lib64/gconv/libJISX0213.so
/lib64/gconv/ASMO_449.so
/lib64/gconv/IBM5347.so
/lib64/gconv/IBM1160.so
/lib64/gconv/IBM420.so
/lib64/gconv/IBM9448.so
/lib64/gconv/MAC-UK.so
/lib64/gconv/IBM891.so
/lib64/gconv/IBM1154.so
/lib64/gconv/EBCDIC-DK-NO.so
/lib64/gconv/libGB.so
/lib64/gconv/EBCDIC-AT-DE.so
/lib64/gconv/IBM933.so
/lib64/gconv/ISO8859-3.so
/lib64/gconv/ECMA-CYRILLIC.so
/lib64/gconv/INIS.so
/lib64/gconv/ISO8859-1.so
/lib64/gconv/EBCDIC-PT.so
/lib64/gconv/IBM424.so
/lib64/gconv/IBM1130.so
/lib64/gconv/LATIN-GREEK.so
/lib64/gconv/GBGBK.so
/lib64/gconv/EUC-TW.so
/lib64/gconv/IBM903.so
/lib64/gconv/IBM1162.so
/lib64/gconv/INIS-CYRILLIC.so
/lib64/gconv/IBM880.so
/lib64/gconv/libCNS.so
/lib64/gconv/IBM437.so
/lib64/gconv/ISO_11548-1.so
/lib64/gconv/IBM1143.so
/lib64/gconv/IBM500.so
/lib64/gconv/IBM1047.so
/lib64/gconv/IBM862.so
/lib64/gconv/ISO8859-7.so
/lib64/gconv/EUC-JP.so
/lib64/gconv/IBM869.so
/lib64/gconv/ISO-2022-CN-EXT.so
/lib64/gconv/ISO_10367-BOX.so
/lib64/gconv/IBM1129.so
/lib64/gconv/CP1125.so
/lib64/gconv/IBM860.so
/lib64/gconv/IBM280.so
/lib64/gconv/EBCDIC-DK-NO-A.so
/lib64/gconv/ISO8859-9E.so
/lib64/gconv/IBM1390.so
/lib64/gconv/IBM037.so
/lib64/gconv/IBM1166.so
/lib64/gconv/IBM937.so
/lib64/gconv/ISO646.so
/lib64/gconv/EBCDIC-UK.so
/lib64/gconv/CWI.so
/lib64/gconv/GREEK7.so
/lib64/gconv/ISO_2033.so
/lib64/gconv/IBM4909.so
/lib64/gconv/KOI8-RU.so
/lib64/gconv/ISO8859-14.so
/lib64/gconv/IBM922.so
/lib64/gconv/BIG5.so
/lib64/gconv/IBM1097.so
/lib64/gconv/NATS-SEFI.so
/lib64/gconv/IBM1158.so
/lib64/gconv/TSCII.so
/lib64/gconv/CP771.so
/lib64/gconv/IBM866NAV.so
/lib64/gconv/IBM1388.so
/lib64/gconv/IBM1026.so
/lib64/gconv/EBCDIC-ES.so
/lib64/gconv/NATS-DANO.so
/lib64/gconv/EBCDIC-IS-FRISS.so
/lib64/audit/sotruss-lib.so
/lib64/krb5/plugins/tls/k5tls.so
/lib64/openssl/engines/libaep.so
/lib64/openssl/engines/lib4758cca.so
/lib64/openssl/engines/libubsec.so
/lib64/openssl/engines/libpadlock.so
/lib64/openssl/engines/libchil.so
/lib64/openssl/engines/libcapi.so
/lib64/openssl/engines/libatalla.so
/lib64/openssl/engines/libcswift.so
/lib64/openssl/engines/libnuron.so
/lib64/openssl/engines/libgmp.so
/lib64/openssl/engines/libsureware.so
/lib64/security/pam_cap.so
/lib64/pkcs11/p11-kit-trust.so

as you can see in ☝️ there is not liblzma in the lambda environment 🤷‍♂️

ahhhhh if I run ldd /opt/bin/ogr2ogr in the lambda env (python3.8) here is what I have 👇

	linux-vdso.so.1 (0x00007ffe70e9e000)
	libgdal.so => /opt/bin/../lib/libgdal.so (0x00007f1e1f62e000)
	libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f1e1f1d9000)
	libgeos_c.so.1 => /opt/bin/../lib/libgeos_c.so.1 (0x00007f1e1efa6000)
	libwebp.so.7 => /opt/bin/../lib/libwebp.so.7 (0x00007f1e1ed3c000)
	libexpat.so.1 => /opt/bin/../lib/libexpat.so.1 (0x00007f1e1eb0c000)
	libopenjp2.so.7 => /opt/bin/../lib/libopenjp2.so.7 (0x00007f1e1e8b4000)
	libnetcdf.so.18 => /opt/bin/../lib/libnetcdf.so.18 (0x00007f1e1e581000)
	libhdf5.so.200 => /opt/bin/../lib/libhdf5.so.200 (0x00007f1e1deaf000)
	libmfhdf.so.0 => /opt/bin/../lib/libmfhdf.so.0 (0x00007f1e1dc85000)
	libdf.so.0 => /opt/bin/../lib/libdf.so.0 (0x00007f1e1d9d4000)
	libjpeg.so.62 => /opt/bin/../lib/libjpeg.so.62 (0x00007f1e1d740000)
	libgeotiff.so.5 => /opt/bin/../lib/libgeotiff.so.5 (0x00007f1e1d50c000)
	libpng16.so.16 => /opt/bin/../lib/libpng16.so.16 (0x00007f1e1d2db000)
	libpq.so.5 => /opt/bin/../lib/libpq.so.5 (0x00007f1e1d092000)
	libzstd.so.1 => /opt/bin/../lib/libzstd.so.1 (0x00007f1e1ce0a000)
	libproj.so.19 => /opt/bin/../lib/libproj.so.19 (0x00007f1e1c946000)
	libsqlite3.so.0 => /opt/bin/../lib/libsqlite3.so.0 (0x00007f1e1c635000)
	libtiff.so.5 => /opt/bin/../lib/libtiff.so.5 (0x00007f1e1c3b7000)
	libdeflate.so.0 => /opt/bin/../lib/libdeflate.so.0 (0x00007f1e1c1aa000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f1e1bf95000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1e1bd77000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f1e1bb6f000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f1e1b96b000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f1e1b707000)
	libcurl.so.4 => /opt/bin/../lib/libcurl.so.4 (0x00007f1e1b48e000)
	libxml2.so.2 => /opt/bin/../lib/libxml2.so.2 (0x00007f1e1b144000)
	liblzma.so.5 => /var/lang/lib/liblzma.so.5 (0x00007f1e1af1e000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f1e1abde000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f1e1a85c000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1e1a646000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f1e1a29b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1e201e5000)
	libgeos-3.9.0.so => /opt/lib/libgeos-3.9.0.so (0x00007f1e19ea4000)
	libhdf5_hl.so.200 => /opt/lib/libhdf5_hl.so.200 (0x00007f1e19c82000)
	libsz.so.2 => /opt/lib/libsz.so.2 (0x00007f1e19a6e000)
	libnghttp2.so.14 => /opt/lib/libnghttp2.so.14 (0x00007f1e19845000)
	libssl.so.10 => /lib64/libssl.so.10 (0x00007f1e195d6000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f1e1938a000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f1e190a6000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f1e18ea2000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f1e18c71000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f1e18a62000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f1e1885e000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f1e18648000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f1e18421000)

liblzma.so.5 => /var/lang/lib/liblzma.so.5 (0x00007f1e1af1e000)

it's interesting that this is not in /lib64. I guess nodejs env don't have /var/lang/lib/liblzma.so.5.

To Do

ldd /opt/bin/ogr2ogr in the NodeJS lambda env

Feb 24 '21 21:02 vincentsarago

Alright I did it


const { exec } = require("child_process");

exports.handler = (event) => {
    
    exec("ldd /opt/bin/ogr2ogr", (error, stdout, stderr) => {
        if (error) {
            console.log(`error: ${error.message}`);
        }
        if (stderr) {
            console.log(`stderr: ${stderr}`);
        }
        console.log(`stdout: ${stdout}`);

        // TODO implement
        const response = {
            statusCode: 200,
            body: JSON.stringify(stdout),
        };
        return response;
    });

};

	libgdal.so => /opt/bin/../lib/libgdal.so (0x00007f9c72879000)
	libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f9c72424000)
	libgeos_c.so.1 => /opt/bin/../lib/libgeos_c.so.1 (0x00007f9c721f1000)
	libwebp.so.7 => /opt/bin/../lib/libwebp.so.7 (0x00007f9c71f87000)
	libexpat.so.1 => /opt/bin/../lib/libexpat.so.1 (0x00007f9c71d57000)
	libopenjp2.so.7 => /opt/bin/../lib/libopenjp2.so.7 (0x00007f9c71aff000)
	libnetcdf.so.18 => /opt/bin/../lib/libnetcdf.so.18 (0x00007f9c717cc000)
	libhdf5.so.200 => /opt/bin/../lib/libhdf5.so.200 (0x00007f9c710fa000)
	libmfhdf.so.0 => /opt/bin/../lib/libmfhdf.so.0 (0x00007f9c70ed0000)
	libdf.so.0 => /opt/bin/../lib/libdf.so.0 (0x00007f9c70c1f000)
	libjpeg.so.62 => /opt/bin/../lib/libjpeg.so.62 (0x00007f9c7098b000)
	libgeotiff.so.5 => /opt/bin/../lib/libgeotiff.so.5 (0x00007f9c70757000)
	libpng16.so.16 => /opt/bin/../lib/libpng16.so.16 (0x00007f9c70526000)
	libpq.so.5 => /opt/bin/../lib/libpq.so.5 (0x00007f9c702dd000)
	libzstd.so.1 => /opt/bin/../lib/libzstd.so.1 (0x00007f9c70055000)
	libproj.so.19 => /opt/bin/../lib/libproj.so.19 (0x00007f9c6fb91000)
	libsqlite3.so.0 => /opt/bin/../lib/libsqlite3.so.0 (0x00007f9c6f880000)
	libtiff.so.5 => /opt/bin/../lib/libtiff.so.5 (0x00007f9c6f602000)
	libdeflate.so.0 => /opt/bin/../lib/libdeflate.so.0 (0x00007f9c6f3f5000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f9c6f1e0000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9c6efc2000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f9c6edba000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f9c6ebb6000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f9c6e952000)
	libcurl.so.4 => /opt/bin/../lib/libcurl.so.4 (0x00007f9c6e6d9000)
	libxml2.so.2 => /opt/bin/../lib/libxml2.so.2 (0x00007f9c6e38f000)
	liblzma.so.5 => not found
	libm.so.6 => /lib64/libm.so.6 (0x00007f9c6e04f000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f9c6dccd000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f9c6dab7000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9c6d70c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9c73430000)
	liblzma.so.5 => not found
	libgeos-3.9.0.so => /opt/lib/libgeos-3.9.0.so (0x00007f9c6d315000)
	libhdf5_hl.so.200 => /opt/lib/libhdf5_hl.so.200 (0x00007f9c6d0f3000)
	libsz.so.2 => /opt/lib/libsz.so.2 (0x00007f9c6cedf000)
	libnghttp2.so.14 => /opt/lib/libnghttp2.so.14 (0x00007f9c6ccb6000)
	libssl.so.10 => /lib64/libssl.so.10 (0x00007f9c6ca47000)
	liblzma.so.5 => not found
	liblzma.so.5 => not found
	liblzma.so.5 => not found
	liblzma.so.5 => not found
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f9c6c7fb000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f9c6c517000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f9c6c313000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f9c6c0e2000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f9c6bed3000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f9c6bccf000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f9c6bab9000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f9c6b892000)

Only liblzma is missing 🤷‍♂️ I think I could add it manually in the built but I just hope that other env don't have other missing libs

Feb 24 '21 21:02 vincentsarago

Ah I see so it's missing from the lib then? I'm just wondering if that one is added, will there be any more missing libraries that I will get an error for

Feb 24 '21 22:02 ferily7

Ah I see so it's missing from the lib then?

yeah for an unknown reason, liblzma is present in AWS official build images, in python lambda env ... but not in NodeJS lambda env 🤷‍♂️

It seems to be the only one missing

Feb 24 '21 22:02 vincentsarago

What is the next step to get this resolved then?

Feb 25 '21 17:02 ferily7

I'm still not 💯 how this has to be resolved. we could compile liblzma ourself but it's really weird that it is missing in the nodejs14 lambda runtime image. Might be worth to open a ticket on AWS

May 01 '21 02:05 vincentsarago

I will see about submitting a ticket to AWS (not sure where for these types of issues).

In the meantime, my app ships a layer that just contains the two missing files:

/lib64/liblzma.so.5.2.2
/lib64/liblzma.so.5

extracted from the official NodeJS Lambda image amazon/aws-lambda-nodejs:14.

Aug 22 '21 05:08 danielsharvey

fixed in latest release of AWS Lambda nodejs16

May 24 '22 16:05 vincentsarago

🤦 no we still have the issue because the official docker image do not match the lambda environment

Jun 01 '22 20:06 vincentsarago

Any update on this issue? I have been trying to add liblzma or disable it the gdal build command to even just use gdalinfo from the node process, but nothing seems to work.

Sep 13 '22 00:09 JackTiber

@JackTiber I didn't had time yet to work on this.

you could try doing https://github.com/lambgeo/docker-lambda/issues/37#issuecomment-903217641

The issue is that the build image has liblzma but the lambda end doesn't for node js. We need to put liblzma.so in the package/layer to fix this.

Sep 13 '22 08:09 vincentsarago

Thanks @vincentsarago for getting back to me! Yup I actually did that yesterday after finding this thread and it worked with the lambda for nodejs:16. @danielsharvey if you were able to submit a ticket, let me know and I will drop one in as well. Spent entirely too long trying to nail down the missing library issue because it is documented as being there!

Sep 13 '22 18:09 JackTiber

Hi @JackTiber @danielsharvey @vincentsarago , I have also run into this issue. I am a little stuck with trying to implement the solution metioned in #37 (comment). I am having trouble figuring out how to extract the files from the official aws lambda image for amazon/aws-lambda-nodejs:14 so that I can create a separate lambda layer with just those files in it. I have not used docker too much so any pointers or advice would be greatly appreciated.

Jan 07 '24 06:01 arobdog

Sorry, missed the earlier comments. I have CDK code which performs this - I will extract and post here.

Jan 07 '24 06:01 danielsharvey

@arobdog I setup a Lambda layer using a Dockerfile to extract the needed lib files and put them in the expected location. Dockerfile is below for reference.

FROM public.ecr.aws/lambda/nodejs:latest-x86_64

# Work Directory
WORKDIR /asset

# Create output directory
RUN mkdir -p /asset/lib

# Move liblzma librarie files to the layer output
RUN cp /lib64/liblzma.so.5 /asset/lib/
RUN cp /lib64/liblzma.so.5.2.5 /asset/lib/

Then I just add this as another layer to the expected runtime in addition to the OSGeo layer and it works. Hope this helps!

Jan 10 '24 15:01 JackTiber

@JackTiber I managed to get it working. Thanks for your help :)

Jan 12 '24 22:01 arobdog

docker-lambda docker-lambda copied to clipboard

missing `liblzma.so` in nodejs environment

To Do

docker-lambda
docker-lambda copied to clipboard