Python 3.8 - Amazon linux 2

[dave-graham]

Hi all,

Thanks for this, it's an amazing idea. I tried it with Python 3.8 as the lambda runtime, which runs Amazon linux 2, and kept getting the error:

"No module named '_gdal'

I don't know if this is me missing something, but if it isn't, and it's possible to get it working on Amazon linux 2, that would be amazing.

Thanks again.

[vincentsarago]

Hi @dave-graham, as you noted, python 3.8 runtime runs in the new amazonlinux 2 image. this is a brand new image and compiling GDAL on it is quite complex. I don't think this will happens soon sadly

[dave-graham]

Cool, thanks for clarifying @vincentsarago

[lhadjchikh]

In case it will be helpful, I got this to work with the lambci/lambda:build-python3.8 image by making the following changes:

For PROJ 7, I added libcurl-devel to the packages installed by yum. I also had to install libtiff:

ENV LIBTIFF_VERSION=4.0.9

RUN \
    mkdir libtiff; \
    wget -qO- https://download.osgeo.org/libtiff/tiff-$LIBTIFF_VERSION.tar.gz \
        | tar xvz -C libtiff --strip-components=1; cd libtiff; \
    ./configure --prefix=$PREFIX; \
    make; make install; \
    cd ..; rm -rf libtiff;

For GDAL, I removed libpng-devel from the installed packages, because I got an error related to a missing libpng15.so.15 file, which went away when I removed libpng-devel.

I moved up the installation of jpeg_turbo to come before the hdf4 installation, to satisfy hdf4's dependency on the jpeg library.

I also added the following lines to package.sh:

cp -P /usr/lib64/libcurl.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libxml2.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libidn2.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libssh2.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libldap*.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/liblber*.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libunistring.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libsasl2.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libssl3.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libsmime3.so* $DEPLOY_DIR/lib/
cp -P /usr/lib64/libnss3.so* $DEPLOY_DIR/lib/

I removed the code for curl because I found it broke yum (which I wanted to be free to use after the build). I replaced it in the GDAL config with the version from libcurl-devel: --with-curl=/usr/bin/curl-config.

With a few other (unrelated) changes, I was able to use this layer to get GeoDjango running in Lambda (using Zappa) without any errors from GDAL. This was all done through trial and error, though, so there may be problems with this approach that I'm unaware of. But I thought I would post my solution in case it's helpful.

[jlaura]

I see that this is quite an old issue. I am still seeing the same error: "No module named '_gdal'. Is the current solution to run the python 3.7 runtime or to build as per @lhadjchikh's solution?

[nghilethanh-atherlabs]

Hi any solution for this?

[nghilethanh-atherlabs]

@lhadjchikh Can I have your full code?