microsoft
Unable to locate package for MS SQL Driver in ARM64 / AARCH64
Hello everyone. I’m currently developing some communication between a Python IoT Edge Module and an SQL server. Specifically, fetching some data.
Overview
My IoT Edge Device runs on arm64v8, and the SQL Database is hosted in Azure.
I tried to initiate communication using the following lines from https://docs.microsoft.com/en-us/sql/connect/python/pyodbc/step-3-proof-of-concept-connecting-to-sql-using-pyodbc?view=sql-server-ver15
import pyodbc
connexn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = connexn.cursor()
The initial Dockerfile.amd64v8 that I used was created from the template by Azure IoT Hub:
FROM arm64v8/python:3.7-slim-buster
WORKDIR /app
RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
CMD [ "python3", "-u", "./main.py" ]
First try
I tried to install the pyodbc package (no specific version) by inserting it in the requirements.txt. Once the pip install reached pyodbc however,
ERROR: Failed building wheel for pyodbc
Running setup.py clean for pyodbc
Failed to build pyodbc
Installing collected packages: pyodbc
Running setup.py install for pyodbc: started
Running setup.py install for pyodbc: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bq0f_5j5/pyodbc_<<32-character-hash>>/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bq0f_5j5/pyodbc_<<32-character-hash>>/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code,
__file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pt5o4499/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7m/pyodbc
cwd: /tmp/pip-install-bq0f_5j5/pyodbc_<<32-character-hash>>/
Complete output (10 lines):
running install
running build
running build_ext
building 'pyodbc' extension
creating build
creating build/temp.linux-aarch64-3.7
creating build/temp.linux-aarch64-3.7/src
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPYODBC_VERSION=4.0.31 -I/usr/local/include/python3.7m -c src/buffer.cpp -o
build/temp.linux-aarch64-3.7/src/buffer.o -Wno-write-strings
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
Second try
I went through many documentations, and much trial-and-error to get the dependencies needed to build the module, and I finally found the right combination of packages to at least let the build finish (I don’t know if any of these packages are redundant, but I didn’t remove any of them for fear of the build failing)
FROM arm64v8/python:3.7-slim-buster
WORKDIR /app
RUN apt-get update
RUN apt-get install -y python3-dev
RUN apt-get install -y unixodbc-dev
RUN apt-get install -y g++
RUN apt-get install -y ffmpeg libsm6 libxext6 libc6 libgcc1 libodbc1 libstdc++6
RUN apt-get install -y gcc libc-dev g++ libffi-dev libxml2 libffi-dev unixodbc-dev
RUN pip install pyodbc
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
CMD [ "python3", "-u", "./main.py" ]
The build finished without a hitch, and I then deployed this module into my IoT Edge Device (arm64). However, the module had following error logs:
IoT Hub Client for Python
Unexpected error ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)")
Traceback (most recent call last):
File "/app/main.py", line 420, in main
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)")
Third try
I searched the internet for instructions on how to install the Driver into my Linux device, and I found the following: https://stackoverflow.com/questions/60105924/docker-ubuntu-18-04-unable-to-install-msodbcsql17-sql-server-odbc-driver-17
My Dockerfile was now:
FROM arm64v8/python:3.7-slim-buster
WORKDIR /app
RUN apt-get update
RUN apt-get install -y curl python3.7 git python3-pip unixodbc-dev
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get install -y --allow-unauthenticated mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN apt-get update
RUN apt-get install -y python3-dev
RUN apt-get install -y unixodbc-dev
RUN apt-get install -y g++
RUN apt-get install -y ffmpeg libsm6 libxext6 libc6 libgcc1 libodbc1 libstdc++6
RUN apt-get install -y gcc libc-dev g++ libffi-dev libxml2 libffi-dev unixodbc-dev
RUN pip install pyodbc
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
CMD [ "python3", "-u", "./main.py" ]
However, now I can’t build it because the command cannot seem to locate the needed packages.
RUN ACCEPT_EULA=Y apt-get install -y --allow-unauthenticated mssql-tools:
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package mssql-tools
------
executor failed running [/bin/sh -c ACCEPT_EULA=Y apt-get install -y --allow-unauthenticated mssql-tools]: exit code: 100
Final question
Is there any way to install both the MS SQL ODBC Driver and the pyodbc package in an ARM64 / AARCH64 device ? Is this even supported? I couldn't find any when I googled today.
The M1 is supported from 17.8 but if you use the ms sql driver through docker (on an M1 - docker itself not being unusual for backend developers) - it will use the aarch64 format within the docker container - and there is no compilation in this format. I need this driver also.
