Error loading transliteration module, dir=(null) at libpostal_setup_datadir (libpostal.c:290) errno: No such file or directory

[hamza1ettaki]

Hi!

I tried to build libpostal on my Conda environment :

conda activate condaenv
conda install libpostal -y
conda install postal -y
pip install scrubadub==2.0.0
pip install scrubadub_spacy
pip install scrubadub_address

After installing all requirements, I tried to send my script to the cluster with the conda environment using spark-submit command for processing:

spark-submit \ 
--name processing_addresses  \  
--master yarn \  
--deploy-mode cluster \  
.
.
.
--conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=./condaenv/bin/python \
--conf spark.yarn.appMasterEnv.PYSPARK_DRIVER_PYTHON=./condaenv/bin/python \
--conf spark.executorEnv.PYSPARK_PYTHON=./condaenv/bin/python \
--conf spark.executorEnv.PYSPARK_DRIVER_PYTHON=./condaenv/bin/python \
--archives /path/to/env/condaenv.tar.gz#condaenv \
./myscript.py

My job fail with this error :

ERR   Error loading transliteration module, dir=(null)
   at libpostal_setup_datadir (libpostal.c:290) errno: No such file or directory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/__init__.py", line 1, in <module>
    from . import detectors
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/detectors/__init__.py", line 1, in <module>
    from .address import AddressDetector
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/detectors/address.py", line 4, in <module>
    import postal.parser
  File "/home/condaenv/lib/python3.7/site-packages/postal/parser.py", line 2, in <module>
    from postal import _parser
SystemError: initialization of _parser raised unreported exception

The libpostal work on my local environment if I don't change the location of my conda environment, Also when I try to change the name of my conda environment locally the libpostal doesn't work for me.

How could I resolve this?

Thank you

[shivamspj]

Same issue for me on windows using msys2.

[selva221724]

Same issue for me on windows using msys2.

https://github.com/selva221724/pypostalwin

[selva221724]

Hi!

I tried to build libpostal on my Conda environment :

conda activate condaenv
conda install libpostal -y
conda install postal -y
pip install scrubadub==2.0.0
pip install scrubadub_spacy
pip install scrubadub_address

After installing all requirements, I tried to send my script to the cluster with the conda environment using spark-submit command for processing:

spark-submit \ 
--name processing_addresses  \  
--master yarn \  
--deploy-mode cluster \  
.
.
.
--conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=./condaenv/bin/python \
--conf spark.yarn.appMasterEnv.PYSPARK_DRIVER_PYTHON=./condaenv/bin/python \
--conf spark.executorEnv.PYSPARK_PYTHON=./condaenv/bin/python \
--conf spark.executorEnv.PYSPARK_DRIVER_PYTHON=./condaenv/bin/python \
--archives /path/to/env/condaenv.tar.gz#condaenv \
./myscript.py

My job fail with this error :

ERR   Error loading transliteration module, dir=(null)
   at libpostal_setup_datadir (libpostal.c:290) errno: No such file or directory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/__init__.py", line 1, in <module>
    from . import detectors
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/detectors/__init__.py", line 1, in <module>
    from .address import AddressDetector
  File "/home/condaenv/lib/python3.7/site-packages/scrubadub_address/detectors/address.py", line 4, in <module>
    import postal.parser
  File "/home/condaenv/lib/python3.7/site-packages/postal/parser.py", line 2, in <module>
    from postal import _parser
SystemError: initialization of _parser raised unreported exception

The libpostal work on my local environment if I don't change the location of my conda environment, Also when I try to change the name of my conda environment locally the libpostal doesn't work for me.

How could I resolve this?

Thank you

Try this one out: https://github.com/selva221724/pypostalwin

[pratt-lightbeam]

same error on ubuntu 20.04.

ERR Error loading transliteration module, dir=(null) at libpostal_setup_datadir (libpostal.c:290) errno: No such file or directory

[CurisZhou]

anyone has solved this issue？

[pratt3000]

Just install afresh. Spent weeks on libpostal errors (this one too). It has something to do with the installation steps. (will comment within a day if I find the commands)

[CurisZhou]

Finally, I have solved this issue.

I use libpostal with python binding. A good way to use libpostal python binding is to install this library pylibpostal 1.0.0 instead.

When I used pylibpostal 1.0.0 in my environment(Centos 7, Jupyter notebook), I met this issue. The reason is that my jupyter notebook cannot read the environment variable LIBPOSTAL_DATA_DIR that path storing the data files of libpostal. Hence, I explicitly set the environment variable LIBPOSTAL_DATA_DIR in my code before importing the pylibpostal os.environ['LIBPOSTAL_DATA_DIR']='your absolute path to store data files of libpostal'

[selva221724]

if you are looking for an unofficial python binding in windows, please use this python package. It can work very well in windows

https://github.com/selva221724/pypostalwin