pyproj
pyproj copied to clipboard
pyproj4
UserWarning: Valid PROJ data directory not found
I am running this code but for, I realized that the FileNotFound Error is happening because it is looking for a file on another old computer? how do I solve that issue? Is it related to the Pyproj warning:
C:\Users\hdela\.conda\envs\chur2\python.exe D:\Chur\Manager\main.py batch_compute_precipitation
running return period: 500
running return period: 500
executing 2 processes
['python', 'D:/Chur/Engine/main.py', 'precipitation', 'D:/Chur/Manager/precipitation_definitions/54ac8997-efe7-4877-922c-f10e68f4dc84']
['python', 'D:/Chur/Engine/main.py', 'precipitation', 'D:/Chur/Manager/precipitation_definitions/f563a296-bc30-4474-ac18-7f272fd58e14']
C:\Users\hdela\.conda\envs\chur2\lib\site-packages\pyproj\__init__.py:91: UserWarning: Valid PROJ data directory not found. Either set the path using the environmental variable PROJ_LIB or with `pyproj.datadir.set_data_dir`.
warnings.warn(str(err))
C:\Users\hdela\.conda\envs\chur2\lib\site-packages\pyproj\__init__.py:91: UserWarning: Valid PROJ data directory not found. Either set the path using the environmental variable PROJ_LIB or with `pyproj.datadir.set_data_dir`.
warnings.warn(str(err))
Traceback (most recent call last):
File "D:\Chur\Engine\main.py", line 1676, in <module>
main(sys.argv[1:])
File "D:\Chur\Engine\main.py", line 606, in main
engine.precipitationSimulator.initialize()
File "D:\Chur\Engine\Models\Precipitation\precipitation.py", line 46, in initialize
self.generator.initialize()
File "D:\Chur\Engine\Models\Precipitation\precipitation.py", line 154, in initialize
self.fh = Dataset(self.ncfile, mode='r')
File "src\netCDF4\_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
File "src\netCDF4\_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'C:/Users/dhoussou/chur/Chur/Engine/Data/SourceData/precipitation/MeteoSwiss/RdisaggH_ch01r.swisscors_200703010100_200704010000.nc'
Traceback (most recent call last):
File "D:\Chur\Engine\main.py", line 1676, in <module>
main(sys.argv[1:])
File "D:\Chur\Engine\main.py", line 606, in main
engine.precipitationSimulator.initialize()
File "D:\Chur\Engine\Models\Precipitation\precipitation.py", line 46, in initialize
self.generator.initialize()
File "D:\Chur\Engine\Models\Precipitation\precipitation.py", line 154, in initialize
self.fh = Dataset(self.ncfile, mode='r')
File "src\netCDF4\_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
File "src\netCDF4\_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'C:/Users/dhoussou/chur/Chur/Engine/Data/SourceData/precipitation/MeteoSwiss/RdisaggH_ch01r.swisscors_200710010100_200711010000.nc'
D:/Chur/Manager/precipitation_definitions/f563a296-bc30-4474-ac18-7f272fd58e14
2023-10-15 23:44:33-- engine: creating output directories at D:/Chur/Engine/Data/SimulationData//Precipitation/f563a296-bc30-4474-ac18-7f272fd58e14/ --
D:/Chur/Manager/precipitation_definitions/54ac8997-efe7-4877-922c-f10e68f4dc84
2023-10-15 23:44:33-- engine: creating output directories at D:/Chur/Engine/Data/SimulationData//Precipitation/54ac8997-efe7-4877-922c-f10e68f4dc84/ --
executed 2 processes
Process finished with exit code 0
My environment variables: C:\Users\hdela.conda\envs\chur2\Lib\site-packages\pyproj C:\Users\hdela.conda\envs\chur2\Library\share\proj
Why did you delete all of the questions on the issue template? They are there for a reason. That reason is that we need that information to give you any useful kind of answer. Please answer these questions and then we'll see if we can help:
#### Installation method/steps
- Installation method (conda, pip wheel, from source, etc...)
- How did you install PROJ? Where is it installed?
- Please provide all commands/steps you used to install pyproj and PROJ.
#### Environment Information
- pyproj version you are attempting to install
- PROJ version (Execute `proj` command and give version here.)
- Python version (`python -c "import sys; print(sys.version.replace('\n', ' '))"`)
- Operation System Information (`python -c "import platform; print(platform.platform())"`)
Rel. 8.1.1, September 1st, 2021 usage: PROJ [-bdeEfiIlmorsStTvVwW [args]] [+opt[=arg] ...] [file ...]
This is my environment information:
3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:39:05) [MSC v.1929 64 bit (AMD64)]
Windows-10-10.0.22621-SP0
I installed it through conda-forge:
These are all the package.:
Name Version Build Channel
attrs 23.1.0 pyh71513ae_1 conda-forge
boost-cpp 1.74.0 h9f4b32c_8 conda-forge
brotli 1.1.0 hcfcfb64_1 conda-forge
brotli-bin 1.1.0 hcfcfb64_1 conda-forge
brotli-python 1.1.0 py39h99910a6_1 conda-forge
bzip2 1.0.8 h8ffe710_4 conda-forge
ca-certificates 2023.7.22 h56e8100_0 conda-forge
cairo 1.16.0 hb19e0ff_1008 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
cfitsio 3.470 h0af3d06_7 conda-forge
cftime 1.6.2 py39hc266a54_1 conda-forge
charset-normalizer 3.3.0 pyhd8ed1ab_0 conda-forge
click 8.1.7 win_pyh7428d3b_0 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.7.2 pyhd8ed1ab_1 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.1.1 py39h1f6ef14_1 conda-forge
curl 7.86.0 heaf79c2_1 conda-forge
cycler 0.12.0 pyhd8ed1ab_0 conda-forge
decorator 4.4.2 py_0 conda-forge
expat 2.5.0 h63175ca_1 conda-forge
fiona 1.8.20 py39hea8b339_2 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 hbde0cde_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.43.0 py39ha55989b_0 conda-forge
freetype 2.12.1 hdaf720e_2 conda-forge
freexl 1.0.6 h67ca5e6_1 conda-forge
gdal 3.3.3 py39h3f5efd6_0 conda-forge
geopandas 0.9.0 pyhd8ed1ab_1 conda-forge
geopandas-base 0.9.0 pyhd8ed1ab_1 conda-forge
geos 3.9.1 h39d44d4_2 conda-forge
geotiff 1.7.0 h350af67_3 conda-forge
gettext 0.21.1 h5728263_0 conda-forge
hdf4 4.2.15 h1b1b6ef_5 conda-forge
hdf5 1.12.1 nompi_h2a0e4a3_104 conda-forge
icu 68.2 h0e60522_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-resources 6.1.0 pyhd8ed1ab_0 conda-forge
importlib_resources 6.1.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2023.2.0 h57928b3_50496 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
jpeg 9e hcfcfb64_3 conda-forge
kealib 1.4.15 h8995ca9_0 conda-forge
kiwisolver 1.4.5 py39h1f6ef14_1 conda-forge
krb5 1.19.3 h1176d77_0 conda-forge
lcms2 2.14 h90d422f_0 conda-forge
lerc 4.0.0 h63175ca_0 conda-forge
libblas 3.9.0 18_win64_mkl conda-forge
libbrotlicommon 1.1.0 hcfcfb64_1 conda-forge
libbrotlidec 1.1.0 hcfcfb64_1 conda-forge
libbrotlienc 1.1.0 hcfcfb64_1 conda-forge
libcblas 3.9.0 18_win64_mkl conda-forge
libcurl 7.86.0 heaf79c2_1 conda-forge
libdeflate 1.14 hcfcfb64_0 conda-forge
libexpat 2.5.0 h63175ca_1 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
libgdal 3.3.3 h08edb8f_0 conda-forge
libglib 2.78.0 he8f3873_0 conda-forge
libhwloc 2.9.1 h51c2c0f_0 conda-forge
libiconv 1.17 h8ffe710_0 conda-forge
libkml 1.3.0 hd45a9bc_1016 conda-forge
liblapack 3.9.0 18_win64_mkl conda-forge
libnetcdf 4.8.1 nompi_h1cc8e9d_102 conda-forge
libpng 1.6.39 h19919ed_0 conda-forge
libpq 13.8 h04fd8be_0 conda-forge
librttopo 1.1.0 hb340de5_6 conda-forge
libspatialindex 1.9.3 h39d44d4_4 conda-forge
libspatialite 5.0.1 h6a90f17_9 conda-forge
libsqlite 3.43.0 hcfcfb64_0 conda-forge
libssh2 1.10.0 h680486a_3 conda-forge
libtiff 4.4.0 hc4f729c_5 conda-forge
libwebp-base 1.3.2 hcfcfb64_0 conda-forge
libxcb 1.13 hcd874cb_1004 conda-forge
libxml2 2.10.4 hc3477c8_0 conda-forge
libzip 1.9.2 hfed4ece_1 conda-forge
libzlib 1.2.13 hcfcfb64_5 conda-forge
lz4-c 1.9.3 h8ffe710_1 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
mapclassify 2.5.0 pyhd8ed1ab_1 conda-forge
matplotlib-base 3.7.1 py39haf65ace_0 conda-forge
mkl 2022.1.0 h6a75c08_874 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
munch 4.0.0 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
netcdf4 1.6.0 nompi_py39hf113b1f_100 conda-forge
networkx 2.6.2 pyhd8ed1ab_0 conda-forge
numpy 1.20.3 py39h6635163_2 conda-forge
openjpeg 2.5.0 hc9384bd_1 conda-forge
openssl 1.1.1w hcfcfb64_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 1.5.3 py39h2ba5b7c_1 conda-forge
pcre 8.45 h0e60522_0 conda-forge
pcre2 10.40 h17e33f8_0 conda-forge
pillow 9.2.0 py39h595c93f_3 conda-forge
pip 23.2.1 pyhd8ed1ab_0 conda-forge
pixman 0.42.2 h63175ca_0 conda-forge
platformdirs 3.11.0 pyhd8ed1ab_0 conda-forge
pooch 1.7.0 pyhd8ed1ab_4 conda-forge
poppler 21.09.0 h24fffdf_3 conda-forge
poppler-data 0.4.12 hd8ed1ab_0 conda-forge
postgresql 13.8 h392df6f_0 conda-forge
proj 8.1.1 h1cfcee9_2 conda-forge
pthread-stubs 0.4 hcd874cb_1001 conda-forge
pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyproj 3.2.1 py39h39b2389_2 conda-forge
pysocks 1.7.1 pyh0701188_6 conda-forge
python 3.9.15 h0269646_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 4_cp39 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
rtree 1.0.1 py39h09fdee3_3 conda-forge
scikit-learn 1.2.2 py39hb6325b5_1 conda-forge
scipy 1.10.1 py39hfbf2dce_0 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
shapely 1.7.1 py39haadaec5_5 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.43.0 hcfcfb64_0 conda-forge
tbb 2021.9.0 h91493d7_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tiledb 2.3.4 h78dabda_0 conda-forge
tk 8.6.13 hcfcfb64_0 conda-forge
typing-extensions 4.8.0 hd8ed1ab_0 conda-forge
typing_extensions 4.8.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
ucrt 10.0.22621.0 h57928b3_0 conda-forge
unicodedata2 15.1.0 py39ha55989b_0 conda-forge
uriparser 0.9.7 h1537add_1 conda-forge
urllib3 2.0.6 pyhd8ed1ab_0 conda-forge
vc 14.3 h64f974e_17 conda-forge
vc14_runtime 14.36.32532 hdcecf7f_17 conda-forge
vs2015_runtime 14.36.32532 h05e6639_17 conda-forge
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge
xarray 2023.1.0 pyhd8ed1ab_0 conda-forge
xerces-c 3.2.4 h63175ca_3 conda-forge
xorg-libxau 1.0.11 hcd874cb_0 conda-forge
xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge
xz 5.2.6 h8d14728_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hcfcfb64_5 conda-forge
zstd 1.5.5 h12be248_0 conda-forge
Side note: I'm a little surprised you're getting such an old version of pyproj, although that's not the problem here most likely. When you created your environment did you specify any version limits for any of your dependencies?
Yes. This is what I needed to run my code so pyproj came as a dependency: networkx== 2.6.2 shapely== 1.7.1 geopandas== 0.9.0 numpy==1.20.3 That is why my version of pyproj is so old.
Is that "Yes" to the activation question? The various PROJ directory environment variables are meant to be set on environment activation.
Yes. I think because I installed an older version version of geopandas it came with an older version of proj. I don't know if I answered your question correctly
Ok, so at some point you are running conda activate chur2? Ok. There is a small-ish chance that something has been fixed since this older version of pyproj in the conda-forge builds, but it is hard to say. I didn't see anything obvious in the commit history on conda-forge. Let me see if I can reproduce the warnings on my linux box. It could also be that I'm just completely missing something so @snowman2 could probably answer this too.
I run the following: conda activate chur2 conda install -c conda-forge networkx== 2.6.2 shapely== 1.7.1 geopandas== 0.9.0 numpy== 1.20.3 fiona netcdf4 That was how I isntalled my packages. Thank you for your help
If you run python -c "import pyproj" do you see the warning?
Edit: I'm trying to figure out the smallest amount of code to reproduce the warning. I'm not seeing it on linux with these package versions, but it wouldn't surprise me that Windows builds have an issue that linux doesn't.
It works fine: (chur2) C:\Users\hdela>python -c "import pyproj"
(chur2) C:\Users\hdela>python Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:39:05) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. import pyproj
Well the warning you were getting is supposed to show up in pyproj/__init__.py when you import it so I'm not sure why you would stop getting it now. You could do:
python -c "import warnings; warnings.simplefilter('error'); import pyproj"
but I wouldn't guess it would be a problem. So perhaps one of the other packages is interfering with pyproj? What order do you import things in your script? Also, it looks like you might be using some multiprocessing, is that right?
Yes I am using some multiprocessing. These are the other import statement: import sys
import numpy as np import pandas as pd import csv import pickle import math import os import json import uuid import shutil import threading
from scipy.stats import norm from time import strftime, time import time import cProfile, pstats, io from pstats import SortKey
from osgeo.gdalconst import * from osgeo import osr, ogr, gdal import geopandas as gpd import subprocess import matplotlib.pyplot as plt import matplotlib.cm as cm from mpl_toolkits.axes_grid1.inset_locator import inset_axes
import matplotlib as mpl
from fiona import collection, open as fiona_open from shapely.geometry import mapping, shape, Point, LineString
Do you think is it that warning that is making python to look for that folder on my old machine. The last line of the error message where FileNotFoundError: [Errno 2] No such file or directory: b'C:/Users/dhoussou/chur/Chur/Engine/Data/SourceData/precipitation/MeteoSwiss/RdisaggH_ch01r.swisscors_200703010100_200704010000.nc' That's the path on my old machine. I do not think but I am using a new machine so I am confused why that is? Do you think it has something to do with PROJ. I have looked through my code and I don't think I provided any path to my old machine so I am a bit confused.
Did you copy your old machine's environment to your new machine?
No I don't think that warning has anything to do with your incorrect netcdf path being used besides that both issues could be pointing to the same problem: something got copied or configured incorrectly when switching to the new machine.
Note: Even with that set of imports in a python file and with:
import warnings
warnings.simplefilter("always")
I don't see a pyproj warning on my linux environment.
Providing a Minimal Reproducible Example may help to debug the issue: https://stackoverflow.com/help/minimal-reproducible-example
