Cannot be installed on Python 3.10 (Fedora 35)

[edwintorok]

Asking pip to install pdftopng tries to install 0.1.0 (and fails to build). Explicitly asking to install 0.2.3 fails saying that it is not available. AFAICT there are no python 3.10 wheels on pypi for pdftopng, which might be causing this:

pip3 install --user pdftopng
Collecting pdftopng
  Using cached pdftopng-0.1.0.tar.gz (11 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: Click>=7.0 in /usr/lib/python3.10/site-packages (from pdftopng) (8.0.1)
Building wheels for collected packages: pdftopng
  Building wheel for pdftopng (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp8wt7jd33
       cwd: /tmp/pip-install-lnvcig9z/pdftopng_0379acfa701c4fd4b7e0369d0ba51682
  Complete output (30 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.10
  creating build/lib.linux-x86_64-3.10/pdftopng
  copying src/pdftopng/__version__.py -> build/lib.linux-x86_64-3.10/pdftopng
  copying src/pdftopng/__init__.py -> build/lib.linux-x86_64-3.10/pdftopng
  running egg_info
  writing src/pdftopng.egg-info/PKG-INFO
  writing dependency_links to src/pdftopng.egg-info/dependency_links.txt
  writing requirements to src/pdftopng.egg-info/requires.txt
  writing top-level names to src/pdftopng.egg-info/top_level.txt
  reading manifest file 'src/pdftopng.egg-info/SOURCES.txt'
  writing manifest file 'src/pdftopng.egg-info/SOURCES.txt'
  copying src/pdftopng/pdftopng.cpp -> build/lib.linux-x86_64-3.10/pdftopng
  running build_ext
  creating tmp
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmpzf7d0e91.cpp -o tmp/tmpzf7d0e91.o -std=c++14
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmphw_edm25.cpp -o tmp/tmphw_edm25.o -fvisibility=hidden
  building 'pdftopng.pdftopng' extension
  creating build/temp.linux-x86_64-3.10
  creating build/temp.linux-x86_64-3.10/src
  creating build/temp.linux-x86_64-3.10/src/pdftopng
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DVERSION_INFO="0.1.0" -Ilib/poppler -Ilib/poppler/fofi -Ilib/poppler/goo -Ilib/poppler/poppler -Ilib/poppler/build -Ilib/poppler/build/poppler -Ilib/poppler/utils -Ilib/poppler/build/utils -I/tmp/pip-build-env-4h1_bfzy/overlay/lib/python3.10/site-packages/pybind11/include -I/usr/include/python3.10 -c src/pdftopng/pdftopng.cpp -o build/temp.linux-x86_64-3.10/src/pdftopng/pdftopng.o -O3 -Wall -shared -fPIC -std=c++14 -fvisibility=hidden
  src/pdftopng/pdftopng.cpp:44:10: fatal error: config.h: No such file or directory
     44 | #include "config.h"
        |          ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for pdftopng
Failed to build pdftopng
ERROR: Could not build wheels for pdftopng which use PEP 517 and cannot be installed directly

pip3 install --user pdftopng==0.2.3
ERROR: Could not find a version that satisfies the requirement pdftopng==0.2.3 (from versions: 0.1.0)
ERROR: No matching distribution found for pdftopng==0.2.3

Using other package managers such as Poetry runs into similar errors:

• Installing pdftopng (0.2.3): Failed

  RuntimeError

  Unable to find installation candidates for pdftopng (0.2.3)

  at /usr/lib/python3.10/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

python3 --version
Python 3.10.0
 pip3 --version
pip 21.2.3 from /usr/lib/python3.10/site-packages/pip (python 3.10)

[Ben621]

same error with Python 3.10.1

 Unable to find installation candidates for pdftopng (0.2.3)

---

  at c:\users\user\pycharmprojects\play\venv\lib\site-packages\poetry\installation\chooser.py:72 in choose_for
       68│
       69│             links.append(link)
       70│
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│
       76│         # Get the best link


Failed to add packages, reverting the pyproject.toml file to its original content.

[rain01]

I have the same error with Python 3.9.10:

% python --version           
Python 3.9.10
% pip install pdftopng==0.2.3
ERROR: Could not find a version that satisfies the requirement pdftopng==0.2.3 (from versions: 0.1.0)
ERROR: No matching distribution found for pdftopng==0.2.3

[bosd]

This also occurs on python 3.7.3. Seems more like a generic bug in this release. @vinayak-mehta Can you prioritize this bug?

(it also prevents camelot to being installed, as it is pointing to pdftopng=>0.2.3)

[jacopone]

Hi this is happening for me as well while trying to install excalibur with poetry. It breaks when it tries to install pdftopng

@vinayak-mehta If you fix this bug it would be amazing!

Thanks

• Installing pdftopng (0.2.3): Failed

  RuntimeError

  Unable to find installation candidates for pdftopng (0.2.3)

  at ~/.poetry/lib/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

[stefanw]

This package does not currently have wheels for Python 3.10 and does not have a source distribution. Unfortunately, it is a hard requirement for camelot-py (even though it doesn't seem to be used there).

[joshuataylor]

If you are using poetry, adding the following to poetry.toml fixes this:

[installer]
no-binary = ["pdftopng"]

[NixBiks]

If you are using poetry, adding the following to poetry.toml fixes this:

[installer]
no-binary = ["pdftopng"]

It doesn't work for me unfortunately. I'm still getting (using Poetry (version 1.2.1) on mac M1)

  RuntimeError

  Unable to find installation candidates for pdftopng (0.2.3)

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/installation/chooser.py:103 in choose_for
       99│ 
      100│             links.append(link)
      101│ 
      102│         if not links:
    → 103│             raise RuntimeError(f"Unable to find installation candidates for {package}")
      104│ 
      105│         # Get the best link
      106│         chosen = max(links, key=lambda link: self._sort_key(package, link))

[yguel]

If you are using poetry, adding the following to poetry.toml fixes this:

[installer]
no-binary = ["pdftopng"]

Hi, Can you share your whole pyproject.toml ? In my case adding those lines with poetry 1.4.1 does not solve the problem.

[aoyono]

I just came accross this issue while trying to install camelot with Python3.11. After a lot of research, trial and errors, I think I understand where the issue comes from: pdftopng has a dependency on poppler, which apparently moved away from github to their own gitlab (probably because of the MSFT acquisition of gh). So the git subproject referred to by dftopng is now "dangling". In fact if you try to follow the link from this repo, you get a 404 from github.

So I found a workaround, although it's quite hacky:

1. clone the pdftopng repo: git clone [email protected]:vinayak-mehta/pdftopng.git
2. clone the poppler repo from their gitlab inside the lib folder of pdftopng: https://gitlab.freedesktop.org/poppler/poppler.git pdftopng/lib/poppler
3. checkout the commit referred to by pdftopng v0.2.3: cd pdftopng/lib/poppler && git checkout fcdff7b
4. generate the config.h include file of the poppler library: mkdir build && cd build && cmake ..
5. move the generated config.h file to the source folder of poppler: mv config.h ../
6. install pdftopng from source: cd ../../../ && python3 -m pip install .

This should solve the issue of installing pdftopng version 0.2.3, provided you have cmake and probably other build dependencies installed on your system

[joshuataylor]

Yeah, this ended up being painful later for developer environments, and a complete PITA with 3.11.

Also, you should use git clone --recursive instead, as this will fetch the submodules for you, so you don't need to fetch through everything. If you forgot to do this, as I am prone to doing.., you can use git submodule update --init. I ended up having a shell alias that does this for me.

Tip, try git clone --recurse-submodules -j8, -j8 means it'll initialise with up to 8 in parallel, which should be much faster. :rocket:

Take a quick look at https://github.com/vinayak-mehta/pdftopng/tree/main/scripts , you should see your system there and how to build for it. It's pretty straight forward for most systems.

What we ended up doing was building wheels then referencing them in our pyproject.toml. It was the least painful way in both development and in production. You can also use a URL, or we just placed them into a vendored folder, which I think is fine for this project as it isn't updated too often. :shrug:

pdftopng = [
   {markers="sys_platform == 'linux' and platform_machine == 'x86_64'", path='vendored/pdftopng-0.2.3-cp311-cp311-linux_x86_64.whl'},
   {markers="sys_platform == 'linux' and platform_machine == 'arm64'", path='vendored/pdftopng-0.2.3-cp311-cp311-linux_aarch64.whl'},
   {markers="sys_platform == 'darwin' and platform_machine == 'x86_64'", version="0.2.3"},
   {markers="sys_platform == 'darwin' and platform_machine == 'arm64'", path='vendored/pdftopng-0.2.3-cp311-cp311-macosx_13_0_arm64.whl'},
]

I've uploaded some of the wheels I have built here, when I have an hour or so I'll move this to python.build, where I build wheels for older packages. I didn't build for MacOS x86 (Intel), as none of our devs use an Intel Mac. I'm just some rando on the internet, so you don't have to trust my wheels, that's why I'll also provide the Github Actions build log, etc.

If you build for MacOS, you're going to want to brew install poppler cmake pkg-config freetype fontconfig libpng jpeg.

Python 3.10: pdftopng-0.2.3-cp310-cp310-linux_aarch64.whl pdftopng-0.2.3-cp310-cp310-linux_x86_64.whl pdftopng-0.2.3-cp310-cp310-macosx_12_0_arm64.whl

Python 3.11: pdftopng-0.2.3-cp311-cp311-linux_aarch64.whl pdftopng-0.2.3-cp311-cp311-linux_x86_64.whl pdftopng-0.2.3-cp311-cp311-macosx_13_0_arm64.whl

[aoyono]

Thanks for the pointer to the wheels, @joshuataylor. I can provide wheels for intel mac, if needed, as this is what I am working with at the moment. Just wanted to say, the recursive clone doesn't solve the issue, you would still get a

fatal: repository 'https://github.com/freedesktop/poppler/' not found

I don't know if it's possible to refer to a gitlab hosted repository in github. That would be a way forward (I can think of at least one issue: that of the config.h file, which is automatically generated by cmake, so a simple pip install wouldn't solve for that one, unless the setup.py is modified to add a step for that)

[mara004]

@joshuataylor I think your wheels only work on the exact system where they were built; when trying to use pdftopng-0.2.3-cp311-cp311-linux_x86_64.whl on Fedora 37 I get "ImportError: libpoppler.so.111: cannot open shared object file: No such file or directory".

[hubutui]

1. According to PKGBUILD for python-pdftopng on AUR, the file config.h from poppler is not needed at all, we could delete the line include this header file.
2. We also need this file: numberofcharacters.h, download it to src/pdftopng. Or we could just copy and paste the file content to src/pdftopng/pdftopng.cpp and delete the line include this header file since it contains only a simple function.