scrapy-cluster icon indicating copy to clipboard operation
scrapy-cluster copied to clipboard

Published 20 hours ago • verified publisher icon istresearch

Python 3

Open deed02392 opened this issue 5 years ago • 6 comments

Issue to register the request to migrate fully to Python 3.

deed02392 avatar Nov 11 '19 14:11 deed02392

@deed02392 Does the python 3 containers not work for you right now? The master branch is not up to date, but the project on the dev branch is fully python 3 compatible. I agree we should migrate all the way over but that might just mean that we drop support for python 2 setups/containers/installs.

madisonb avatar Nov 18 '19 22:11 madisonb

Hey @madisonb, well, they were not working well with the latest Alpine Linux based Python 3 container. I basically had to update a lot of packages to get things working on the Scrapy container. I only got as far as making that run again after updating all the dependencies, but I think I will need to do the same for the other Docker components.

deed02392 avatar Nov 19 '19 23:11 deed02392

Hi @madisonb what is the current status with running the master branch with python 3? I also noticed the requirements.txt file is depreciated, what specifically should I alter/add to ensure all requirements are met? Thanks.

cyclehacker avatar Mar 21 '20 11:03 cyclehacker

All of the unit and integration tests for the python3 docker container continue to pass, here is the build from lasst week.

https://travis-ci.org/github/istresearch/scrapy-cluster/jobs/664497952

I'm not sure there is much more to be done for the base version of this project if you are using the dev branch.

madisonb avatar Mar 25 '20 20:03 madisonb

I tried to run this on Python3.8 but it doesn't work. lots of dependencies need to work on

psdon avatar Dec 04 '20 22:12 psdon

I will gladly accept a PR to upgrade the project to something beyond python 3.6. I have not done any testing beyond what runs in the dev branch.

madisonb avatar Dec 22 '20 23:12 madisonb