Modernize the repo

[mraarif]

python-modernize isort itertools.izip_longest

TODO items (when ready to add support for python3) need to replace from cStringIO import StringIO in tests. But right now tests are running fine on 3.5 even with this import. Debugging this issue.

[awais786]

@mraarif imports order still failing.

[awais786]

@jmbowman any idea why imports order quality failing ?

[jmbowman]

Add the --diff parameter to all isort calls in the Makefile to get more details. Some of the earlier packages to use isort were configured before we realized this option existed.

[codecov[bot]]

Codecov Report

Merging #776 into master will increase coverage by 1.35%. The diff coverage is 87.16%.

@@            Coverage Diff             @@
##           master     #776      +/-   ##
==========================================
+ Coverage   74.39%   75.74%   +1.35%     
==========================================
  Files         211      210       -1     
  Lines       24435    24350      -85     
==========================================
+ Hits        18179    18445     +266     
+ Misses       6256     5905     -351     

Impacted Files	Coverage Δ
edx/analytics/tasks/common/spark.py	0.00% <0.00%> (ø)
edx/analytics/tasks/common/tests/test_mysql.py	100.00% <ø> (ø)
edx/analytics/tasks/launchers/local.py	0.00% <0.00%> (ø)
edx/analytics/tasks/launchers/remote.py	0.00% <0.00%> (ø)
edx/analytics/tasks/tools/analyze/main.py	0.00% <0.00%> (ø)
edx/analytics/tasks/tools/analyze/measure.py	0.00% <0.00%> (ø)
edx/analytics/tasks/tools/analyze/parser.py	0.00% <0.00%> (ø)
edx/analytics/tasks/tools/analyze/report.py	0.00% <0.00%> (ø)
edx/analytics/tasks/tools/s3util.py	0.00% <0.00%> (ø)
...tics/tasks/warehouse/financial/affiliate_window.py	41.66% <11.11%> (+0.54%)	:arrow_up:
... and 192 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 4c63a60...5b75ff5. Read the comment docs.

[mraarif]

command test-docker-py3 is failing (trying to execute tests against python 3)

[mraarif]

@jmbowman after @brianhw 's comment on #777 , do we still want to continue working on getting tests run for python 3?

[jmbowman]

This can wait until DE is ready to try moving forward with the Python 3 upgrade again. The main outcome we wanted from BOM-1041 was to fix the code in sample-themes, edx-themes, and edx-microsite that were Python 3 incompatible. It looks like that's done now, which should solve the problem as far as edx-platform is concerned.

[awais786]

@jmbowman still no update on this repo ?

[jmbowman]

Hmm, I think we completely lost track of this when upgrading Insights and edx-analytics-data-api. Yeah, we really should get this off of 2.7. There's a good overview of the last attempt in https://github.com/edx/edx-analytics-pipeline/pull/776 , probably should start there and start ticketing the remaining parts. Please coordinate with the eduNEXT team on this, since they did a lot of related work for the other analytics services.

[morenol]

Hi @mraarif, could you add to the travis file a way to run this tests: make test-docker-py3 it is currently failing due to it is using an old version of ansible

or should it be done in another PR?

I think that we should have that test in a different travis worker

[jmbowman]

@morenol Is there a log of the failure somewhere? Do we just need changes made to the image at https://hub.docker.com/r/edxops/analytics_pipeline/tags ? That would seem like a configuration repo PR.

[morenol]

@jmbowman no, I ran that locally using the devstack, I think that we dont need to install anything else, just to run make test-docker-py3 in the tests just like we run make test-docker

But in anyway I think that the changes of this PR are OK, and that the changes for the python3 tests should be done in another PR

[mraarif]

@jmbowman as noted by @morenol in this PR, do we still have plans for this change to go ahead?

[jmbowman]

@mraarif As noted on the other PR, the DE team has suggested holding off on updating this repo because they plan to deprecate it and stop running the service in the relatively near future.