Show job priority in Jobs page and listjobs.json

[my8100]

This PR is created as schedule.json accepts parameter priority in PR #161.

[my8100]

The Jobs page

Output of listjobs.json

{'node_name': 'hostname',
 'status': 'ok',
 'pending': [{'project': 'TESTING2',
   'spider': 'test',
   'id': '-1',
   'priority': -1.0},
  {'project': 'TESTING2', 'spider': 'test', 'id': '-2', 'priority': -2.0}],
 'running': [{'project': 'TESTING',
   'spider': 'test',
   'id': '-2',
   'priority': -2.0,
   'pid': 115780,
   'start_time': '2019-06-24 18:34:51.158649'},
  {'project': 'TESTING2',
   'spider': 'test',
   'id': '2',
   'priority': 2.0,
   'pid': 26548,
   'start_time': '2019-06-24 18:34:56.168935'},
  {'project': 'TESTING2',
   'spider': 'test',
   'id': '1',
   'priority': 1.0,
   'pid': 18600,
   'start_time': '2019-06-24 18:35:01.951266'},
  {'project': 'TESTING2',
   'spider': 'test',
   'id': '0',
   'priority': 0.0,
   'pid': 38252,
   'start_time': '2019-06-24 18:35:06.158507'}],
 'finished': [{'project': 'TESTING',
   'spider': 'test',
   'id': '1',
   'priority': 1.0,
   'start_time': '2019-06-24 18:32:41.167214',
   'end_time': '2019-06-24 18:34:49.547557'},
  {'project': 'TESTING',
   'spider': 'test',
   'id': '2',
   'priority': 2.0,
   'start_time': '2019-06-24 18:32:46.197501',
   'end_time': '2019-06-24 18:34:54.385833'},
  {'project': 'TESTING',
   'spider': 'test',
   'id': '0',
   'priority': 0.0,
   'start_time': '2019-06-24 18:32:51.171786',
   'end_time': '2019-06-24 18:34:59.517127'},
  {'project': 'TESTING',
   'spider': 'test',
   'id': '-1',
   'priority': -1.0,
   'start_time': '2019-06-24 18:32:56.163071',
   'end_time': '2019-06-24 18:35:05.510470'}]}

[my8100]

I would update the test cases ASAP.

[codecov[bot]]

Codecov Report

Merging #343 into master will decrease coverage by 0.04%. The diff coverage is 91.66%.

@@            Coverage Diff             @@
##           master     #343      +/-   ##
==========================================
- Coverage   68.37%   68.32%   -0.05%     
==========================================
  Files          17       17              
  Lines         860      862       +2     
  Branches      104      104              
==========================================
+ Hits          588      589       +1     
- Misses        242      243       +1     
  Partials       30       30

Impacted Files	Coverage Δ
scrapyd/website.py	59.57% <ø> (ø)	:arrow_up:
scrapyd/webservice.py	29.41% <ø> (ø)	:arrow_up:
scrapyd/poller.py	86.66% <100%> (+0.45%)	:arrow_up:
scrapyd/spiderqueue.py	95.23% <100%> (ø)	:arrow_up:
scrapyd/sqlite.py	88.88% <100%> (ø)	:arrow_up:
scrapyd/launcher.py	43.2% <50%> (-0.55%)	:arrow_down:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3ff7c1c...91993bc. Read the comment docs.

[Digenis]

It seems that there's no backward compatible way to include it anywhere at all.

[my8100]

What to backward compatible?

[Digenis]

Anyone using a custom laucher, poller, spiderqueue, sqlitequeue will have their code broken. It's been long since the last release. Let's make the transition smooth. Let's release 1.3 without it and include it in 1.4

[my8100]

Any plan on the release of 1.3? When and what new features (but not implemented yet) to be included?

[Digenis]

https://github.com/scrapy/scrapyd/milestone/4

[jpmckinney]

Can do along with #272 (now #474), so closing here as there are conflicts and we'd need to make new commits either way.

[jpmckinney]

To add some detail to this comment:

Anyone using a custom laucher, poller, spiderqueue, sqlitequeue will have their code broken.

This PR would require changing these interfaces:

• ISpiderQueue.list to return a list of (msg, priority) tuples instead of a list of msg.
• ISpiderQueue.pop to return a (msg, priority) tuple instead of a msg.
• IPoller.next to return a dict with a _priority key.

There's no actual problem with launcher, and "sqlitequeue" (presumably SqliteSpiderQueue) implements ISpiderQueue, so it has no additional problems to the above.

Edit: So, considering there was no demand expressed via a feature request, I'll just abandon this PR rather than try to make it part of #474.