Npm integration

[dmiro]

npm node.js integration https://www.npmjs.com/

[dmiro]

I don't found an API to NPM. I believe they don't have. What do you think if we use web scraping by using lxml (https://github.com/lxml/lxml)?

example:

https://www.npmjs.com/package/csv

.
.
        <li>
          <strong class="pretty-number daily-downloads">12947</strong>
          downloads in the last day
        </li>
        <li>
          <strong class="pretty-number weekly-downloads">62971</strong>
          downloads in the last week
        </li>
        <li>
          <strong class="pretty-number monthly-downloads">243211</strong>
          downloads in the last month
        </li>
.
.

this is more complicated...

    <li>
      <strong>0.4.6</strong>
      is the latest
        of 40 releases
    </li>

[Alir3z4]

That's such a shame for NPM to be honest. We can sure make a simple wrapper around that using any tools.

How folks at shields.io do it ? I'm gonna have a look at it.

[Alir3z4]

It seems they do: https://api.npmjs.org/downloads/point/last-month/money-currencies

https://github.com/badges/shields/blob/master/server.js#L1349

[Alir3z4]

And https://github.com/badges/shields/blob/master/server.js#L1385 https://github.com/badges/shields/blob/master/server.js#L1420 https://github.com/badges/shields/blob/master/server.js#L1449

It seems they have a separate endpoint for each usage. That't not bad. It will require to do some tiny changes on the its service integration to change the API URL on each action

[dmiro]

Ok, perfect! NPM had really hidden away the API X-) I don't found any reference to it in your site. thanks...

[dmiro]

shit, I cannot install correctly in a Windows 7

(badge) C:\badg\badge-master>pip install -r requiriments.txt

(badge) C:\badg\badge-master>pip freeze ecdsa==0.13 Fabric==1.10.2 klein==15.3.1 paramiko==1.16.0 pycrypto==2.6.1 redis==2.10.5 requests==2.9.1 Twisted==15.5.0 Wand==0.4.2 Werkzeug==0.11.8 yarg==0.1.9 zope.interface==4.1.3

finally, I need install ImageMagick wrapper because this is not in requiriments.txt. I also installed ImageMagick binaries.

(badge) C:\badg\badge-master> pip install magickwand

(badge) C:\badg\badge-master>pip freeze ecdsa==0.13 Fabric==1.10.2 klein==15.3.1 magickwand==0.2 paramiko==1.16.0 pycrypto==2.6.1 redis==2.10.5 requests==2.9.1 Twisted==15.5.0 Wand==0.4.2 Werkzeug==0.11.8 yarg==0.1.9 zope.interface==4.1.3

(badge) C:\badg\badge-master>python server.py Traceback (most recent call last): File "server.py", line 5, in from badge.views import app File "C:\badg\badge-master\badge\views.py", line 10, in from service import service_registry File "C:\badg\badge-master\service__init__.py", line 10, in from service.aur import AURService File "C:\badg\badge-master\service\aur__init__.py", line 4, in from service.base import ServiceBase File "C:\badg\badge-master\service\base.py", line 12, in from painter.draw import Draw File "C:\badg\badge-master\painter\draw.py", line 12, in from wand.drawing import Drawing File "C:\badg\badge\lib\site-packages\wand\drawing.py", line 13, in from .api import library, MagickPixelPacket, PointInfo, AffineMatrix File "C:\badg\badge\lib\site-packages\wand\api.py", line 205, in 'Try to install:\n ' + msg) ImportError: MagickWand shared library not found. You probably had not installed ImageMagick library. Try to install: http://docs.wand-py.org/en/latest/guide/install.html#install-imagemagick-on-windows

...

[Alir3z4]

Imagemagick binaries should be installed on the OS itself. Wand itself is the only thing required on the python side. Haven't used Windows on this case, it seems this question on stackoverflow might be helpful for you: http://stackoverflow.com/questions/12645195/how-to-install-python-imagemagick-at-windows-7-i-followed-these-instruction

You might find c9.io useful if you couldn't get along on Windows.

[dmiro]

Thans!.

At last! I installed correctly The problem was that the wrapper was 32-bit and binary was 64-bit

.... and it's not necessary install MagickWand

Yes, I like it c9.io

[Alir3z4]

Awesome. Glad you got it working.

Lemme know if you hit any issues

[dmiro]

Ops.... I need Redit to run?

File "C:\badg\badge\lib\site-packages\redis\connection.py", line 442, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 10061 connecting to localhost:6379. No se puede establecer una conexi¾n ya que el equipo de destino deneg¾ expresamente dicha conexi¾n.

2016-04-21 14:15:32+0200 [HTTPChannel,9,127.0.0.1] Unhandled Error writing response

Al least, unittest is OK:

(badge) C:\badg\badge-master>python test_pypi_service.py

.............

Ran 13 tests in 0.000s

OK

(badge) C:\badg\badge-master>python test_aur_service.py

......

Ran 6 tests in 0.000s

OK

(badge) C:\badg\badge-master>python test_registry.py

......

Ran 6 tests in 0.000s

OK

[Alir3z4]

@dmiro I haven't tested on Windows, there might some piece of code that works mostly for *unix based systems only.

For example have a look at the: https://github.com/SavandBros/badge/blob/master/basement/settings.py#L7 which is to keeping some files in the /tmp dir. However I don't think that would cause much of problem while developing new features. While developing you can just turn it off or leave it as it is.

As for redis, badge uses it for caching the results from the service integration. it works with no issue on CI that doesn't have the redis installed, that is the reason why tests didn't fail.

Feel free to hack without redis installed.

Also for running the the test, you can easily run all of them with:

$ fab run_tests

[Alir3z4]

@dmiro where you able to get any further with this ? any other blocker ?

[Alir3z4]

I can hack this Redis dependencies with having a dummy cache backend for local and testing environments.