scraperwiki-python

scraperwiki-python copied to clipboard

Bumps ubuntu from 14.04 to 22.04. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...

dependabot[bot]

WIP: Replace Travis with GitHub Actions

1

StevenMaude

The whole idea around `unique_keys=["col"]` doesn't work for buffered rows. Since not all modifications are committed directly to the database, this make the unique constraint working unreliably. To make it...

jfilter

Output for Python 3.5, but same test failure for Python 3.6, 3.7 and 3.8. ``` $ nosetests --exe ...................................F. ====================================================================== FAIL: test_empty (tests.TestUniqueKeys) ---------------------------------------------------------------------- Traceback (most recent call last): File...

StevenMaude

pdftoxml in utils.py is not portable to Windows.

3

1. The [`/dev/null`](https://github.com/scraperwiki/scraperwiki-python/blob/master/scraperwiki/utils.py#L47) needs to be `NUL` on Windows. 2. [`NamedTemporaryFile`](https://github.com/scraperwiki/scraperwiki-python/blob/master/scraperwiki/utils.py#L38-L54) behaves [differently in Windows to Unix](http://stackoverflow.com/questions/94153/how-do-i-persist-to-disk-a-temporary-file-using-python/#9155528).

StevenMaude

Proposal: scraperwiki.sql.variable for getting/setting variables.

6

Basically, like scraperwiki.sqlite.save_var, but cleaned up. - same interface in Python and JavaScript. - share basic datatypes (string, number, list, dict) between Python and JavaScript. Bonus for "native" dates. -...

fawkesley

Python3 unicode

1

In python3 we do not have unicode strings, so we do not need to decode them.

MaxKurama

The sql.save function is a useful quick way of saving data. However, it has very odd functionality if you try to build data incrementally from various sources. The use case...

Ar-Kaine

utils.py uses both urllib2 and requests to make requests

3

Could be tidied.

StevenMaude

`table-xtract-tool` has the JavaScript UI put the boxUrl in a `settings.json` file, just so that the Python code can read it out. This would be much less error prone (and...

drj11