?install= option for installing extra packages

[simonw]

Many Datasette plugins won't work, but a few might. Supporting ?install=datasette-json-html would be a neat way to try them out.

[simonw]

I tried this prototype:

diff --git a/index.html b/index.html
index 5a6eac5..de860bb 100644
--- a/index.html
+++ b/index.html
@@ -97,8 +97,9 @@ const urlParams = new URLSearchParams(location.search);
 const initialUrl = urlParams.get('url');
 const csvUrls = urlParams.getAll('csv');
 const sqlUrls = urlParams.getAll('sql');
+const installUrls = urlParams.getAll('install');
 
-datasetteWorker.postMessage({type: 'startup', initialUrl, csvUrls, sqlUrls});
+datasetteWorker.postMessage({type: 'startup', initialUrl, csvUrls, sqlUrls, installUrls});
 
 let loadingLogs = ["Loading..."];
 
diff --git a/webworker.js b/webworker.js
index c51354c..1556468 100644
--- a/webworker.js
+++ b/webworker.js
@@ -53,6 +53,11 @@ async function startDatasette(settings) {
     # Workaround for Requested 'h11<0.13,>=0.11', but h11==0.13.0 is already installed
     await micropip.install("h11==0.12.0")
     await micropip.install("datasette==0.62a0")
+    # Install any extra ?install= dependencies
+    install_urls = ${JSON.stringify(settings.installUrls)}
+    if install_urls:
+        for install_url in install_urls:
+            await micropip.install(install_url)
     # Execute any ?sql=URL SQL
     sqls = ${JSON.stringify(sqls)}
     if sqls:

Thin hit this URL: http://localhost:8009/?install=datasette-json-html

And got this error:

Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 234, in __step
    result = coro.throw(exc)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 500, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 353, in run_async
    await coroutine
  File "<exec>", line 23, in <module>
  File "/lib/python3.10/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
    future.result()
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 234, in __step
    result = coro.throw(exc)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 183, in install
    transaction = await self.gather_requirements(requirements, ctx, keep_going)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 173, in gather_requirements
    await gather(*requirement_promises)
  File "/lib/python3.10/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
    future.result()
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 291, in add_requirement
    await self.add_wheel(
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 316, in add_wheel
    await self.add_requirement(recurs_req, ctx, transaction)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 276, in add_requirement
    raise ValueError(
ValueError: Requested 'datasette', but datasette==0.62a0 is already installed

[simonw]

That looks like it might be related to this issue:

• https://github.com/pyodide/pyodide/issues/2408

Easiest fix though would be for me to ship Datasette 0.62 with all of the changes that have been accumulating in the various alpha versions.

[simonw]

Now that I've released Datasette 0.62 I got this working. I've pushed the feature but it's not documented yet because I want to try it out a bit more.

[simonw]

Here's a demo: https://lite.datasette.io/?install=datasette-json-html#/content?sql=select+json_object%28%27pre%27%2C+%27hello+there%27%29

[simonw]

Plugins that work, with demo links:

• datasette-packages https://lite.datasette.io/?install=datasette-packages#/-/packages datasette-dateutil https://lite.datasette.io/?install=datasette-dateutil#/fixtures?sql=select%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0A-++dateutil_parse%28%222020-03-04%22%29%2C%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%3B datasette-schema-versions - https://lite.datasette.io/?install=datasette-schema-versions#/-/schema-versions
• datasette-debug-asgi - https://lite.datasette.io/?install=datasette-debug-asgi#/-/asgi-scope datasette-query-links - https://lite.datasette.io/?install=datasette-query-links#/fixtures?sql=select%0D%0A++'select+*+from+[facetable]'+as+query%0D%0Aunion%0D%0Aselect%0D%0A++'select+sqlite_version()'%0D%0Aunion%0D%0Aselect%0D%0A++'select+this+is+invalid+SQL+so+will+not+be+linked' datasette-json-html - https://lite.datasette.io/?install=datasette-json-html#/fixtures?sql=select+%27%5B%0A++++%7B%0A++++++++%22href%22%3A+%22https%3A%2F%2Fsimonwillison.net%2F%22%2C%0A++++++++%22label%22%3A+%22Simon+Willison%22%0A++++%7D%2C%0A++++%7B%0A++++++++%22href%22%3A+%22https%3A%2F%2Fgithub.com%2Fsimonw%2Fdatasette%22%2C%0A++++++++%22label%22%3A+%22Datasette%22%0A++++%7D%0A%5D%27+as+output datasette-haversine - https://lite.datasette.io/?install=datasette-haversine#/fixtures?sql=select+haversine%280%2C+154%2C+1%2C+131%29 datasette-jellyfish https://lite.datasette.io/?install=datasette-jellyfish#/fixtures?sql=SELECT%0A++++levenshtein_distance%28%3As1%2C+%3As2%29%2C%0A++++damerau_levenshtein_distance%28%3As1%2C+%3As2%29%2C%0A++++hamming_distance%28%3As1%2C+%3As2%29%2C%0A++++jaro_similarity%28%3As1%2C+%3As2%29%2C%0A++++jaro_winkler_similarity%28%3As1%2C+%3As2%29%2C%0A++++match_rating_comparison%28%3As1%2C+%3As2%29%3B&s1=barrack+obama&s2=barrack+h+obama datasette-pretty-json https://lite.datasette.io/?install=datasette-pretty-json#/fixtures?sql=select+%27%7B%22this%22%3A+%5B%22is%22%2C+%22nested%22%2C+%22json%22%5D%7D%27 datasette-yaml https://lite.datasette.io/?install=datasette-yaml#/fixtures/compound_three_primary_keys.- yaml datasette-copyable https://lite.datasette.io/?install=datasette-copyable#/fixtures/compound_three_primary_keys.copyable?_table_format=github

Plugins that will probably work but I need a better test database for them:

• datasette-render-images
• datasette-render-timestamps
• datasette-render-binary

[simonw]

Lots of plugins don't work yet because they need to be able to load JavaScript and CSS - datasette-cluster-map and datasette-vega for example. See:

• #8

[simonw]

Demo of two plugins installed at once: https://lite.datasette.io/?install=datasette-dateutil&install=datasette-copyable#/fixtures.copyable?sql=select%0A++dateutil_parse(%2210+october+2020+3pm%22)%2C%0A++dateutil_parse_fuzzy(%22This+is+due+10+september%22)%2C%0A++dateutil_parse(%221%2F2%2F2020%22)%2C%0A-++dateutil_parse(%222020-03-04%22)%2C%0A++dateutil_parse_dayfirst(%222020-03-04%22)%3B

[simonw]

This needs to be documented in the README.