container-demos icon indicating copy to clipboard operation
container-demos copied to clipboard
verified publisher icon ipfs-shipyard

Dataset Containers

Open hosh opened this issue 11 years ago • 7 comments

  1. Make a data container with IPFS, load up data.
  2. Start container elsewhere.
  3. Boom, access to data!

Overview

This is the most useful for open data. Examples:

  1. http://www.fastonline.org/CD3WD_40/CD3WD/INDEX.HTM CD3WD. This is a CD-ROM containing 900+ publications for basic infrastructure technology designed to be distributed for 3rd world countries. This library could be reformatted to work with a web. We can add an nginx server or something similar: startup this container via IPFS, and you now have access to this library.
  2. https://www.govtrack.us/developers/data GovTrack publishes raw data from US Congress, specifically related to legislations that are being created or debated. Because of the size of the data, GovTrack has only a single firehose to PopVox. Can take a snapshot of this and make this available, or better yet, have GovTrack publish this data. [This might be too big to function as a good demo for this]
  3. http://www.nyc.gov/html/doitt/html/open/data.shtml New York City open data
  4. https://www.kickstarter.com/projects/publicresource/public-safety-codes-of-the-world-stand-up-for-safe US Public Safety codes. These are publications painstakingly created and put out into the public. He has other data here: https://public.resource.org/
  5. Zipcode geolocation dataset
  6. International Timezone dataset
  7. OpenStreetMap dataset?

hosh avatar Mar 13 '15 20:03 hosh

  • ipfs mount (fuse in the container -- probably need to run dockerd in privileged mode)
  • prepared datasets

jbenet avatar Mar 14 '15 17:03 jbenet

@jbenet do you think there is a way to have the dataset inside the container?

Though that might be a stretch, carting it around. As a seed it would work very well though. Hmm.

hosh avatar Mar 15 '15 21:03 hosh

@hosh oh yeah we could ship a container with a node initialized and the data already added to the container. We'd just have to generate an ephemeral ID on first run (so multiple containers have different IDs).

jbenet avatar Mar 15 '15 21:03 jbenet

@jbenet roger

hosh avatar Mar 15 '15 21:03 hosh

@hosh another way would be to just issue an ipfs pin add -r command through the http api after the container boots. in that case the data would have to be somethere already though.

jbenet avatar Mar 15 '15 21:03 jbenet

@jbenet I didn't know the http API lets you issue ipfs commands. Huh.

hosh avatar Mar 16 '15 18:03 hosh

Yeah that's how the cli issues commands when daemon is on. (And why the api is separate from gateway route, so we can expose gateway without exposing api)

— Sent from Mailbox

On Mon, Mar 16, 2015 at 11:03 AM, Ho-Sheng Hsiao [email protected] wrote:

@jbenet I didn't know the http API lets you issue ipfs commands. Huh.

Reply to this email directly or view it on GitHub: https://github.com/jbenet/ipfs-container-demos/issues/8#issuecomment-81846339

jbenet avatar Mar 16 '15 20:03 jbenet