peepin icon indicating copy to clipboard operation
peepin copied to clipboard

create JSON schemas for peep IPFS data

Open sponnet opened this issue 6 years ago • 4 comments

If we can validate the IPFS data with a JSON schema - we'd be able to validate data before attempting to parse it. Should be a thing to build a data standard for peeps.

sponnet avatar Aug 23 '18 15:08 sponnet

Currently I'm decorating the raw IPFS data with

  • the function call arguments
  • avatarUrl ( which points to an S3 URL ) gets fetched, stored on IPFS and the hash is added as an extra field avatarIPFSHash
  • idem for backgroundUrl => backgroundIPFSHash

So the final payload that is extracted for this function is similar to

  {
    "info": "",
    "location": "San Francisco",
    "realName": "AntonMu",
    "website": "http://antonmuehlemann.de/",
    "avatarUrl": "peepeth:24BWprZL:jpg",
    "backgroundUrl": "peepeth:dUiQ8LHs:png",
    "messageToWorld": "",
    "untrustedTimestamp": 1521156101,
    "_name": "Anton\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000",
    "avatarIPFSHash": "QmYbJoSDQEFEkMqmobVGZoXhLSBVv7HSmyq4pQiNKryXZv",
    "backgroundIPFSHash": "QmXfZZqQiaQrDG6uqaAaosPRGsLrbcztTfoPW8nDXxv8BQ"
  }

This data could be saved for later processing.

Should we also provide a JSON schema for the output data ? Or only a schema to validate the input to the contract ?!

sponnet avatar Aug 23 '18 21:08 sponnet

I think both will be valuable in the long run. I'll start working on a schema for input validation right away, and once it's part of the core we can move on to schema validations for everything else.

sevvie avatar Aug 24 '18 02:08 sevvie

I've started by creating the schemas themselves, though I think these schemas will need to be added to Peepeth as well, by Bevan, so the schemas have an $id to match against.

https://github.com/sevvie/peepin/blob/master/src/validators/schemas/peep.schema.json https://github.com/sevvie/peepin/blob/master/src/validators/schemas/decoration.schema.json

sevvie avatar Aug 24 '18 03:08 sevvie

I created an epic with a single story for each type we have now. #2

If you want to take a stab at making the schema's for the input formats - you can comment or post insights / questions in its related issue ! An attempt to keep things organized ;)

I can create a basic version of each type that dumps the data in a local database or on disk - so we can easily feed all peeps / replies / tips / share / batchJSON through the schema to test if it's complete.

Then we'll also have an idea how consistent the data is. Since writing hashes to the contract is permissionless, in theory it could contain random data.

sponnet avatar Aug 24 '18 11:08 sponnet