Start with simple read only use cases and abstraction layers

[bmann]

I had a longer discussion with @expede and talked a bit to @csuwildcat about getting to a place of interop sooner rather than later rather than trying to "eat the whole elephant".

Between myself and @expede we'll do some work on figuring out how to write this up more fully, but it boils down into a couple of pieces.

1. File Locations and Permissions based on a File Hierarchy Model

We're going to propose using identifiers that map to file paths, that should work for "static" on disk files or server paths, but also work essentially as arguments for permissions.

e.g. /Public/Apps/Blog/ is a way to ask for permission / access to everything in the Blog folder and below. /Public/Apps/Blog/somepost.md is access to a file.

I think @csuwildcat wants to see things like /Collection/Images to be somewhat standardized paths, which I think is useful to work towards. How to implement / execute / return such a query is up to each system, with the simplistic version being "files" at that location.

2. Read access

Requesting read access to a public file hierarchy: it's public, so it's more about returning the location of these files to be able to be read (and possibly "relaying" them?)

Literally all projects should be able to complete this very basic interface so we get an early win -- a request is made for a file location, and all contents are returned.

Requesting encrypted read access: right at this stage we'll need to figure out key exchange and probably revocation, both of which are tricky.

I guess the next step is to "specify" this, can we put this up for discussion on next week's implementors agenda please.

I apologize for not being able to spend more time on this myself.

cc @carsonfarmer @oed

[csuwildcat]

@bmann just wanted to clarify our feature intent with pathing: we certainly want a URL and message-based API that allows for path-esque queries, but my personal desire with Collections is not to create high-level groups, like Social, but instead opt for addressing data directly by schema/type: /collections/schema.org/SocialMediaPost, /collections/gs1.org/Product, etc. We can have both, but addressing data by schema vs class groupings is something we want to see as an explicitly defined option/convention in the spec.

[bmann]

@csuwildcat great, thanks for adding that detail -- I forgot about the schema signature. There's probably some nuance there, like /collections/Product/gs1.org (optional schema goes last or something) -- that's exactly the stuff we can put in an actual spec doc, just wanted to make sure to capture the general intent here.

[csuwildcat]

I was thinking about the query API exposed to callers and I feel like a file path may be too low level, given there is a desire (at least from us and a few others) to query/index by type/schema and other facets that extend beyond what paths can express. Also, the DID standard is based on DID URLs, which inherently means that a DID URL query may contain parameters. A URL/message-based top-level query approach would allow for a lot more freedom/flexibility than strict file path lookups, so perhaps that would make more sense. for example, I am not sure how I could use paths for the following: /collections/https://schema.org/SocialMediaPosting, given file paths would not allow direct addressing of data by 1:1 type/schema reference.

[TallTed]

Is there a pretense here that RDF does not exist, and hence must be reinvented? I would expect such queries as were described in the call today to be along the lines of "list all the things of type:x that I'm allowed to see", where typing uses whatever ontology makes sense, and where some degree of reasoning and ontological intersection allows "type:x" to be something like "interpersonal messages" which class includes subclasses like Tweets, email messages, txt/sms messages, etc. None of this uses paths per se, though paths might come into play as identifiers for the individual messages, which would then be queryable using those identifiers.

[csuwildcat]

@TallTed we explicitly do not want to get into defining one-off rolled up categories that act as buckets of object types defined at high-levels in the spec. The desire is to address data directly by its contextual schema URI so that there is no real API to learn, and discovery can be inferential. This means that if you know the contextual schema URI of some type of data you want to interact with, which is defined by schemas used across existing industries, not this spec, you can ask directly for it. For example: in the online event/ticket sales industry if they all use https://schema.org/Event objects to represent data (or whatever may be common in a given industry) you just literally ask for https://schema.org/Event objects. The goal is to avoid obscuring direct addressing with any other layers of API or syntax - just ask for what you know you want.

[TallTed]

(Among other considerations,) "what you know you want" (https://schema.org/Event) may be one type on one system (https://schema.org/SportsEvent), and another type on another system (https://schema.org/MusicEvent), where both are sub-types of "what you know you want" (https://schema.org/Event) — and if you ask for "what you know you want" (https://schema.org/Event), you may get neither because their specialized types ((https://schema.org/SportsEvent), (https://schema.org/MusicEvent)) don't map exactly to "what you know you want" (https://schema.org/Event), unless you've looked into the schema.org ontology.

You can't just ask for "what you know you want" and expect success.

[csuwildcat]

@TallTed sure you can: if you want Event, MusicEvent, and SportsEvent type objects, you'd ask for all three. This is based on direct object type requests, not a system where you cascade lookups across classes of things that could externally be derived. The way to win here is to simply avoid that complexity and have apps/services ask for the verbose list of types they want. By doing so, we keep the datastore from turning into an overly complex system that requires dealing with magic and understanding external schema hierarchies.

[TallTed]

You're lowering spec author and client (especially) and server app developer pain in favor of user pain.

You might think that these are all https://schema.org/Event, and of course, they are, but you won't get them if you just ask for those.

Sometimes you want the specific sub-class, and it makes sense to ask for that.

Sometimes you want all sub-classes that fit into one super-class, in which case, it makes sense to ask for the super-class -- not to enumerate all the sub-classes plus the super-class (so you don't miss any that weren't sub-classed).

I'm seeing design decisions that don't appear to have justification other than "we think this is the way to go", and that is concerning.

[gobengo]

@bmann Do you think what's there now has unblocked your 'read only use cases'?

[Identitywoman]

@TallTed we think the point you make

(Among other considerations,) "what you know you want" (https://schema.org/Event) may be one type on one system (https://schema.org/SportsEvent), and another type on another system (https://schema.org/MusicEvent), where both are sub-types of "what you know you want" (https://schema.org/Event) — and if you ask for "what you know you want" (https://schema.org/Event), you may get neither because their specialized types ((https://schema.org/SportsEvent), (https://schema.org/MusicEvent)) don't map exactly to "what you know you want" (https://schema.org/Event), unless you've looked into the schema.org ontology.

You can't just ask for "what you know you want" and expect success.

We think you are raising important points - please open an issue about them - so we can discuss more fully.

[Identitywoman]

I had a longer discussion with @expede and talked a bit to @csuwildcat about getting to a place of interop sooner rather than later rather than trying to "eat the whole elephant".

Between myself and @expede we'll do some work on figuring out how to write this up more fully, but it boils down into a couple of pieces.

1. File Locations and Permissions based on a File Hierarchy Model

We're going to propose using identifiers that map to file paths, that should work for "static" on disk files or server paths, but also work essentially as arguments for permissions.

e.g. /Public/Apps/Blog/ is a way to ask for permission / access to everything in the Blog folder and below. /Public/Apps/Blog/somepost.md is access to a file.

I think @csuwildcat wants to see things like /Collection/Images to be somewhat standardized paths, which I think is useful to work towards. How to implement / execute / return such a query is up to each system, with the simplistic version being "files" at that location.

2. Read access

Requesting read access to a public file hierarchy: it's public, so it's more about returning the location of these files to be able to be read (and possibly "relaying" them?)

Literally all projects should be able to complete this very basic interface so we get an early win -- a request is made for a file location, and all contents are returned.

Requesting encrypted read access: right at this stage we'll need to figure out key exchange and probably revocation, both of which are tricky.

I guess the next step is to "specify" this, can we put this up for discussion on next week's implementors agenda please.

I apologize for not being able to spend more time on this myself.

cc @carsonfarmer @oed

@bmann would you like to come to one of our meetings and share more about how you could imagine doing this sort of interop.

[bmann]

Hi @Identitywoman

I really encourage file paths (much like HTTP URI access on a web server) as a starting point.

I believe schema negotiation should be handled separately (although I acknowledge that a capabilities model blurs this a bit). Key exchange for read access of encrypted files and getting that right should be more than enough as a first step -- we're trying to handle transport layer efforts here.

LMK when next meeting is and I can come speak to this a bit more.

[andorsk]

@bmann, Given the changes in the spec https://identity.foundation/decentralized-web-node/spec/#interfaces, this issue was agreed upon at the last DWN call as no longer relevant given the work done on the interfaces. Motion to close this out, will keep this open for a week if you have any reservations about this.

[bmann]

Yeah you can close, obviously this whole effort morphed into whatever DWNs are.