solidcommunity.net icon indicating copy to clipboard operation
solidcommunity.net copied to clipboard
verified publisher icon solid

migration to CSS

Open bourgeoa opened this issue 3 years ago • 7 comments

CSS has a file-system mode that could allow migration/mapping at the filesystem level.

This is a work in progress, to summarize some of the questions raised by solidcommunity.net migration/mapping from NSS to CSS

file system mapping

  • resource mapping is using the same mapping convention : resource + contentType is mapped
    • to file if file mimeType = contentType
    • else to file$.ext with mimeType of ext = contentType
    • profile is stored in card$.ttl
    • .meta and .acl are turtle resources stored as files at the same relative location.
  • pod mapping
    • CSS a pod is allways stored in a folder named user
    • while NSS pod data is stored in a folder named subdomain

IdP

  • authentication have different entry points (login) :
    • CSS is using email as entryPoint versus WebId for NSS
    • NSS may have multiple pod's with the same email, not CSS solidcommunity.net on 20220603 has 59030 accounts but only 48445 emails
  • idp migration
    • data migration (?)
    • or batch pod creation see issue https://github.com/solid/community-server/pull/1165

WebId structure

  • NSS https://user.server/profile/card#me
  • CSS can use subdomain configuration like NSS or domain/user configuration https://server/user/profile/card#me

WebId profile

  • in CSS extended profile (preferences) is not created by default : supported via custom pod templates (this is not a migration isssue)
  • Resolved : NSS was missing solid:oidcIssuer for old pod's. see issue comment and chat Resolved on current pod with https://github.com/RubenVerborgh/add-oidc-issuer

CSS functionalities

  • CSS container as turtle do not display contentType by default
    • v4 has it by default
  • app authorisation
    • CSS do not use the trustedApp mechanism.
      • Note that this goes beyond the v0.9 spec; the mechanism is not secure and will not be supported in the default CSS configuration. However, no breakage occurs; the statements are simply ignored.
    • NSS uses trustedApp at the pod or ACL document level
  • notifications ?
    • CSS supports the unsecure Solid WebSockets API 0.1 in all versions up to and including v4.0.0, but the mechanism is scheduled to be removed from the default configuration in v5.0.0. (Note that this does conform to 0.9 since support is indicated as SHOULD, not MUST.) However, it will still be available as a custom module that can be plugged in, so keeping that functionality will not be an issue.
  • owner implementation ? to allow unlock ACL Control : yes it does

mashlib issues

  • mashlib can be linked to CSS has an external webApp and CSS v3.0.0 has a recipe using mashlib by default
  • extended profile is not available in CSS missing properties are :
    • space:storage
    • solid:account
    • preference files
  • CSS domain is allways seen as a podStorage : pim:Storage is available on container representation even with no profile

bourgeoa avatar Mar 05 '22 12:03 bourgeoa

On the issues with mashlib

  • You can easily configure mashlib in order to generate new files for a pod on pod creation using handlebars templates. I have added handlebars files to my configuration: https://github.com/redpencilio/app-solid which are responsible for generating preference files. These are located in templates/pod. In order to use these templates, you can configure a TemplatedResourcesGenerator in the components.js config file to use the correct pod-templates folder.

elpoelma avatar Mar 22 '22 14:03 elpoelma

Top post edited with @RubenVerborgh responses

One new question : does CSS implement : "owner implementation ? to allow unlock ACL Control"

bourgeoa avatar May 30 '22 17:05 bourgeoa

Thanks

bourgeoa avatar May 30 '22 19:05 bourgeoa

as of 20220603 there are 59030 accounts but only 48445 emails

This covers 2 different situations : no email, or same email on multiple pods

# find /mnt/v*/solid.community -maxdepth 1 -type d | wc -l
# find /mnt/v*/.db/oidc/users/users-by-email | wc -l

bourgeoa avatar Jun 04 '22 17:06 bourgeoa

The plan I was tracking on this topic is:

  • [x] First experiments with mashlib on CSS (reporting bugs)

  • [x] Prioritize and fix high prio problems of SolidOS(mashlib) to be able to move to CSS

  • [ ] SolidOS team concerns about CSS

    • [x] Tim migrated and switched timbl.com to CSS
    • [x] all SolidOS team has local CSS installations - gather valuable learning experience
    • [ ] collecting issues on multi-user setup
    • [ ] fix any big multi-user setup problems
    • [x] git solidOS clone needs to automatically setup CSS instead of NSS for local dev
    • [x] solidcommunity.net test environment setup for CSS

  • [x] Working on checking/documenting Solid Apps working with CSS

    • [x] Working with the latest DPoP

  • [ ] Migrate Pods

    • Existing Pods Data
      • [x] added solid:oidcIssuer missing tripple to solidcommunnity.net and inrupt.net
      • [x] collect info on pod users
      • [ ] collect potential issues
      • [ ] find solutions to issues
      • [x] develope a migration script for data (with testing)
    • Existing Pods username/password
      • [ ] collect concerns about migration of login experience
      • [ ] solve/reconcile any login to CSS issues
    • [ ] migration plan approved by SolidOS team & Solid Team
      • [ ] prepare communication to community

Ongoing:

  • WebSockets https://github.com/solidos/solid-ui/issues/360 - low prio (for this task but high prio in general)
  • Discovery infrastructure https://github.com/solidos/solidos/issues/66 and https://github.com/solidos/mashlib/issues/79 -> low prio (for this task but high prio in general)
  • header map https://github.com/solidos/solidos/issues/79 - low prio
  • header/footer: https://github.com/solidos/solidos/issues/124 - low prio

Done:

  • https://github.com/solid/solid-panes/issues/301

All tickets are monitored here: https://github.com/orgs/SolidOS/projects/1/views/9

@timbl @justinwb @RubenVerborgh @bourgeoa @oolivo let's keep ourselves up to date with this ticket/comment. I will make sure to propagate this in SolidOS and Solid Team meetings to keep everyone (us and everyone else) informed.

timea-solid avatar Jun 27 '22 13:06 timea-solid

This is the number of unique email addresses recorded in the IdP.

Le mer. 8 juin 2022 à 18:50, Jeff Zucker @.***> a écrit :

Is that 48,445 accounts that have email addresses? Or is it 48, 445 unique email addresses?

— Reply to this email directly, view it on GitHub https://github.com/solid/solidcommunity.net/issues/62#issuecomment-1150157202, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ5TZTVQSTO6BUEFI5357LVODFLPANCNFSM5P7TMRFQ . You are receiving this because you were mentioned.Message ID: @.***>

bourgeoa avatar Oct 11 '22 07:10 bourgeoa

Now we have https://github.com/RubenVerborgh/NSS2CSS -- thanks to @RubenVerborgh -- I have checked off the box above "develop a migration script for data". I had understood originally that CSS and NSS used different password hash algorithms, and so a conversion from one to the other was impossible. But now it seems that was not in fact the case, and the conversion script is possible. That is a major step

timbl avatar Nov 03 '23 19:11 timbl