hash icon indicating copy to clipboard operation
hash copied to clipboard

Data mapping prototype / MVP

Open CiaranMn opened this issue 2 years ago • 0 comments

🌟 What is the purpose of this PR?

Introduce a basic tool for "data mapping", which is the term we use to describe converting from one data structure to another.

This allows data to be loaded into blocks when it doesn't match the expected structure.

How it works

  • Click on the context menu and click 'Data map' to show the UI
  • Property paths are derived from the block's schema, e.g. name, email, employer.location
  • A "source tree" is constructed from the blockEntity (root entity sent to a block) and blockGraph (the entities linked to a block, resolved to a depth of 1), and paths derived from it, i.e. the graph of entities is turned into a tree with the block entity at the root.
  • The user can choose paths from the source tree to assign to each path in the block schema, and the values from the source will be sent to the block under the relevant path.
  • Default values can also be provided to satisfy block requirements in the absence of anything suitable in the source tree.
  • Click the 'preview' icon (eyes) at the bottom to preview the source tree, transformed data, and any validation errors (e.g. missing properties).
  • Each data map is a conversion from a specific entity type to a specific block component, e.g. userEntityType:https://blockprotocol.org/foo-block, and will be applied wherever that combination appears (in the same browser, because the maps are stored in local storage).

See known issues for caveats / issues.

🔗 Related links

🔍 What does this change?

  • Adds data mapping UI components and supporting functions
  • Adds data mapping UI visibility to the context shared across block handle and content, toggle-able from the context menu
  • Send the block schema into ComponentView
  • Transform block data if a map exists for the type:block combo
  • Delete a couple of unused files that confused me when writing this
  • A couple of explanatory comments that would have been useful for me when writing this

⚠️ Known issues

  1. Because maps are stored in local storage, only the user who created them will have the data transformed when being loaded into the block. Other users visiting the same page will not, and will have to create their own map for an entity to appear correctly in a block (where it requires transformation). This is a temporary solution until the new API is available, and we figure out a definition, scoping and storage solution for data maps (which itself will probably be somewhat temporary).
  2. Due to limitations in our current schema system, particularly not being able to properly represent links between entities (to be fixed in the new type system):
    • we use the source entity data rather than its schema. This means that the available source paths to map are the ones that are defined on the source entity, which may not be all the possible fields the type has.
    • the map can only target properties on the root entity the block expects - it cannot construct synthetic links and linked entities.
  3. Loading / swapping entities does not work for text blocks (existing issue). Don't try this with that.

🐾 Next steps

  • Build a more complex block to try out more complex data mapping issues.
  • Figure out how this will work in more detail.

❓ How to test this?

  1. Put a non-text block on the page
  2. Click 'add entity' in the context menu to load some non-suitable entity into the block
  3. Click 'map data' in the context menu to assign properties, then close it to see the results
  4. Check the 'preview' button (eyes in the bottom right)

📹 Demo

https://user-images.githubusercontent.com/37743469/183946546-eb275031-c9be-4a09-ba1b-ca81187607bf.mp4

CiaranMn avatar Aug 10 '22 15:08 CiaranMn