dkan icon indicating copy to clipboard operation
dkan copied to clipboard

Datastore schemas as frictionless table schema

Open dafeder opened this issue 4 years ago • 1 comments

We initially implemented dataset column types in #3686 as full frictionless table schemas, but the change was too disruptive. Stashing that code here to come back to for a v2.

From the original PR:

...we are adopting the Frictionless table schema as a standard for describing column metadata elsewhere in the application, and it seems like a good moment to implement this in the datastore API for outgoing data as well. So we will see the schema object change slightly in these responses. Because this is a breaking change I believe we should make this a minor release (2.12) and be careful about deploying until there is a frontend fix for it. So let's not be too hasty about merging this! If the disruption seems to be too great we can always pull back a bit from the frictionless adoption and keep using existing paradigms for this endpoint for now.

For instance, this:

  "schema": {
    "d7b7e4f8-a947-57a2-b995-6d462a714880": {
      "fields": {
        "objectid": {
          "type": "text",
          "description": "OBJECTID"
        },
        "roadway": {
          "type": "text",
          "description": "ROADWAY"
        },
        "road_side": {
          "type": "text",
          "description": "ROAD_SIDE"
        },
        "lncd": {
          "type": "text",
          "description": "LNCD"
        },
        "descr": {
          "type": "text",
          "description": "DESCR"
        },
        "begin_post": {
          "type": "text",
          "description": "BEGIN_POST"
        },
        "end_post": {
          "type": "text",
          "description": "END_POST"
        },
        "shape_leng": {
          "type": "text",
          "description": "Shape_Leng"
        }
      },
      "primary key": [
        "record_number"
      ]
    }
  }

will now look like this:

  "schema": {
    "d7b7e4f8-a947-57a2-b995-6d462a714880": {
      "fields": [
        {
          "name": "objectid",
          "title": "OBJECTID",
          "type": "number"
        },
        {
          "name": "roadway",
          "title": "ROADWAY",
          "type": "string"
        },
        {
          "name": "road_side",
          "title": "ROAD_SIDE",
          "type": "string"
        },
        {
          "name": "lncd",
          "title": "LNCD",
          "type": "string"
        },
        {
          "name": "descr",
          "title": "DESCR",
          "type": "string"
        },
        {
          "name": "begin_post",
          "title": "Begin post",
          "type": "number"
        },
        {
          "name": "end_post",
          "title": "End post",
          "type": "number"
        },
        {
          "name": "shape_leng",
          "title": "The shape length",
          "type": "number"
        }
      ]
    }

dafeder avatar Oct 06 '21 15:10 dafeder

Probably not doing this anymore -- keeping open for a bit longer to re-assess later when data dictionary work further along.

dafeder avatar May 27 '22 14:05 dafeder