framework icon indicating copy to clipboard operation
framework copied to clipboard

Public API for datapackage <> sql metadata conversion

Open ezwelty opened this issue 4 years ago • 3 comments

Overview

Say you have a frictionless.Package and want the equivalent SQL metadata (as sqlalchemy.Metadata). Maybe you want to generate the SQL commands that would generate the database using sqlalchemy.schema.CreateTable. Here is how this can be currently accomplished:

import frictionless

def package_to_sql(package, db):
  storage = frictionless.plugins.sql.SqlStorage(db)
  for resource in package.resources:
    _ = storage._SqlStorage__write_convert_schema(resource)
  return storage._SqlStorage__metadata

So although the conversions already exist within frictionless.plugins.sql, accessing them has two major downsides: the methods are private and they require an active database connection, even though only the sql dialect is needed (e.g. 'mysql', 'postgresql', etc).


Please preserve this line to notify @roll (lead of this repository)

ezwelty avatar Jun 11 '21 15:06 ezwelty

@ezwelty Good idea. We can extract this as a static helper

roll avatar Jun 11 '21 15:06 roll

@roll I could work on this if you'd like. Would you want these as @staticmethod of SqlStorage, helper functions directly in the plugins.sql module, or elsewhere ?

ezwelty avatar Jun 14 '21 08:06 ezwelty

It's great @ezwelty,

I think we can have a high-level package.to_sql_metadata in the Package class and on the plugin level, a static method is a good idea in my opinion.

I'm assigning you to show that's in work :+1:

roll avatar Jun 14 '21 09:06 roll

PARTIALLY IMPLEMENTED the follow-ups - https://github.com/frictionlessdata/framework/issues/1365

roll avatar Jan 05 '23 16:01 roll