framework icon indicating copy to clipboard operation
framework copied to clipboard

Integer out of range error when writing resource to postgres

Open fjuniorr opened this issue 2 years ago • 0 comments
trafficstars

When running:

from frictionless import Resource, formats

resource = Resource(
    {
        "name": "reprex",
        "data": [
            ["account_code", "account_description"],
            ["10000000000000", "Assets"],
            ["20000000000000", "Liabilities"],
            ["90000000000000", "Equity"],
        ],
        "schema": {
            "fields": [
                {"name": "account_code", "type": "integer"},
                {"name": "account_description", "type": "string"},
            ]
        },
    }
)

conn_string = 'postgresql://username:password@host:port/database'

resource.write(conn_string, control = formats.SqlControl(table=resource.name))

I get:

sqlalchemy.exc.DataError: (psycopg2.errors.NumericValueOutOfRange) integer out of range

[SQL: INSERT INTO reprex (account_code, account_description) VALUES (%(account_code__0)s, %(account_description__0)s), (%(account_code__1)s, %(account_description__1)s), (%(account_code__2)s, %(account_description__2)s)]
[parameters: {'account_code__0': 10000000000000, 'account_description__0': 'Assets', 'account_code__1': 20000000000000, 'account_description__1': 'Liabilities', 'account_code__2': 90000000000000, 'account_description__2': 'Equity'}]
(Background on this error at: https://sqlalche.me/e/20/9h9h)

I'm wondering if frictionless-py should detect the proper integer for the database backend (bigint in this case) or if the integer field need a new property such as bigint at the spec level.

fjuniorr avatar Oct 17 '23 23:10 fjuniorr