asyncpg icon indicating copy to clipboard operation
asyncpg copied to clipboard

greenplum->copy_records_to_table support needed

Open HuangKaibo2017 opened this issue 7 years ago • 1 comments

  • asyncpg version:0.17.0
  • Greenplum version:5.7
  • Python version:3.7
  • Platform:Windows 10/windows 10 WSL
  • Do you use pgbouncer?:no
  • Did you install asyncpg with pip?:yes
  • If you built asyncpg locally, which version of Cython did you use?:n/a
  • Can the issue be reproduced under both asyncio and uvloop?:n/a

try to run copy_records_to_table against greenplum 5.7 which report error as: File "D:\Anaconda3\envs\DataLightFire37\lib\site-packages\asyncpg\connection.py", line 848, in _copy_in_records copy_stmt, None, None, records, intro_stmt, timeout) File "asyncpg\protocol\protocol.pyx", line 504, in copy_in asyncpg.exceptions.PostgresSyntaxError: syntax error at or near "("

Try to run those against PostgreSQL 9.2. It passed. Also, execute and executemany work fine with Greenplum.

HuangKaibo2017 avatar Jul 30 '18 13:07 HuangKaibo2017

import asyncpg

async def copy_records_to_table(connection, table_name, records):
    try:
        # Check if it's Greenplum
        is_greenplum = await connection.fetchval("SELECT version() LIKE '%%Greenplum%%'")

        # Use different copy statement syntax based on the database
        if is_greenplum:
            copy_stmt = f"COPY {table_name} FROM stdin WITH CSV"
        else:
            copy_stmt = f"COPY {table_name} FROM stdin WITH CSV DELIMITER ','"

        # Start the copy operation
        await connection.execute(copy_stmt)

        # Prepare and send the data
        for record in records:
            await connection.copy_records_to_table(table_name, records=[record])

        # Complete the copy operation
        await connection.copy_records_done()

    except Exception as e:
        print(f"Error during copy: {e}")
        raise

async def main():
    conn = await asyncpg.connect(
        user='your_user',
        password='your_password',
        database='your_database',
        host='your_host'
    )

    table_name = 'your_table'
    records = [
        (1, 'John'),
        (2, 'Jane'),
        # Add more records here
    ]

    await copy_records_to_table(conn, table_name, records)

if __name__ == '__main__':
    import asyncio
    asyncio.run(main())

ljluestc avatar Jan 13 '24 07:01 ljluestc