[QST]BlazingSQL+ PostgreSQL

[jeancabouat]

Dear all,

Thank you for your great work on SQL query using GPU, it seems to be awesome ! As a beginner, I've tried to use BlazingSQL for querying data stored in a PostgreSQL database as described there, but it failed.

Please find below my code and the error I got:

from blazingsql import BlazingContext
import cudf
bc = BlazingContext()

bc_my_table = 'bc_table_a'
my_table = 'table_a'
my_schema = 'usr_name'
my_database = 'prod'

bc.create_table(bc_my_table,           # the name you want to give the table
                my_table,              # the name of the table in PostgreSQL
                from_sql = 'postgresql',            
                hostname = os.environ["PGHOST"],
                port = os.environ["PGPORT"],
                username = os.environ["PGUSER"],
                password = os.environ["PGPWD"],
                schema=my_schema,            
                database = my_database)

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-39-e5d1d852875f> in <module>
      7                 password = os.environ["PGPWD"],
      8                 schema = my_schema,
----> 9                 database = my_database)

/opt/conda/envs/blazingenv/lib/python3.7/site-packages/pyblazing/apiv2/context.py in create_table(self, table_name, input, **kwargs)
   2096         """
   2097 
-> 2098         kwargs_validation(kwargs, "create_table")
   2099 
   2100         get_blazing_logger(is_dask=False).info("create_table start for " + table_name)

/opt/conda/envs/blazingenv/lib/python3.7/site-packages/pyblazing/apiv2/context.py in kwargs_validation(kwargs, bc_api_str)
    965                 + "' does not exists. Please make sure you are using the correct parameter:"
    966                 + "\nTo get the correct parameters, check:  "
--> 967                 + params_info
    968             )
    969 

Exception: ERROR: The parameter 'database' does not exists. Please make sure you are using the correct parameter:
To get the correct parameters, check:  https://docs.blazingdb.com/docs/create_table

Actually, I don't understand the error message as the 'database' parameter is required in your example. Could you please help me on this ?

Do I have also to mention the schema as I did it above ?

Thanks and best regards,

Jean

[gcca]

Hi, @jeancabouat. I was reviewing, so the database parameter is supported from branch-0.20. Maybe the current version are you using is not built from that branch.

Also, blazingsql does not support schema parameter for now. It assumes the default public schema.

I have the following script working:

from blazingsql import BlazingContext

bc = BlazingContext()

bc.create_table('nation_blazingsql', 
                'nation_postgresql',
                from_sql = 'postgresql',
                username = 'myusr',
                password = 'mypwd',
                hostname = 'localhost',
                port = 5432,
                database = 'demodb',
                table_batch_size = 3000)

query = 'select * from nation_blazingsql'

df = bc.sql(query)

print(df)

[jeancabouat]

Thanks for your prompt answer. I'm currently using a version built from branch-0.19 which seems not to support postgresql. Am I wrong ? The online doc mentions it but I didn't succeed in using it.

Schema parameter may be a useful functionality in one of the next versions.

Regards,

[felipeblazing]

It only works in 0.20 and onwards that is correct.

[jeancabouat]

OK, thanks.

[jeancabouat]

How can I set the postgresql schema in the create_table ?