ksql-python icon indicating copy to clipboard operation
ksql-python copied to clipboard

Changes for Python 3.10 support

Open dgkncelik opened this issue 2 years ago • 3 comments

Change http connection dependency Add httpx in requirements.txt Change api.py

dgkncelik avatar Jun 03 '22 19:06 dgkncelik

@bryanyang0528 this one is critical for migration to Python 3.10.

@dgkncelik @bryanyang0528 has been working to resolve CI pipeline issues and merging pull requests. can you fix the branch conflicts? I think once we have yours we can get a new release out!

KenCox94 avatar Aug 03 '22 18:08 KenCox94

Sorry for the late response, let me correct it ASAP

dgkncelik avatar Aug 04 '22 16:08 dgkncelik

@KenCox94 I don't think I can migrate all CI processes to Gitlab Action in this days. So I add python3.10 on Travis CI temporary. @dgkncelik Now you can test the code on travis CI

bryanyang0528 avatar Aug 06 '22 08:08 bryanyang0528

@cordawyn thank you for bringing this up. @dgkncelik are there any updates on the conflicts? Also, do you think you can correspond with @cordawyn to resolve the issues he mentioned above? The quicker this gets resolved, the sooner we can move to a new release.

@bryanyang0528

KenCox94 avatar Aug 16 '22 14:08 KenCox94

@KenCox94 with all the changes I suggested above, I made it to the point where it tries to get the query response, I guess. Here's the error I get now:

{"@type":"statement_error","error_code":40001,"message":"Error starting pull query: Cannot determine which host contains the required partitions to serve the pull query. \nThe underlying persistent query may be restarting (e.g. as a result of ALTER SYSTEM) view the status of your by issuing <DESCRIBE foo>.\nStatement: SELECT * FROM ksql_test_table_1;: SELECT * FROM ksql_test_table_1;","statementText":"SELECT * FROM ksql_test_table_1;","entities":[]}

I remember I used to get this error when these env variables for Kafka were not set as suggested:

  • transaction.state.log.replication.factor: 1
  • transaction.state.log.min.isr: 1
  • offsets.topic.replication.factor: 1

... but they are properly set now. Could be another reason I guess.

The query itself is created in KSQL and I can "select" from it using ksql CLI. It worked fine in ksql-python with non-HTTP/2 querying though.

cordawyn avatar Aug 19 '22 09:08 cordawyn

OK, here is where I am at with this..........this API will NOT WORK w/ python >=3.10 if we don't implement some of these changes.

So, @dgkncelik, If we don't hear anything back concerning the resolution of the branch conflicts within 48 hours I am going to close this pull request, make the changes you implemented, and open it as a new pull request. Please don't take this as me being terse but we have been sitting on this for a while now. The lapse isn't just on you; I understand that, but things have to get moving.

@cordawyn @bryanyang0528 this pull request title is so encompassing that we really need to make this more of an epic. There isn't going to be a catch-all pull request to settle all problems with 3.10 as @cordawyn has demonstrated.

@cordawyn could you take on what you have put forth? Just fix and submit a new pull request based on those. If not, could you submit these as issues so we remember these need addressed?

I think it's important to get this request pushed through as it's such a simple fix but it will be renamed.

Please let me know your thoughts on this and apologies for the sabbatical. I will be continuing some more aggressive work on this until the holidays here in the US.

KenCox94 avatar Sep 16 '22 20:09 KenCox94

Closing will open a new pull request.

KenCox94 avatar Sep 20 '22 17:09 KenCox94

@KenCox94, any word on a new PR for migrating this library from hyper to httpx? I have just opened an issue that requires the httpx migration #120

Robbie-Palmer avatar Dec 09 '22 17:12 Robbie-Palmer