KsqlDB interpreter doesn't work with authenticated ksqldb clusters

[CarKsby]

Is the issue already present in https://github.com/cloudera/hue/issues or discussed in the forum https://discourse.gethue.com?

No, not that I can see.

Describe the bug:

The KSQLAPI from the ksql module supports more than a single argument: url, api_key and secret (see the readme here), however, Hue's KsqlApi does not support adding the api_key and secret argument for the client, even if the options field in the KSqlDB interpreter in the hue-ini file has them in the options JSON, in particular, here is the specific point where the client is initialised: Line 73

Steps to reproduce it?

Have a confluent cloud ksqlDB that has basic authentication, and create a set of keys. In the hue-ini file, add under interpretors:

[[[ksqlDB]]]
name=ksqlDB
interface=ksql
options='{"url": "your_url", "api_key": "your_api_key", "secret": "your_secret"}'

When Hue is restarted, you should see a ksqlDB interpretor, but any query or schema refresh will give you a 401 error with logs:

Traceback (most recent call last):
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/ksql.py", line 47, in decorator
    return func(*args, **kwargs)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/ksql.py", line 72, in execute
    db = self._get_db()
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/ksql.py", line 65, in _get_db
    return KSqlClientApi(user=self.user, url=self.url)
  File "/usr/share/hue/desktop/libs/kafka/src/kafka/ksql_client.py", line 73, in __init__
    self.client = client = KSQLAPI(self._api_url)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/ksql/client.py", line 19, in __init__
    self.get_ksql_version()
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/ksql/client.py", line 36, in get_ksql_version
    raise ValueError("Status Code: {}.\nMessage: {}".format(r.status_code, r.content))
ValueError: Status Code: 401.
Message: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/decorators.py", line 119, in wrapper
    return f(*args, **kwargs)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/api.py", line 236, in execute
    response = _execute_notebook(request, notebook, snippet)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/api.py", line 211, in _execute_notebook
    raise ex
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/api.py", line 161, in _execute_notebook
    response['handle'] = interpreter.execute(notebook, snippet)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/ksql.py", line 50, in decorator
    raise QueryError(message)
notebook.connectors.base.QueryError: Status Code: 401.
Message: b''

Then, go to any python client, pip install ksql and check the client works with:

from ksql import KSQLAPI  
client = KSQLAPI(url='your_url', api_key="your_key", secret="your_secret")
client.ksql('show tables')

Which should result in a successful query.

Hue version or source? (e.g. open source 4.5, CDH 5.16, CDP 1.0...). System info (e.g. OS, Browser...). gethue/hue:latest gethue/nginx:latest, using the kubernetes helm chart

I would be happy at giving this a go at changing, I think all that is needed is to create the client by using **options instead of with only the api_url, in a similar vein to how the JDBC interpreters have multiple options in.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity and is not "roadmap" labeled or part of any milestone. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity and is not labeled "Prevent stale". Remove "stale" label or comment or this will be closed in 10 days.