opik
opik copied to clipboard
comet-ml
Unable to access Opik backend running on a remote Jupyter server
Hi,
We're currently evaluating Opik as a potential LLM evaluation tool and I'm setting up an Opik server for internal teams to try and provide feedback. Since this is just a PoC phase, we decided to skip a production deployment on Kubernetes and instead run Opik locally on a SageMaker notebook instance.
In our test setup, the goal is to access the Opik UI and run experiments from different environments within the same AWS account (e.g., other notebook instances).
We successfully brought up the server and made the UI accessible in the browser by setting:
VITE_BASE_URL=/proxy/5173/
VITE_BASE_API_URL=/proxy/5173/api
The Opik UI is now accessible at:
https://<notebook-name>-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/default/home
However, when trying to access this Opik server from another SageMaker notebook instance (to send metrics), the connection fails.
I set the OPIK_URL_OVERRIDE to:
https://<notebook-name>-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api
but it didn’t work.
Running the opik healthcheck command returns the following:
*** HEALTHCHECK STARTED ***
Python version: 3.10.6
Opik version: 1.7.7
*** CONFIGURATION FILE ***
Config file path: /home/ec2-user/.opik.config
Config file exists: no
*** CURRENT CONFIGURATION ***
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Setting ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ api_key │ None │
│ background_workers │ 4 │
│ check_tls_certificate │ True │
│ console_logging_level │ INFO │
│ default_flush_timeout │ None │
│ enable_litellm_models_monitoring │ True │
│ file_logging_level │ None │
│ logging_file │ opik.log │
│ project_name │ Default Project │
│ pytest_experiment_enabled │ True │
│ sentry_enable │ True │
│ track_disable │ False │
│ url_override │ [https://<notebook-name>-5www.notebook.…] │
│ workspace │ default │
└──────────────────────────────────┴───────────────────────────────────────────┘
*** BACKEND WORKSPACE AVAILABILITY ***
--> Checking backend workspace availability at:
https://<notebook-name>-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api
Backend workspace available: no
Error while checking backend workspace availability: status_code: 302, body:
*** HEALTHCHECK COMPLETED ***
Can you guide me on how to make the Opik server accessible from other SageMaker notebook instances in this setup?
@cmpeburak - while the team gets back to you there might be a much easier path here. Comet (and Opik in it) is available inside SageMaker - https://docs.aws.amazon.com/sagemaker/latest/dg/partner-apps.html Sagemaker will spin up an instance in your SM VPC and it will be fully managed by AWS. You would have to pay a monthly fee to AWS for a license but if it's just for a POC a single license would do ($149)
Thank you so much for the suggestion! @gidim
Using SageMaker’s managed Comet/Opik integration is a great option, but arranging the license purchase and coordinating with our AWS administrators would introduce some delays. To keep our POC moving swiftly, I’ll prefer proceeding with a local Opik setup for now.
@cmpeburak one of the possible reasons can be the redirects which were disabled by default in Opik. We've just made a library release that allows redirects, could you please try it out?
Thank you for looking into this @alexkuzmik.
After installing the latest opik 1.7.8 version, the health check now returns success.
*** HEALTHCHECK STARTED ***
Python version: 3.10.6
Opik version: 1.7.8
*** CONFIGURATION FILE ***
Config file path: /home/ec2-user/.opik.config
Config file exists: no
*** CURRENT CONFIGURATION ***
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Setting ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ api_key │ None │
│ background_workers │ 4 │
│ check_tls_certificate │ False │
│ console_logging_level │ INFO │
│ default_flush_timeout │ None │
│ enable_litellm_models_monitoring │ True │
│ file_logging_level │ None │
│ logging_file │ opik.log │
│ project_name │ Default Project │
│ pytest_experiment_enabled │ True │
│ sentry_enable │ True │
│ track_disable │ False │
│ url_override │ [https://buraksivrikaya-v2-5www.notebook.…](https://buraksivrikaya-v2-5www.notebook.%E2%80%A6/) │
│ workspace │ default │
└──────────────────────────────────┴───────────────────────────────────────────┘
*** BACKEND WORKSPACE AVAILABILITY ***
--> Checking backend workspace availability at:
https://buraksivrikaya-v2-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api
Backend workspace available: yes
*** HEALTHCHECK COMPLETED ***
Then I tried use an API endpoint to create a dataset for testing with the following code.
import os
from opik import Opik
os.environ["OPIK_URL_OVERRIDE"] = f"https://buraksivrikaya-v2-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api"
os.environ["OPIK_CHECK_TLS_CERTIFICATE"] = "false"
client = Opik()
ds = client.get_or_create_dataset(name="test_burak_dataset")
ds.insert(
[
{"input": "What is the capital of France?"},
{"input": "What is the capital of Italy?"},
{"input": "What are the three principles of the Zen of Python?"},
]
)
I see the following error:
Traceback (most recent call last):
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/opik/rest_api/datasets/client.py", line 662, in get_dataset_by_identifier
object_=_response.json(),
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/httpx/_models.py", line 832, in json
return jsonlib.loads(self.content, **kwargs)
File "/Users/burak.sivrikaya/.pyenv/versions/3.10.15/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/Users/burak.sivrikaya/.pyenv/versions/3.10.15/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/burak.sivrikaya/.pyenv/versions/3.10.15/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/burak.sivrikaya/Code/opik-example-codes/test.py", line 12, in <module>
ds = client.get_or_create_dataset(name="test_burak_dataset")
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/opik/api_objects/opik_client.py", line 671, in get_or_create_dataset
return self.get_dataset(name)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/opik/api_objects/opik_client.py", line 564, in get_dataset
self._rest_client.datasets.get_dataset_by_identifier(dataset_name=name)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/tenacity/__init__.py", line 338, in wrapped_f
return copy(f, *args, **kw)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/tenacity/__init__.py", line 477, in __call__
do = self.iter(retry_state=retry_state)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/tenacity/__init__.py", line 378, in iter
result = action(retry_state)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/tenacity/__init__.py", line 400, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
File "/Users/burak.sivrikaya/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/Users/burak.sivrikaya/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/tenacity/__init__.py", line 480, in __call__
result = fn(*args, **kwargs)
File "/Users/burak.sivrikaya/.virtualenvs/opik-example-codes/lib/python3.10/site-packages/opik/rest_api/datasets/client.py", line 667, in get_dataset_by_identifier
raise ApiError(status_code=_response.status_code, body=_response.text)
opik.rest_api.core.api_error.ApiError: status_code: 200, body:
<!DOCTYPE html>
<html lang="en">
<head><style >html { background-color: #ffffff; }</style></head>
<body class="">
<script type="text/javascript">
(function() {
var redirectUrl = function getHashArgsRedirectUrl(location, hashArgsQueryParam, oauthStartQueryParam, oauthFlowStartTime) {
const hashArgs = "#" + (location.href.split("#")[1] || "");
const redirectUrl = new URL(location.href);
redirectUrl.pathname = location.pathname
.replace(/\\/g, "/")
.replace(/\/\/+/g, "/");
redirectUrl.hash = "";
redirectUrl.searchParams.append(hashArgsQueryParam, hashArgs);
if (oauthFlowStartTime && oauthStartQueryParam) {
redirectUrl.searchParams.append(oauthStartQueryParam, oauthFlowStartTime.toString());
}
return redirectUrl;
};
window.location.replace(redirectUrl(window.location, "hashArgs", "oauthStart", 1744928341194).toString());
})();
</script>
<noscript>
<div style="width: 22em; position: absolute; left: 50%; margin-left: -11em; color: red; background-color: white; border: 1px solid red; padding: 1em; font-family: sans-serif">
Your web browser must have JavaScript enabled
in order for this application to display correctly.
</div>
</noscript>
</body>
</html>
I suspect this is what Jupyter’s notebook‑proxy returns when a request is not authenticated. Do you have any clue for the solution of this?
@cmpeburak Where are you running this code snippet from? (account / insatnce/ laptop)
from opik import Opik
os.environ["OPIK_URL_OVERRIDE"] = f"https://buraksivrikaya-v2-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api"
os.environ["OPIK_CHECK_TLS_CERTIFICATE"] = "false"
client = Opik()
ds = client.get_or_create_dataset(name="test_burak_dataset")
ds.insert(
[
{"input": "What is the capital of France?"},
{"input": "What is the capital of Italy?"},
{"input": "What are the three principles of the Zen of Python?"},
]
)
Seems like Jupyter’s notebook‑proxy is trying to redirect you to an oauth flow ... which indicates that the opik SDK calls are not authenticated like AWS is expecting
you can take a look at out sagemaker auth integration which works with aws sagemaker ai apps and try to do somethign similar. but I suspect its not trivial.
https://github.com/comet-ml/opik/blob/main/sdks/python/src/opik/integrations/sagemaker/auth.py
I'm trying to run the code from two environments:
- Another notebook instance under the same account as the one running Opik
- My laptop, using the same AWS credentials as the notebook instance
I'll check the SageMaker auth you shared—thanks for that @Nimrod007
@cmpeburak in case you try to authorize the requests so that they are accepted by the proxy, you can also take a look at our hook for customizing httpx clients - https://github.com/comet-ml/opik/blob/f4f50fe5878f3713fad134fab73c3452d927fd95/sdks/python/src/opik/hooks.py#L7.
For example, if you need to add a manually created header with an auth token for all the requests, you can do it via this hook (idk what exactly is being checked by the sagemaker in the external requests to authorize them)
Thanks, @alexkuzmik ,
I tried a hacky way to mimic how browsers handle authentication for notebook-proxy, and it worked.
I retrieved the cookie from the browser and injected it into the request header:
import opik.hooks
from opik import Opik
import os
os.environ["OPIK_URL_OVERRIDE"] = "https://<your-notebook>-5www.notebook.us-east-1.sagemaker.aws/proxy/5173/api"
os.environ["OPIK_CHECK_TLS_CERTIFICATE"] = "false"
def add_cookie_header(client):
client.headers["Cookie"] = "<cookie>"
opik.hooks.register_httpx_client_hook(add_cookie_header)
client = Opik()
ds = client.get_or_create_dataset("test_dataset")
That said, I’m wondering if there’s a more secure and long-lived authentication flow we can use for programmatic SDK access to notebook-proxy.
I am able to get successful response from the API using a presigned notebook url with a code like below:
import boto3, requests, time
from urllib.parse import urljoin
class NotebookProxySession:
def __init__(self, nb_name):
self.nb_name = nb_name
self.sm = boto3.client("sagemaker")
self._session = requests.Session()
self._expires = 0
def _login(self):
url = self.sm.create_presigned_notebook_instance_url(
NotebookInstanceName=self.nb_name,
SessionExpirationDurationInSeconds=43200
)["AuthorizedUrl"]
self._session.get(url, allow_redirects=True)
self._expires = time.time() + 43100
def request(self, method, path, **kw):
if time.time() >= self._expires:
self._login()
full = urljoin(f"https://{self.nb_name}.notebook.us-east-1.sagemaker.aws", path)
return self._session.request(method, full, **kw)
proxy = NotebookProxySession("mlp-v2")
r = proxy.request("GET", "proxy/5173/api/v1/private/datasets/?workspace_name=default&size=1&page=1")
r.json()
I'm now looking for a way to register this piece as an client hook when making requests using the opik python SDK.
Thank you for reaching out and for providing all the details. Since the original issue appears to be resolved, we’ll go ahead and close this ticket. If you have any further questions or need additional support, please feel free to reopen the issue or open a new one. We’re happy to help!
