tfx

tfx copied to clipboard

I'm trying to figure out how to authenticate tfx cli against a vanilla kubeflow to use as an engine, the problem is that the cli fails "silently" in what I think is related to authentication issues.

for example, trying to list pipelines, for simplicity, returns empty, since this is the example kubeflow deployment, and it's protected by Dex OpenID IDP, I'm expecting some challenge of sorts to be able to access the service, but nothing happens

$ tfx pipeline list --endpoint=https://mykubeflowurl.domain --engine=kubeflow
2022-07-12 12:51:10.134013: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
CLI
Listing all pipelines
No pipelines to display.

I've tried locating how the authentication works, but I don't see any reference, even showing an error would be useful.

The question is how do I authenticate this tfx cli request so I can perform other actions, such as creating a pipeline or running one?

Thank you for the question! I agree that we need more improvement on this issue. TFX CLI uses KFP(Kubeflow Pipelines) SDK underneath and all authentication workflow happens in it. Could you take a look at the Kubeflow Pipelines tutorial at https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api/ ?

I think the problem is here https://github.com/tensorflow/tfx/blob/09490d9ebe6bb6097f17bdfa685415e76eaaba5a/tfx/tools/cli/handler/kubeflow_handler.py#L51

if I was using kfp and submitting a pipeline alone, I could get the credentials via curl or requests and then add it to the client

    client = kfp.Client(
        cookies=f"authservice_session={session_cookie}",
        host=f"{HOST}/pipeline",
        namespace=NAMESPACE,
    )

but since the kubeflow_handler instanciates the client as is, tfx cli can't submit the pipeline to a kubeflow behind authentication it seems

Thank you for the investigation! Then we need to add those arguments to TFX CLI flags. Until then, you might need to use kfp CLI instead. https://www.kubeflow.org/docs/components/pipelines/sdk/sdk-overview/#kubeflow-pipelines-cli-tool

You can get the pipeline bundle for kfp from a TFX pipeline using compile command. https://www.tensorflow.org/tfx/guide/cli#compile

@dvaldivia,

You might use kfp CLI instead of TFX CLI as per above comment. Hope this helps. Thank you!