How can one use Kubecost to tune requests for Argo Workflows?

[alexec]

Our users often run workflows with large pods, the need lots of CPU/GPU/memory.

I'd like to show users how to optimise these workflows. I'm not an expert on tuning requests, I'm more like a user myself. So as a user I want to know:

• How do I know which workflows are costly?
• How should I tune the workflow?
• How can I monitor cost change over time?

Would anyone be interested in help me out with this please?

[kbrwn]

Hello @alexec for determining which workloads are the most costly you would want to use the kubecost allocations view. You can set a date range and choose how to aggregate (seems like you want "pod") as well as apply filters such as a k8s label. To tune workloads it would be best to use the request sizing feature. To monitor cost over time you should created a Saved Report with the needed aggregations and filters.

You might find our documentation site guide.kubecost.com helpful.

[alexec]

Thank you. I think I probably didn't phrase my request correctly.

Workflows are different to deployments. They run to completion. Pods stop, have generated name, they are created and deleted regularly.

I think it would be neat to have some kind of guide on specifically how to tune this. It would be good for Argo Workflows ecosystem users, as. well as for Kubecost as it would promote the tool. Maybe the guide would be a video or blog post.

Other companies have done similar things, e.g.

https://www.eksworkshop.com/advanced/410_batch/

[kbrwn]

@alexec Thanks for the clarification. We don't really have any specific tooling for identifying cost related to Argo Workflows at this time. As Workflows bring up pods and make use of namespaces you can filter on name and labels of these k8s objects in kubecost reports.

[dwbrown2]

@alexec i'd absolutely love to see Kubecost support this and also provide educational content!

have you seen the integration with spinnaker? https://blog.kubecost.com/blog/spinnaker-kubecost-integration/

i've talked with half a dozen joint users that also expressed interest in something similar to this. let me know if'd want to chat about it or if there is more you can share!

[alexec]

I think a chat would be a great idea. Do you calendly? https://bit.ly/book-30m-with-argo-team

[dwbrown2]

Great, I just grabbed time. Sorry for my delay... I recently had a kid, and have been... busy!

[dwbrown2]

@alexec saw this was meeting was canceled on the argo side. know if that was intentional?

Update: looks like it was scheduled on a holiday! Will rebook :)

[kbrwn]

argo v2.2 now has a more specific annotation to track resources:

  annotations:
    argocd.argoproj.io/tracking-id: my-app:apps/Deployment:default/nginx-deployment

[dwbrown2]

@alexec draft outline available here! https://docs.google.com/document/d/1JmvkyJe0iJ8TFGX3fT5kP_S6jB_CJ6u_yn9H0YH03So/edit

[mattray]

This issue is being closed because it may not be relevant to the OpenCost project and appears stale. If you feel this was closed in error, please open a new OpenCost issue with updated details or if it is still relevant for Kubecost please open an issue with Kubecost Support.