cluster-api-provider-aws icon indicating copy to clipboard operation
cluster-api-provider-aws copied to clipboard

Add Opentelemetry tracing

Open richardcase opened this issue 4 years ago • 16 comments

/kind feature /help

Describe the solution you'd like I'd like to be able to see tracing of the various controllers using opentelemetry. This will enable us to see how many and how long reconcile loops, AWS API calls etc take which will be very helpful for issue investigation.

Environment:

  • Cluster-api-provider-aws version:
  • Kubernetes version: (use kubectl version):
  • OS (e.g. from /etc/os-release):

richardcase avatar Dec 29 '20 20:12 richardcase

@richardcase: This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

/kind feature /help

Describe the solution you'd like I'd like to be able to see tracing of the various controllers using opentelemetry. This will enable us to see how many and how long reconcile loops, AWS API calls etc take which will be very helpful for issue investigation.

Environment:

  • Cluster-api-provider-aws version:
  • Kubernetes version: (use kubectl version):
  • OS (e.g. from /etc/os-release):

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Dec 29 '20 20:12 k8s-ci-robot

We should probably help take this to completion https://github.com/kubernetes-sigs/controller-runtime/pull/1211

randomvariable avatar Jan 05 '21 12:01 randomvariable

I agree @randomvariable .

We could also start to add OpenTelemetry spans like they have done in CAPZ and then when https://github.com/kubernetes-sigs/controller-runtime/pull/1211 is agreed and implemeted we'd need to make sure that we use the existing parent spans from the context when we create our spans.

richardcase avatar Jan 05 '21 12:01 richardcase

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot avatar Jun 09 '21 19:06 fejta-bot

/lifecycle frozen

richardcase avatar Jun 09 '21 20:06 richardcase

/priority backlog

randomvariable avatar Nov 08 '21 19:11 randomvariable

Hi guys, I am Bisakh, wish to improve/add the observability of this module through tracing and metrics-based monitoring with Prometheus as part of GSoC 2022. I am new to this project and want to get myself involved into the core infrastructure. Could you please guide me here by passing on some of your workloads so that I can learn by doing different things? Eagerly waiting for your reply. Thanks : )

bisakhmondal avatar Mar 05 '22 12:03 bisakhmondal

Hi @bisakhmondal, welcome to the community.

You can start with some issues with good-first-issue label to get a grip on the project. Here is the getting started doc: https://cluster-api-aws.sigs.k8s.io/getting-started.html Feel free to reach us at #cluster-api-aws slack channel for questions.

We haven't started working on the design yet. Applications for GSoC will start in April AFAIK, and it is not certain that our project is accepted yet.

sedefsavas avatar Mar 08 '22 04:03 sedefsavas

Hi @sedefsavas, thanks for the detailed reply. I'll start working on the issues then.

We haven't started working on the design yet.

No issues, I can also help there. Feel free to include me in your design meeting.

Applications for GSoC will start in April AFAIK, and it is not certain that our project is accepted yet.

Yes, the first week of April. I have a huge interest in cloud-native stuffs and recently I am working with monitoring tools and solutions so I want to explore more. Just curious, what do you mean by the selection here? I see the project is listed on cncf page (cncf/mentoring repo ig).

bisakhmondal avatar Mar 11 '22 18:03 bisakhmondal

Hi, I am xylonx wanting to sovle the issue by introducing some observation tool like prometheus or jaeger and collecting the metrics info by opentelemetry. Actually, I want to treat it as my GSoC 2022 project. I am new in this community and hope it can encourage my cloud native knowledge and skill. I have used opentelemetry and prometheus / jaeger before, therefore I think I can solve it. Could you give me some practical advice ? Hoping for your reply :)

xylonx avatar Mar 31 '22 07:03 xylonx

@rvacaru @xylonx @bisakhmondal - looks like the deadline for submitting a proposal for GSoC is 19th April. Would it be worth us having a call to go through this issue?

richardcase avatar Apr 06 '22 17:04 richardcase

It would be worth having a look at the demo Bryan did in the CAPI office hours on 16th Sept 2021: https://youtu.be/aThFgrYthOc

richardcase avatar Apr 07 '22 17:04 richardcase

There was an issue on CAPI as well to add this but it was closed as it went stale: https://github.com/kubernetes-sigs/cluster-api/issues/3760.

Adttionally CAPZ added otel support see:

  • https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/311
  • https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/1113

richardcase avatar Apr 07 '22 17:04 richardcase

Hey! @sedefsavas and @richardcase, I'm Aditya, I want to work on the project for GSoC. Is it already assigned to anyone else? Can you please acknowledge me on this project? Can you help me with this problem?

HeyAdityaPatidar avatar Apr 19 '22 17:04 HeyAdityaPatidar

Hi @HeyAdityaPatidar. I believe the deadline to submit a proposal for GSoC has now passed. You could try logging on to the portal to see if it still lets your submit a proposal?

richardcase avatar Apr 21 '22 08:04 richardcase

/remove-lifecycle frozen

richardcase avatar Jul 12 '22 15:07 richardcase

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Oct 23 '22 19:10 k8s-triage-robot

@richardcase I would like to work on this issue. I went through the discussion that you guys had. Could you tell me what needs to be done for this? I would like to assign myself for this issue.

Aut0R3V avatar Nov 13 '22 21:11 Aut0R3V

/assign Aut0R3V

richardcase avatar Nov 14 '22 13:11 richardcase