serving icon indicating copy to clipboard operation
serving copied to clipboard

fix cold start time increase issue

Open kinderyj opened this issue 1 year ago • 10 comments

Fixes #15477

Proposed Changes

  • For Kubernetes clusters without mesh enabled, it's preferable to probe the pod directly. However, if podsAddressable is set to false, the activator will continuously probe the cluster, leading to significantly longer cold start times. To re-enable pod probing, you must restart the activator pod. This is because the podsAddressable setting resets to its default value of true only during the activator's startup and initialization.

Release Note

Fix cold start time increase issue.

kinderyj avatar Aug 22 '24 02:08 kinderyj

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: kinderyj / name: Matt Wang (bd3567511271775f80bd1a6bab8c05a6f32567c6)

Welcome @kinderyj! It looks like this is your first PR to knative/serving 🎉

knative-prow[bot] avatar Aug 22 '24 02:08 knative-prow[bot]

Hi @kinderyj. Thanks for your PR.

I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

knative-prow[bot] avatar Aug 22 '24 02:08 knative-prow[bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kinderyj Once this PR has been reviewed and has the lgtm label, please assign davidhadas for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

knative-prow[bot] avatar Aug 22 '24 02:08 knative-prow[bot]

/ok-to-test

dprotaso avatar Dec 30 '24 23:12 dprotaso

@kinderyj do you have repro steps on triggering the podAddressable flag change?

There's also another instance where podAddressable is switched to false - https://github.com/knative/serving/blob/df030277a7be5bfa869e1b6e1f60e29502393a3a/pkg/autoscaler/metrics/stats_scraper.go#L238

So I don't know if your PR covers all cases here

dprotaso avatar Dec 30 '24 23:12 dprotaso

@kinderyj: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
unit-tests_serving_main bd3567511271775f80bd1a6bab8c05a6f32567c6 link true /test unit-tests

Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

knative-prow[bot] avatar Dec 30 '24 23:12 knative-prow[bot]

Change breaks some unit tests

dprotaso avatar Dec 31 '24 02:12 dprotaso

@kinderyj any interesting in following up on this PR?

dprotaso avatar Jan 14 '25 02:01 dprotaso

This Pull Request is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen with /reopen. Mark as fresh by adding the comment /remove-lifecycle stale.

github-actions[bot] avatar Apr 15 '25 01:04 github-actions[bot]

This Pull Request is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen with /reopen. Mark as fresh by adding the comment /remove-lifecycle stale.

github-actions[bot] avatar Jul 18 '25 01:07 github-actions[bot]