[Metrics] Prometheus port mismatch

[jacekwachowiak]

Kubernetes version of get _metric_addr https://github.com/ucbrise/clipper/blob/df96eb4924809fedc5a70ef1bc080129c2bf962c/clipper_admin/clipper_admin/kubernetes/kubernetes_container_manager.py#L756-L767 uses the https://github.com/ucbrise/clipper/blob/df96eb4924809fedc5a70ef1bc080129c2bf962c/clipper_admin/clipper_admin/container_manager.py#L9 but the service defined port is https://github.com/ucbrise/clipper/blob/df96eb4924809fedc5a70ef1bc080129c2bf962c/clipper_admin/clipper_admin/kubernetes/prom_service.yaml#L10-L13 so the link will not work. The fix is easy if we decide which way to go.

[rkooo567]

Hi, @jacekwachowiak Thanks for raising an issue. I am a bit busy this week, but I will try to handle issues tomorrow! Thank you for the patience.

[rkooo567]

Good catch. 1390 is a port number for the deployment. Could you create a PR to handle this?

[jacekwachowiak]

Just to confirm - we want to use 1390? I'll change the service yaml then.

[rkooo567]

I think we should use 1390 because other services use same port for deployment and services defined in https://github.com/ucbrise/clipper/blob/develop/clipper_admin/clipper_admin/container_manager.py.

But before that, let's get confirmation from @simon-mo in case.

@simon-mo Could you tell us why prom service uses different port number from 1390? Also can you confirm it is okay to change?

[jacekwachowiak]

Bump @simon-mo @rkooo567

[rkooo567]

@jacekwachowiak Thanks for the reminder. I just DM simon. I will create a PR for this as soon as it is confirmed.

[simon-mo]

Good point. We should use 9090 to stay consistent.

[simon-mo]

For context, CLIPPER_INTERNAL_METRIC_PORT was used originally to expose the query frontend exporter port: https://github.com/ucbrise/clipper/blob/a66648a301c618fed4528e7b103fe737bf803768/monitoring/front_end_exporter.py#L82-L84

So if we change the port there, that would be great.