Enable monitor CF Isolation Segment on CF/Apps dashboard

[daichi703n]

Update Grafana dashboards Cloudfoundry/Apps: System, Apps: Requests, Apps: Latency for using CF Isolation Segment.

Please review, merge and release new version.

Example Changes

Before:

avg(firehose_container_metric_cpu_percentage{environment=~"$environment",bosh_deployment=~"$bosh_deployment",application_id=~"$cf_application_id"})

After:

avg(firehose_container_metric_cpu_percentage{environment=~"$environment", application_id=~"$cf_application_id"})

Why Change

Metrics for variable templating cf_application_info's deployment is different from firehose_container_metric_cpu_percentage's bosh_deployment on CF Isolation Segment. (Similarly other metrics)

If CF Isolation Segment using, monitoring metrics firehose_container_metric_cpu_percentage has different bosh_deployment on CF basic's.

Since application_id is unique key, bosh_deployment filter key is unneeded.

Example Metrics

• CF basic (deployment="cf",bosh_deployment="cf")

cf_application_info{application_id="36852824-xxxx-xxxx-xxxx-xxxxxxxxxxxx",application_name="app",buildpack="java_buildpack_offline",deployment="cf",environment="p-bosh",instance="localhost:9193",job="cf_exporter",organization_id="f2c75a5b-xxxx-xxxx-xxxx-xxxxxxxxxxxx",organization_name="org",space_id="f3be9cb8-xxxx-xxxx-xxxx-xxxxxxxxxxxx",space_name="staging",stack_id="e1d11f77-xxxx-xxxx-xxxx-xxxxxxxxxxxx",state="STARTED"}
firehose_container_metric_cpu_percentage{application_id="36852824-xxxx-xxxx-xxxx-xxxxxxxxxxxx",bosh_deployment="cf",bosh_job_id="38b0b513-xxxx-xxxx-xxxx-xxxxxxxxxxxx",bosh_job_ip="172.26.x.x",bosh_job_name="diego_cell",environment="p-bosh",instance="localhost:9186",instance_index="0",job="firehose_exporter",origin="rep"}	

• CF Isolation Segment (deployment="cf",bosh_deployment="p-isolation-segment-xxx)(in case of Pivotal-CF)

cf_application_info{application_id="4bfa9c7c-xxxx-xxxx-xxxx-xxxxxxxxxxxx",application_name="app",buildpack="java_buildpack_offline",deployment="cf",environment="p-bosh",instance="localhost:9193",job="cf_exporter",organization_id="f2c75a5b-xxxx-xxxx-xxxx-xxxxxxxxxxxx",organization_name="org",space_id="cfee18fa-xxxx-xxxx-xxxx-xxxxxxxxxxxx",space_name="develop",stack_id="e1d11f77-xxxx-xxxx-xxxx-xxxxxxxxxxxx",state="STARTED"}	
firehose_container_metric_cpu_percentage{application_id="4bfa9c7c-xxxx-xxxx-xxxx-xxxxxxxxxxxx",bosh_deployment="p-isolation-segment-5af2xxxxxxxxxxxxxxxx",bosh_job_id="7a06a28c-xxxx-xxxx-xxxx-xxxxxxxxxxxx",bosh_job_ip="172.26.x.x",bosh_job_name="isolated_diego_cell",environment="p-bosh",instance="localhost:9186",instance_index="0",job="firehose_exporter",origin="rep"}

[frodenas]

Removing bosh_deployment from the expression query will affect other people that want to filter by that field. If what you want is to enable seeing metrics for all deployments, you can enable includeAllat the grafana templating filters.

[daichi703n]

Sorry for lack of consideration. Problem I point is Isolation segment's bosh_deployment name is not listed on Deployment Variables, I'll re-consider how to get variables with no editing metrics graph.

[frodenas]

@daichi703n No problem. Just do a quick test, go to the grafana dashboard, edit it, and at the templates section enable All values for the bosh_deployment property. That should allow you to not filter by the property.

[daichi703n]

@frodenas Sorry for late response.

I concluded that this case can not be expected.

cf_application_info is used for templating, firehose_xxx (firehose_container_metric_cpu_percentage, etc,) is used for collectiong metrics (graphs).

cf_application_info does not have bosh_deployment key and only have deployment key (cf deployment name), therefore it is impossible to list "p-isolation-segment-5af2xxxxxxxxxxxxxxxx" on Deployment.

Summary of my environment's params (using PCF Isolation Segment).

• CF (Pivotal Application Service) cf_application_info: deployment="cf" firehose_xxx: bosh_deployment="cf"

• CF (PCF Isolation Segment) cf_application_info: deployment="cf" firehose_xxx: bosh_deployment="p-isolation-segment-5af2xxxxxxxxxxxxxxxx"

Becaurse application_id in firehose_container_metric_cpu_percentage (and the other) is UUID, bosh_deployment is unnecessary for collectiong metrics query. (only used for templating)
(With this idea, environment in firehose_xxx is also unnecessary, but I didn't delete because I didn't know the impact...)

firehose_container_metric_cpu_percentage{
  environment=~\"$environment\",
  bosh_deployment=~\"$bosh_deployment\",
  application_id=~\"$cf_application_id\"  <- UUID
}

How do you think?

If it still has a risk of affect other people, I'll close this PR with no merge.

Regards

[daichi703n]

Hi, @frodenas

On PCF/PAS2.5, Breaking changes: Removed Option to Override Deployment Name to CF will break Apps/xxx dashboards. Because of cf-deployment name will be changed to cf-GUID from cf. (Sorry, not sure about OSS CF) https://docs.pivotal.io/pivotalcf/2-5/pcf-release-notes/breaking-changes.html#deployment-name

This changes will fix the issue. Please re-consider to merge.

Regards.

[Coinsintegrity]

Ok👍

[github-actions[bot]]

This PR is stale because it has been open 60 days with no activity. Comment or this will be closed in 5 days.

[github-actions[bot]]

This PR was automatically closed because it has been stalled for 5 days with no activity.