apm-server
apm-server copied to clipboard
elastic
Different processors and namespaces to different data_streams for apm-server
i have a test simple configuration.
---
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
name: apm-apm-sample
namespace: elastic
spec:
data_stream:
namespace: test2
config:
apm-server.data_streams.namespace: testname
count: 1
elasticsearchRef:
name: es-apm-sample
http:
service:
metadata: {}
spec: {}
tls:
certificate: {}
kibanaRef:
name: kb-apm-sample
podTemplate:
metadata:
creationTimestamp: null
spec:
containers: null
version: 8.0.1
---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: es-apm-sample
namespace: elastic
spec:
auth: {}
http:
service:
metadata: {}
spec: {}
tls:
certificate: {}
monitoring:
logs: {}
metrics: {}
nodeSets:
- config:
node.store.allow_mmap: false
count: 1
name: default
podTemplate:
metadata:
creationTimestamp: null
spec:
containers: null
transport:
service:
metadata: {}
spec: {}
tls:
certificate: {}
updateStrategy:
changeBudget: {}
version: 8.0.1
---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: kb-apm-sample
namespace: elastic
spec:
config:
xpack.fleet.packages:
- name: apm
version: latest
count: 1
elasticsearchRef:
name: es-apm-sample
http:
service:
metadata: {}
spec: {}
tls:
certificate: {}
monitoring:
logs: {}
metrics: {}
podTemplate:
metadata:
creationTimestamp: null
spec:
containers: null
version: 8.0.1
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: petclinic
namespace: elastic
labels:
app.kubernetes.io/component: test-application
app.kubernetes.io/name: petclinic
app.kubernetes.io/part-of: elastic-stack
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/component: test-application
app.kubernetes.io/name: petclinic
app.kubernetes.io/part-of: elastic-stack
template:
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/component: test-application
app.kubernetes.io/name: petclinic
app.kubernetes.io/part-of: elastic-stack
spec:
volumes:
- name: elastic-apm-agent
emptyDir: {}
initContainers:
- name: elastic-java-agent
image: 'docker.elastic.co/observability/apm-agent-java:1.29.0'
command:
- cp
- '-v'
- /usr/agent/elastic-apm-agent.jar
- /elastic/apm/agent
resources: {}
volumeMounts:
- name: elastic-apm-agent
mountPath: /elastic/apm/agent
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
containers:
- resources: {}
terminationMessagePath: /dev/termination-log
name: petclinic
env:
- name: ELASTIC_APM_SERVER_URL
value: 'https://apm-apm-sample-apm-http:8200'
- name: ELASTIC_APM_SERVICE_NAME
value: petclinic
- name: ELASTIC_APM_APPLICATION_PACKAGES
value: org.springframework.samples.petclinic
- name: ELASTIC_APM_ENVIRONMENT
value: dev
- name: JAVA_TOOL_OPTIONS
value: '-javaagent:/elastic/apm/agent/elastic-apm-agent.jar'
- name: ELASTIC_APM_GLOBAL_LABELS
value: namespace=test333
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: ELASTIC_APM_VERIFY_SERVER_CERT
value: 'false'
ports:
- name: petclinic-http
containerPort: 8080
protocol: TCP
imagePullPolicy: IfNotPresent
volumeMounts:
- name: elastic-apm-agent
mountPath: /elastic/apm/agent
terminationMessagePolicy: File
image: 'arey/springboot-petclinic:latest'
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
revisionHistoryLimit: 10
progressDeadlineSeconds: 600
traces and metrics are written. but there are a few questions (maybe I didn't find it in the documentation)
- why don't all transactions have kubernetes.namespace?
- how to change data_streams.namespace by default?
- how to make the data written to different datastreams according to the template < type>-<service.name>-<kubernetes.namespace> (traces-apm.app.petclinic-elastic, metrics-apm.app.petclinic-elastic) ? in apm-server 7.15 it was, but in 8.0.0 it was deprecated
output.elasticsearch.index: "apm-%{[observer.version]}-%{[processor.event]}-%{[kubernetes.namespace]}-%{+yyyy.MM.dd}"
ps:
- OpenShift version 4.9.10
- Elasticsearch (ECK) Operator 2.0.0 provided by Elastic
First, the spec.data_stream field in your manifest doesn't exist (API doc).
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
name: apm-apm-sample
namespace: elastic
spec:
data_stream:
namespace: test2
Your manifest should be rejected:
> kubectl apply -f eck-5441-apm.yml
error: error validating " eck-5441-apm.yml ": error validating data: ValidationError(ApmServer.spec):
unknown field "data_stream" in co.elastic.k8s.apm.v1.ApmServer.spec;
if you choose to ignore these errors, turn validation off with --validate=false
Second, the standalone APM Server is now considered as legacy and will be soon deprecated because the Elastic APM integration became generally available starting 7.16. This method of installing APM Server will be removed in a future release. Please consider getting started with the Elastic APM integration instead.
Third, the following questions are not ECK related but directly related to using APM, so I will transfer the issue to get help from the APM team who will be in a better position to answer it.
why don't all transactions have kubernetes.namespace?
how to change data_streams.namespace by default?
how to make the data written to different datastreams according to the template < type>-<service.name>-<kubernetes.namespace> (traces-apm.app.petclinic-elastic, metrics-apm.app.petclinic-elastic) ? in apm-server 7.15 it was, but in 8.0.0 it was deprecated
spec.data_stream.namespace: test2 - Yes. it was just a test and didn't delete it.
data_stream.namespace by default can be changed through integrations, but I have an elastic-agent with an apm-server in one namespace, and java-agents will write there from other namespaces. through the fleet and the elastic-agent it is also not possible to change the data_stream.namespace for apm and java-agent. data_stream.namespace
---
kind: Namespace
apiVersion: v1
metadata:
name: elastic
---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: kibana
spec:
version: 8.0.1
count: 1
elasticsearchRef:
name: elasticsearch
config:
xpack.fleet.agents.elasticsearch.hosts: ["https://elasticsearch-es-http.elastic.svc:9200"]
xpack.fleet.agents.fleet_server.hosts: ["https://fleet-server-agent-http.elastic.svc:8220"]
xpack.fleet.packages:
- name: apm
version: latest
xpack.fleet.agentPolicies:
- name: Default Fleet Server on ECK policy
is_default_fleet_server: true
package_policies:
- package:
name: fleet_server
name: fleet_server-1
- name: Default Elastic Agent on ECK policy
is_default: true
unenroll_timeout: 900
package_policies:
- package:
name: apm
name: apm-1
inputs:
- type: apm
enabled: true
vars:
- name: host
value: 0.0.0.0:8200
---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: elasticsearch
spec:
version: 8.0.1
nodeSets:
- name: elastic
count: 1
config:
node.store.allow_mmap: false
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
name: fleet-server
spec:
image: docker.elastic.co/beats/elastic-agent:8.0.1
version: 8.0.1
kibanaRef:
name: kibana
elasticsearchRefs:
- name: elasticsearch
mode: fleet
fleetServerEnabled: true
deployment:
replicas: 1
podTemplate:
spec:
serviceAccountName: fleet-server
automountServiceAccountToken: true
securityContext:
runAsUser: 0
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
name: elastic-agent
spec:
image: docker.elastic.co/beats/elastic-agent:8.0.1
version: 8.0.1
kibanaRef:
name: kibana
fleetServerRef:
name: fleet-server
mode: fleet
deployment:
replicas: 1
podTemplate:
spec:
serviceAccountName: fleet-server
securityContext:
runAsUser: 0
---
apiVersion: v1
kind: Service
metadata:
name: apm
spec:
selector:
agent.k8s.elastic.co/name: elastic-agent
ports:
- protocol: TCP
port: 8200
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fleet-server
rules:
- apiGroups: [""]
resources:
- pods
- namespaces
- nodes
verbs:
- get
- watch
- list
- apiGroups: ["coordination.k8s.io"]
resources:
- leases
verbs:
- get
- create
- update
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: fleet-server
namespace: elastic
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: fleet-server
subjects:
- kind: ServiceAccount
name: fleet-server
namespace: elastic
roleRef:
kind: ClusterRole
name: fleet-server
apiGroup: rbac.authorization.k8s.io
@b2ronn . Hi, were you able to solve the issue? I am also not able to change the namespace of APM integration through UI, it is always creating index and data stream in default space. I am using eck 8.2.3 . Have created a issue https://github.com/elastic/fleet-server/issues/1628 for the same. Any help much appreciated.
@SanjuTechie87 perhaps unrelated, but we recently fixed a bug in 8.3.0 where changing the APM Server data stream didn't work: https://www.elastic.co/guide/en/apm/guide/current/release-notes-8.3.html#_bug_fixes
