Monitor Spring Boot Apps using Prometheus on Kubernetes
This posts show how to monitor Spring Boot apps using Prometheus when running on Kubernetes. You should already have a running Kubernetes cluster.
I'm going to use Prometheus Operator to create and manage Prometheus on Kubernetes. Prometheus Operator is installed using Helm, so Helm needs to be installed first. On macOS, simply use Homebrew. This post uses Helm
$ brew install kubernetes-helm
Then you need to initialize Helm.
$ helm init
Install Prometheus Operator
Before installing Prometheus Operator, you need to make sure the current account has the permission to manage
CustomResourceDefinitions. A simple way is to assign the role
cluster-admin to current user, see here. Otherwise, you'll see error like below.
Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" is forbidden: User "system:serviceaccount:kube-system:default" cannot delete resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
If you are using the default system service account, you can use the following command to create the role binding. Replace
system:serviceaccount:kube-system:default with the name of your own account.
$ kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=system:serviceaccount:kube-system:default
Now you can install Prometheus Operator using Helm.
$ helm install stable/prometheus-operator --name prometheus-operator
After installation, you can use the following command to access the Prometheus web UI on http://localhost:9090.
prometheus-operated is the service to access Prometheus API and web UI.
$ kubectl port-forward svc/prometheus-operated 9090:9090
By default, Prometheus Operator monitors various targets of the Kubernetes cluster. To add your own targets, you need to create
You need to have a
Service for your Spring Boot application first.
The yaml file below creates a new
ServiceMonitor that matches the label
app with name
my-service. To expose metrics of the Spring Boot app, you need to add Spring Boot Actuator and have the
prometheus endpoint enabled. The path of the metrics endpoint is
/actuator/prometheus. The port
api is the port defined in the service. Use
kubectl create to create this
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: my-service labels: release: prometheus-operator spec: selector: matchLabels: app: my-service endpoints: - port: api path: '/actuator/prometheus' interval: 10s honorLabels: true
Please note, the label
release: prometheus-operator is important for the
ServiceMonitor to work. If you run
kubectl get servicemonitor prometheus-operator -o yaml, you can see the following output. This means Prometheus Operator only selects
ServiceMonitors with the label
serviceMonitorSelector: matchLabels: release: prometheus-operator
Now you should see the target for the Spring Boot app in the list of http://localhost:9090/targets.