Starting from Kubernetes 1.8, resource usage metrics, such as container CPU and memory usage, are available in Kubernetes through the Metrics API. With Metrics Server installed in Kubernetes cluster, you can access the API to get the metrics. Metrics API doesn't store metric values, you need to use tools like Prometheus to store metrics data. This post uses Prometheus Operator to install and configure Prometheus. Please refer to Monitor Spring Boot Apps using Prometheus on Kubernetes for installation and configuration of Prometheus Operator.

Metrics exported in Kubernetes Metrics API cannot be consumed directly by Prometheus. metrics-server-exporter is used to export those metrics. You can run the exporter in a container with image deadc/metrics-server-exporter. The GitHub repo already provides YAML files to deploy the exporter to Kubernetes. You can use these files to deploy the exporter. When deploying service-metrics-server-exporter.yaml file, you need to modify this file to add a name api to the port.

Now we need to add ServiceMonitor in Prometheus Operator to collect the metrics from exporter.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: pod-status
  labels:
    release: prometheus-operator
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server-exporter
  endpoints:
  - port: api
    path: '/metrics'
    interval: 10s
    honorLabels: true

Now you can see the collected metrics kube_metrics_server_pods_mem and kube_metrics_server_pods_cpu in Prometheus.