a9s Prometheus – Metrics Monitoring Solution for Cloud Foundry & Kubernetes – Beta version

a9s-prometheus-blog-illustration

anynines’ Data Services team extends its a9s Data Service Bundle with the beta release of its metrics monitoring solution a9s Prometheus Service for Cloud Foundry and Kubernetes.

The a9s Prometheus Service for Cloud Foundry and Kubernetes enables the provision of a Prometheus system, including Alertmanager, Grafana, and a number of exporters.

a9s Prometheus for Cloud Foundry and Kubernetes brings the graphite metric mappings and default alerts needed to monitor the a9s Data Services. Those default alerts can be adjusted to the needs of the customer.

How can you get access to the a9s Prometheus service for Cloud Foundry and Kubernetes?

If you already have a license of the a9s Data Service Bundle, the a9s Prometheus service for Cloud Foundry and Kubernetes will be available as soon as you upgrade to version 4.4.0 of anynines deployment.

Additionally, the a9s Prometheus service for Cloud Foundry and Kubernetes will be available soon on our a9s Public PaaS offering. There you will also be able to discover the power of this service.

What is a9s Prometheus for Cloud Foundry and Kubernetes and how can you benefit?

In traditional setups, monitoring was handled by Icinga, which collects state and metric information and forwards the metrics to a graphing solution. However, such a setup was quite complex and needed to be administered manually, and monitored services needed to be added manually to the server configuration.

Regarding modern cloud setups, such as Cloud Foundry, where the number of instances to be monitored changes daily or even hourly, a better scaling solution than Icinga is needed. Prometheus is a good solution at this point, since it is pretty good at collecting metrics in cloud environments.

Since collecting and analyzing metrics from your cloud setup is essential for the success of your cloud environment, we built the a9s Prometheus for Cloud Foundry and Kubernetes. a9s Prometheus is an automated, on-demand provisioning metrics monitoring and visualization solution for Cloud Foundry and Kubernetes. A default a9s Prometheus deployment consists of Prometheus, Alertmanager, Grafana, and Graphite Exporter.

Such an a9s Prometheus System can be deployed with just one click by the users of your platform.

chart-a9s-prometheus-functionality

a9s Prometheus collects the metrics of your data services and alerts you if some metrics are in an invalid state.

How does it work?

a9s Prometheus runs a graphite exporter. This graphite exporter receives metrics from the logstash instances, that are running locally on each of the a9s Data Service instances. The graphite exporter comes with an anynines compatible graphite mapping to translate the graphite metrics to Prometheus metrics.

This graphite exporter is scraped by Prometheus every 60 seconds. Within Prometheus, a set of default alerts for the anynines Data Service metrics are defined. If an alert is triggered, the Alertmanager will be triggered. Grafana is also connected to Prometheus to visualize the metrics in graphs.

I already have a system in place. Can I still use a9s Prometheus for Cloud Foundry and Kubernetes?

Of course! The a9s Prometheus Service for Cloud Foundry and Kubernetes can easily be integrated in an already existing setup with third party components for collecting, alerting, and displaying metrics. You can use a9s Prometheus in combination with the a9s Data Services. You can also use it in combination of third party Data Services or metrics producing components. Or you can also use a mix of both.

a9s Prometheus for Cloud Foundry and Kubernetes is super easy to deploy. Your customers can deploy their own a9s Prometheus instances themselves. Every a9s Prometheus instance remains totally isolated in the customer’s space. This guarantees that no sensitive data accidently become available to other users on the cloud.

Getting started with a9s Prometheus

In this chapter, I will show you how you can start using the a9s Prometheus service. We assume the a9s Prometheus service is already available in Cloud Foundry and an a9s Redis service as well.

Therefore, in the marketplace, you have exactly those two services:

`:~$ cf marketplace
a9s-prometheus     prometheus-platform-logging-ssl, prometheus-single-big, prometheus-single-big-ssl, prometheus-single-medium, prometheus-single-medium-ssl, prometheus-single-small, prometheus-single-small-ssl, prometheus-platform-logging

This is a service creating and managing dedicated Prometheus to monitor applications and service instances, powered by the anynines Service Framework   prometheus-service-broker

a9s-redis50        redis-cluster-big, redis-cluster-medium, redis-cluster-small, redis-single-big, redis-single-medium, redis-single-nano, redis-single-small                                                                                                                                                                                                                                                 This is a service creating and managing dedicated Redis service instances, powered by the anynines Service Framework                                    redis-service-broker

Please ignore the SSL plans at this point. Since it is still in beta, the SSL plans could not work as expected and are only here for testing purposes.

Now, we will create an a9s Prometheus single instance and an a9s Redis cluster instance:


:~$ cf create-service a9s-prometheus prometheus-single-small prom-blogpost
Creating service instance prom-blogpost in org system / space manual as admin...
OK

Create in progress. Use 'cf services' or 'cf service prom-blogpost' to check operation status.

:~$ cf create-service a9s-redis50 redis-cluster-small redis-blogpost
Creating service instance redis-blogpost in org system / space manual as admin...
OK

Create in progress. Use 'cf services' or 'cf service redis-blogpost' to check operation status.

When the service instances are created, create a service key for your a9s Prometheus instance to get the info needed for a9s Redis to stream its metrics:


:~$ cf create-service-key prom-blogpost key1
Creating service key key1 for service instance prom-blogpost as admin...
OK

:~$ cf service-key prom-blogpost key1
Getting key key1 for service instance prom-blogpost as admin...

{
 "alertmanager_urls": [
  "http://msd785be-alertmanager-0.node.dc1.consul.dsf2:9093/"
 ],
 "grafana_urls": [
  "http://msd785be-grafana-0.node.dc1.consul.dsf2:3000/"
 ],
 "graphite_exporter_port": 9109,
 "graphite_exporters": [
  "msd785be-prometheus-0.node.dc1.consul.dsf2"
 ],
 "password": "a9se6ad2a0a5e7c35f9d919136d7764bdbe2f401844",
 "prometheus_urls": [
  "http://msd785be-prometheus-0.node.dc1.consul.dsf2:9090/"
 ],
 "username": "a9s-brk-usr-72e02e451112e327477cf00a59f3ed403a177a28"
}

Note down the graphite exporter and the graphite exporter port and execute an update service for your a9s Redis service instance:


:~$ cf update-service redis-blogpost -c '{ "graphite": [ "msd785be-prometheus-0.node.dc1.consul.dsf2:9109" ] }'

As soon as this update is done, your a9s Redis instance streams its metrics to your a9s Prometheus instance.

To see your metrics, you need to go to the dashboard. Since a9s Prometheus consists of Prometheus itself, Alertmanager, and Grafana, each component has its own dashboard. To access this dashboard, read the information about your a9s Prometheus service instance.


:~$ cf service prom-blogpost
Showing info of service prom-blogpost in org system / space manual as admin...

name:             prom-blogpost
service:          a9s-prometheus-ms-1556535050
tags:
plan:             prometheus-single-small
description:      This is a service creating and managing dedicated Prometheus to monitor applications and service instances, powered by the anynines Service Framework
documentation:    https://www.anynines.com
dashboard:        https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/5c07945e-d6cd-40df-8f50-f2a17844968f
service broker:   prometheus-service-broker-ms-1556535050

Showing status of last operation from service prom-blogpost...

status:    create succeeded
message:
started:   2019-04-30T06:52:50Z
updated:   2019-04-30T07:01:18Z

There are no bound apps for this service.

Copy the URL next to “dashboard” to your browser’s address field. To access the single components, simply add to this URL /grafana to access Grafana, /alertmanager to access Alertmanager, or /promtheus to access a9s Prometheus.
When you log in to Grafana, for example, click on the top line “Home” and you will then see an overview of the existing dashboards:

Select the System Metrics Dashboard from the “anynines” folder. Here, you will see the metrics streamed by your a9s Redis cluster:

Metrics streamed by your a9s Redis

What features are in the pipeline?

For the next release of the a9s Prometheus Service for Cloud Foundry and Kubernetes, we concentrate on SSL integration and adding one single default dashboard.

Leave a Reply

Your email address will not be published. Required fields are marked *