How to Build a PostgreSQL Cloud Foundry Service – Part 2 – Part 3

In the last part of our series about How to build a PostgreSQL Cloud Foundry service we talked about the automation technology (BOSH) we have chosen and the requirements which led to this decision. Now we want to show you details of architecture and the user experience.

Table of Contents

• CF User Experience
  • Create Service
  • Single-Node PostgreSQL Server
  • Update Service
• Architecture
  • The anynines Service Broker
  • The PostgreSQL Service Provider Interface (SPI)
  • The anynines Deployer
• Lessons Learned

CF User Experience

Implementing a Cloud Foundry means to map the desired functionality to the Cloud Foundry service API. Therefore, the PostgreSQL data service should be fully compatible with the Cloud Foundry Service Broker API.

So after gathering requirements and being through the major design decisions the resulting user experience is as follows.

The following use cases in particular require special attention

Create Service

As described in the requirements the user should be able to decide on whether his database service instance will be based on a single virtual machine or on a cluster of several virtual machines.

Single-Node PostgreSQL Server

Creating a single VM PostgreSQL database should be as simple as:

$> cf create-service postgresql single-small my-single-postgres-1

Replicated Three-Node PostgreSQL Cluster

Provisioning a three-node PostgreSQL cluster should be of the sample simplicity:

$> cf create-service postgresql cluster-small my-3node-postgres-cluster-2

Update Service

It should also be possible to scale an existing PostgreSQL database by increasing corresponding virtual machine(s).
This needs to be possible regardless whether it is a single server or a cluster:

$> cf create-service postgresql single-small my-single-postgres-1 $> cf service-update my-3node-postgres-cluster-1 -p cluster-large

Bind Service

Of course – just to mention – service binding need to work as expected – delivering access credentials to the corresponding PostgreSQL database service instances.

Architecture

Now with a clear understanding of what and a direction on how things need to be done an architectural draft can be derived.

Simplified architectural diagram of the anynines PostgreSQL service.

The PostgreSQL service is designed for PostgreSQL specific components to be kept to a minimum resulting into a reusable framework for creating Cloud Foundry data services.

The main architecture consists of a service broker, a deployment component – the deployer as well as PostgreSQL specific service provider interface (SPI).

The anynines Service Broker

It implements the Cloud Foundry Service Broker API and maps this to the desired behaviour.

The service broker offers a set of service plans which will loosely map to deployment templates of the anynines deployer.

The PostgreSQL Service Provider Interface (SPI)

The Service Broker Interface is – comparable to the Cloud Service Interface in Bosh – an abstraction implemented as a pluggable strategy.
Every data service comes with a data service specific SPI. For PostgreSQL, this is the PostgreSQL SPI.

The PostgreSQL SPI has the following responsibilities:

• Delivering metadata used for the service broker metadata catalog including information about service plans
• Issuing of new credentials including admin credentials during the creation of a new service instance
• Creation of new credentials for existing service instances during the creation of a service binding

The PostgreSQL SPI is a JSON RPC web service. It encapsulates all PostgreSQL specific behaviour allowing the anynines service broker to be generic and reusable across all anynines data services.

The anynines Deployer

The primary responsibility of the anynines deployer is to manage BOSH deployments. It does that by managing two major resources: templates and deployments. A template is an abstraction from a deployment manifest and can be pictured as a Bosh manifest with placeholders.

More precisely a anynines deployer template is a BOSH manifest with ERB logic. ERB is a Ruby templating language also used as part of the well known Ruby on Rails framework and allows Ruby code to be used embedded in text files such as YAML files.
Based on a given template the anynines service deployer can be used to create and trigger a BOSH deployment.
This is what happens when a Cloud Foundry user performs a create-service command.

The cloud controller delegates the creation of a new PostgreSQL service instance to the anynines PostgreSQL service broker. The anynines service broker then picks the deployer template corresponding to the service plan chosen by the CF user.
The service broker also makes an API call to the PostgreSQL SPI, whose responsibility is to issue credentials for the PostgreSQL instance to be created. The credentials, among others, include the admin credentials to later access the database.

Together the deployment template and deployment attributes including the previously generated credentials will be parameters to the creation of a new deployment.
This deployment is triggered by the service broker which invokes the REST API of the anynines deployer. The anynines deployer then loads the specified deployment template, evaluates the template by filling in the given deployment attributes, uploads the deployment to BOSH and executes the BOSH deployment. The requested PostgreSQL server or cluster will then be provisioned by BOSH.

The anynines PostgreSQL makes use of asynchronous operations as specified in the  Cloud Foundry Service Broker API. The service broker therefore immediately returns after creating a new service instance while the provisioning of the service instance, mainly determined by the provisioning time of the corresponding BOSH deployment, may take several minutes to complete. Once the deployment is complete, the user can access the newly created dedicated PostgreSQL server or cluster.

Sequence diagram illustrating the information flow of creating a PostgreSQL service instance.

Lessons Learned

Building a production grade PostgreSQL has been a journey and still is.
By far the most important lesson to learn was that without dedicated service instances, the tenant isolation is too weak.

→ Dedicated PostgreSQL servers are mandatory for production use for any public platform

The second most important choice was to support the provisioning of clusters, not only single server instances. Although introducing the majority of the resulting architectural complexity, this has also been the area of most intense learning.

Being able to deploy dedicated data service clusters is a key milestone towards offering a production grade Cloud Foundry. Picking BOSH as the central automation technology has turned out to be an excellent decision. BOSH has proven itself and in our humble opinion is one of the most underestimated technologies on the market.

<< Read part two