Principles and Strategies of Data Service Automation – Part 1

This article leads through generic strategies that will guide the decision-making process when striving for a full lifecycle automation of on-demand provisioning a large number of data services.

With the rise of applications platforms, the operation of applications has been revolutionized. Application platforms allow the deployment and scaling of applications without any help from sysops. Developers experience a full on-demand self-service. Now, applications don’t exist in a vacuum. Stateful apps have to store their state somewhere. With modern microservice-based applications and the influence of the NoSQL movement, the demand for more data services and many more data service instances overexert existing operational models.

Consequently, the automation revolution that occurred for running applications needs to take place for data services, as well.

Automating data services is harder than automating the lifecycle of stateless applications. Where stateless application operations can be unified, the automation of data services cannot be automated with a single code base. As a matter of principle, the state causes more technical challenges. Known as the CAP Theorem, data services cannot fulfill the desirable requirements consistency, availability, and partition tolerance. Therefore, the implementation of data services will always make a tradeoff between them.

Broadly speaking, different data services solve different problems, provide different interaction models, and – subsequently – behave differently in operations. The resulting heterogeneity in the operational models of data services is what makes a common automation code base of full lifecycle management impossible.

More data service automation frameworks will appear in the next years but they all will address similar challenges. Starting with the most important aspect read more about automation of data services.

Table of Contents

• Automate
• The Mission
• The a9s Data Services Mission Statement
• Conclusion

Automate

The Mission

When automating data services there is a million possibilities. Engineers should have a freedom of choice on how a certain problem is solved. Simultaneously, decisions should follow a certain pattern, point into a defined direction and strive towards a defined goal.

A mission statement provides support for these concerns. Additionally, the mission statement is the single most important source of motivation that helps a team to go through hard times. If a team identifies with the mission, they will overcome any obstacle no matter how impossible it may appear.

At the same time, the mission statement acts like a compass for decision making. Without the need to specify the details of a task, the mission statement helps to fill the blanks.

The a9s Data Services Mission Statement

As this text is influenced by the path of anynines, the mission of a9s Data Services helps to illustrate the importance and structure of a mission statement.

The a9s Data Service mission statement is:

It sets a high bar with demanding a full automation of the entire lifecycle of data services. This clearly separates the wheat from the chaff. It is one thing to install a data service, it’s another thing to guide it through years of operations.

Automating a wide range of data stores is also an important decision. It is a clearly counter-strategy to many data service companies which are highly specialized on one particular data service. It seems the right thing to do, at first.
A closer look reveals that a large cloud-native platform does not need single but dozens of differently data services leading to the risk of a software operation nightmare. With a common operational model across a growing set of data services, their software operations become part of the automation product. This lifecycle automation will change the way data services are operated, sustainably.

The target audience for fully automated data services is growing as cloud-native platform technologies appear and start to compete among another. The automation of data services needs to follow the leading platform technologies as this is where most of the demand will be.

A major aspect of cloud-native platforms is that they tend to abstract from the underlying infrastructure. This is very important as it makes an investment into cloud-native applications sustainable by avoiding a vendor-lock to vendor-specific APIs in favor to open standards. The data service automation should not violate this aspect of cloud-native platforms and therefore be infrastructure agnostic.

Conclusion

Make your mission statement big. Make it matter. It should set a high bar and seem a bit too far-fetched. Then focus all your energy on over achieving the mission goal.

It’s ok not to know everything upfront. Growth of the team will be required to fulfill the mission, and it may no it will take years to achieve it. Everything less wouldn’t be a mission but a project.

Equipped with a meaningful mission, the automation journey may begin.