Getting Started With Concourse CI and Docker – Part 4

Create a pipeline to test your application using your Docker Image as a Resource.

Welcome to the last part of our series about Concourse CI and Docker. Well, we have already seen a small introduction to Concourse CI and Docker in Part I, we learned how to write a Dockerfile and build Docker Images from Dockerfiles in Part II, Part III was about building our Docker Images pipeline in Concourse CI. So let’s get through with it today with creating a pipeline to test a Go application with our Docker Image as a resource and afterwards deploying the app to anynines.

For this part, we will use a modified version of our go example application,one of the requirements to test an application is, that there exists at least one test for the application – that’s the case for our new version.

Writing our application test pipeline

For our final pipeline we will need everything, which means resources, jobs and tasks but let’s start with the things we’ve already seen in Part III. The same way as before we will name our YAML file for the pipeline: pipeline.yml.

We will add a resource for a git repository first, if you are not yet familiar with resources please check out Part III of our series  to get more information about this topic. As I already mentioned before, we add a resource of git type, with name golang_example, the source will be the uri: [https://github.com/benjaminguttmann-avtq/a9s-go-example.git on branch master](https://github.com/benjaminguttmann-avtq/a9s-go-example.git on branch master).

After you are finished with adding the resource, your pipeline.yml should look something like this:

resources:

• name: golang_example  type: git  source:    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git    branch: master

Next thing to add is a job, we’ve done this already so let’s shorten up this part a little bit. If you don’t not completely understand what to do at this point, please check once again Part III for further explanations.
OK, the name of our job will be “anynines_blog_test_app_job“ and the execution of  different steps should be serial. Now we add a plan with the aggregate step. Within this one, we get our golang_example resource and set the trigger to true.

This is what your pipeline.ml should look like at this moment:

jobs:

• name: anynines_blog_test_app_job  serial: true  plan:  - aggregate:    - get: golang_example      trigger: true

resources:

• name: golang_example  type: git  source:    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git    branch: master

Let’s take a closer look at our pipeline.yml so far. In line 3 we have the keyword ‘serial’ set to true that will cause the builds to queue up and execute one-by-one and not in parallel. In line 5 the word ‘aggregate’ appears for the first time, it is used for collecting multiple inputs at once, here we only got one get but you can put more of them in there.
Afterwards, we get our git resource every time it changes which is indicated by the ‘trigger’ set to true. You can find more information about building plans on the official Concourse Page.

Let’s have a look at something new now. We will add a task to our job, configure it, that means set all the parts together, which is needed to successfully test our application.

1. First add the keyword task with a name for the task in our case: golang-unit-tests

– task: golang-unit-tests

3. start the config block

config:

5. the os or platform we’ve chosen is linux

platform: linux

7. we will build our container from our Docker Image at Docker Hub

image: docker:///bguttmannavtq/anynines-blog-build-image

9. start the inputs block

inputs:

11. add some content to our container, or even better let’s add our app to the container

– name: golang_example

13. specify the path of golang_example

path: .

15. start a run block

run:

17. specify to run script

path: ./golang_testsuite

Your pipeline.yml should now look like this:

jobs:

• name: anynines_blog_test_app_job  serial: true  plan:  - aggregate:    - get: golang_example      trigger: true

 - task: golang-unit-tests    config:      platform: linux      image: docker:///bguttmannavtq/anynines-blog-build-image      inputs:      - name: golang_example        path: .      run:        path: ./golang_testsuite

resources:

• name: golang_example  type: git  source:    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git    branch: master

There are still two important things to do. First, write a golang_testsuite script which can be executed by our task and second, add the possibility to deploy our app in our case to anynines.

The script golang_testsuite is in the repository of our app, looks like this:

#!/bin/bash PORT=5000 go run server.go & go test

That’s all, but you have to adjust the permissions to 777, otherwise you will get an permission denied error in Concourse.

With this script we will start our go server listening on port 5000, afterwards we will run the tests.

That all already  looks pretty cool but as it was mentioned in our very first part, we will integrate continuous deployment into our pipeline. That’s coming up next.

To practically follow these steps you need to sign up, create an anynines account and install the Cloud Foundry CLI.

To integrate continuous deployment we need to add another resource to our pipeline.yml; this resource will be of cf type (Cloud Foundry), see the cf-resource Github repository for more information.

1. Add resource with name anynines-deploy-resource

– name: anynines-deploy-resource

3. add type cf

type: cf

5. start source block

source:

7. specifiy the api of the Cloud Foundry provider of your choice ( for us it is anynines ;-) )

api: https://api.de.a9s.eu

9. add credentials to login

username: {{anynines_user}}
password: {{anynines_password}}

11. specify the organization and space you want to deploy to

organization: {{anynines_org}}
space: {{anynines_space}}

13. decide if you want to skip ssl verification

skip_cert_check: false

We will provide our sensitive data in the credentials.yml just like in part III. If you don’t know how to get the information about your cf account I will list some commands to get this data. For more instructions see the Cloud Foundry Documentation of the cli.

Get your Cloud Foundry …

• … api: cf api
• … organizations: cf orgs
• … spaces: cf spaces

This should be your final result of the resources part:

resources:

• name: golang_example  type: git  source:    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git    branch: master

• name: anynines-deploy-resource  type: cf  source:    api: https://api.de.a9s.eu    username: {{anynines_user}}    password: {{anynines_password}}    organization: {{anynines_org}}    space: {{anynines_space}}    skip_cert_check: false

One more job and we are done.

1. Add job with name “job-deploy-app“

– name: job-deploy-app

3. Start plan block

plan:

5. Get our golang_example

– get: golang_example

7. Set a trigger, otherwise the job will not be executed automatically

trigger: true

9. Define to get the result of the test job

passed: [anynines_blog_test_app_job]

11. Put/Deploy it to our cf resource

– put: anynines-deploy-resource

13. Start Params block

params:

15. Path to the manifest.yml

manifest: golang_example/manifest.yml

17. Add environment variable port

environment_variables:
PORT: ‘80’

For more options of the resource check out the github repository of the cf resource.

Please make sure that you quoted the 80 because alternatively, you perhaps will get the error message: error reading request from stdin: json: cannot unmarshal number into Go value of type string.

That’s what our final pipeline.yml should look like:

jobs:

• name: anynines_blog_test_app_job  serial: true  plan:  - aggregate:    - get: golang_example      trigger: true

 - task: golang-unit-tests    config:      platform: linux      image: docker:///bguttmannavtq/anynines-blog-build-image      inputs:      - name: golang_example        path: .      run:        path: ./golang_testsuite

• name: job-deploy-app  plan:    - get: golang_example      trigger: true      passed: [anynines_blog_test_app_job]    - put: anynines-deploy-resource      params:        manifest: golang_example/manifest.yml        environment_variables:          PORT: ‘80’

resources:

• name: golang_example  type: git  source:    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git    branch: master

• name: anynines-deploy-resource  type: cf  source:    api: https://api.de.a9s.eu    username: {{anynines_user}}    password: {{anynines_password}}    organization: {{anynines_org}}    space: {{anynines_space}}    skip_cert_check: false

That’s the moment we worked for four parts long, well then, finish it all up and get the pipeline running.

fly set-pipeline -p anynines-blog-test-and-deploy -c pipeline.yml -l credentials.yml

Refresh the concourse main page and unpause the pipeline by clicking the play button or with the command line tool and the command:

fly unpause-pipeline -p anynines-blog-test-and-deploy

Well, that would be all of it. At the very end of this series, let’s quickly think about what we have seen and learned. We got a brief introduction to Docker and Concourse CI, we  reviewed how to write Dockerfiles, we set up a pipeline for our Docker Images and finally we set up a pipeline to test an application and deploy it to anynines.

Thank you for spending your time reading this series, I hope you enjoyed it.

<< Read Getting started with Concourse CI and Docker Part 3