Getting started with Concourse CI and Docker Part 4

a9s_concourseCI_dockerCreate a pipeline to test your application using your Docker Image as a Resource.

Welcome to the last part of our series about Concourse CI and Docker. Well, we have already seen a small introduction to Concourse CI and Docker in Part I, we learned how to write a Dockerfile and build Docker Images from Dockerfiles in Part II, Part III was about building our Docker Images pipeline in Concourse CI. So let’s get through with it today with creating a pipeline to test a Go application with our Docker Image as a resource and afterwards deploying the app to anynines.

For this part, we will use a modified version of our go example application,one of the requirements to test an application is, that there exists at least one test for the application – that’s the case for our new version.

Writing our application test pipeline

For our final pipeline we will need everything, which means resources, jobs and tasks but let’s start with the things we’ve already seen in Part III. The same way as before we will name our YAML file for the pipeline: pipeline.yml.

We will add a resource for a git repository first, if you are not yet familiar with resources please check out Part III of our series  to get more information about this topic. As I already mentioned before, we add a resource of git type, with name golang_example, the source will be the uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git on branch master.

After you are finished with adding the resource, your pipeline.yml should look something like this:

resources:
- name: golang_example
  type: git
  source:
    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git
    branch: master

Next thing to add is a job, we’ve done this already so let’s shorten up this part a little bit. If you don’t not completely understand what to do at this point, please check once again Part III for further explanations.
OK, the name of our job will be “anynines_blog_test_app_job“ and the execution of  different steps should be serial. Now we add a plan with the aggregate step. Within this one, we get our golang_example resource and set the trigger to true.

This is what your pipeline.ml should look like at this moment:

jobs:
- name: anynines_blog_test_app_job
  serial: true
  plan:
  - aggregate:
    - get: golang_example
      trigger: true

resources:
- name: golang_example
  type: git
  source:
    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git
    branch: master

Let’s take a closer look at our pipeline.yml so far. In line 3 we have the keyword ‘serial’ set to true that will cause the builds to queue up and execute one-by-one and not in parallel. In line 5 the word ‘aggregate’ appears for the first time, it is used for collecting multiple inputs at once, here we only got one get but you can put more of them in there.
Afterwards, we get our git resource every time it changes which is indicated by the ‘trigger’ set to true. You can find more information about building plans on the official Concourse Page.

Let’s have a look at something new now. We will add a task to our job, configure it, that means set all the parts together, which is needed to successfully test our application.

  1. First add the keyword task with a name for the task in our case: golang-unit-tests
  2. – task: golang-unit-tests

  3. start the config block
  4. config:

  5. the os or platform we’ve chosen is linux
  6. platform: linux

  7. we will build our container from our Docker Image at Docker Hub
  8. image: docker:///bguttmannavtq/anynines-blog-build-image

  9. start the inputs block
  10. inputs:

  11. add some content to our container, or even better let’s add our app to the container
  12. – name: golang_example

  13. specify the path of golang_example
  14. path: .

  15. start a run block
  16. run:

  17. specify to run script
  18. path: ./golang_testsuite

Your pipeline.yml should now look like this:

jobs:
- name: anynines_blog_test_app_job
  serial: true
  plan:
  - aggregate:
    - get: golang_example
      trigger: true

  - task: golang-unit-tests
    config:
      platform: linux
      image: docker:///bguttmannavtq/anynines-blog-build-image
      inputs:
      - name: golang_example
        path: .
      run:
        path: ./golang_testsuite

resources:
- name: golang_example
  type: git
  source:
    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git
    branch: master

There are still two important things to do. First, write a golang_testsuite script which can be executed by our task and second, add the possibility to deploy our app in our case to anynines.

The script golang_testsuite is in the repository of our app, looks like this:

#!/bin/bash
PORT=5000 go run server.go &
go test

That’s all, but you have to adjust the permissions to 777, otherwise you will get an permission denied error in Concourse.

With this script we will start our go server listening on port 5000, afterwards we will run the tests.

That all already  looks pretty cool but as it was mentioned in our very first part, we will integrate continuous deployment into our pipeline. That’s coming up next.

To practically follow these steps you need to sign up, create an anynines account and install the Cloud Foundry CLI.

To integrate continuous deployment we need to add another resource to our pipeline.yml; this resource will be of cf type (Cloud Foundry), see the cf-resource Github repository for more information.

  1. Add resource with name anynines-deploy-resource
  2. – name: anynines-deploy-resource

  3. add type cf
  4. type: cf

  5. start source block
  6. source:

  7. specifiy the api of the Cloud Foundry provider of your choice ( for us it is anynines ;-) )
  8. api: https://api.de.a9s.eu

  9. add credentials to login
  10. username: {{anynines_user}}
    password: {{anynines_password}}

  11. specify the organization and space you want to deploy to
  12. organization: {{anynines_org}}
    space: {{anynines_space}}

  13. decide if you want to skip ssl verification
  14. skip_cert_check: false

We will provide our sensitive data in the credentials.yml just like in part III. If you don’t know how to get the information about your cf account I will list some commands to get this data. For more instructions see the Cloud Foundry Documentation of the cli.

Get your Cloud Foundry …

  • … api: cf api
  • … organizations: cf orgs
  • … spaces: cf spaces

This should be your final result of the resources part:

resources:
- name: golang_example
  type: git
  source:
    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git
    branch: master

- name: anynines-deploy-resource
  type: cf
  source:
    api: https://api.de.a9s.eu
    username: {{anynines_user}}
    password: {{anynines_password}}
    organization: {{anynines_org}}
    space: {{anynines_space}}
    skip_cert_check: false

One more job and we are done.

  1. Add job with name “job-deploy-app“
  2. – name: job-deploy-app

  3. Start plan block
  4. plan:

  5. Get our golang_example
  6. – get: golang_example

  7. Set a trigger, otherwise the job will not be executed automatically
  8. trigger: true

  9. Define to get the result of the test job
  10. passed: [anynines_blog_test_app_job]

  11. Put/Deploy it to our cf resource
  12. – put: anynines-deploy-resource

  13. Start Params block
  14. params:

  15. Path to the manifest.yml
  16. manifest: golang_example/manifest.yml

  17. Add environment variable port
  18. environment_variables:
    PORT: ‘80’

For more options of the resource check out the github repository of the cf resource.

Please make sure that you quoted the 80 because alternatively, you perhaps will get the error message: error reading request from stdin: json: cannot unmarshal number into Go value of type string.

That’s what our final pipeline.yml should look like:

jobs:
- name: anynines_blog_test_app_job
  serial: true
  plan:
  - aggregate:
    - get: golang_example
      trigger: true

  - task: golang-unit-tests
    config:
      platform: linux
      image: docker:///bguttmannavtq/anynines-blog-build-image
      inputs:
      - name: golang_example
        path: .
      run:
        path: ./golang_testsuite

- name: job-deploy-app
  plan:
    - get: golang_example
      trigger: true
      passed: [anynines_blog_test_app_job]
    - put: anynines-deploy-resource
      params:
        manifest: golang_example/manifest.yml
        environment_variables:
          PORT: ‘80’

resources:
- name: golang_example
  type: git
  source:
    uri: https://github.com/benjaminguttmann-avtq/a9s-go-example.git
    branch: master

- name: anynines-deploy-resource
  type: cf
  source:
    api: https://api.de.a9s.eu
    username: {{anynines_user}}
    password: {{anynines_password}}
    organization: {{anynines_org}}
    space: {{anynines_space}}
    skip_cert_check: false

That’s the moment we worked for four parts long, well then, finish it all up and get the pipeline running.

fly set-pipeline -p anynines-blog-test-and-deploy -c pipeline.yml -l credentials.yml

Refresh the concourse main page and unpause the pipeline by clicking the play button or with the command line tool and the command:

anynines_concourse_app_test_pipeline
fly unpause-pipeline -p anynines-blog-test-and-deploy
anynines_concourse_pipeline_success

Well, that would be all of it. At the very end of this series, let’s quickly think about what we have seen and learned. We got a brief introduction to Docker and Concourse CI, we  reviewed how to write Dockerfiles, we set up a pipeline for our Docker Images and finally we set up a pipeline to test an application and deploy it to anynines.

Thank you for spending your time reading this series, I hope you enjoyed it.

<< Read Getting started with Concourse CI and Docker Part 3

3 thoughts on “Getting started with Concourse CI and Docker Part 4

  1. Pingback: Getting started with Concourse CI and Docker Part 3 | a9s Blog

  2. A google search for “error reading request from stdin: json: cannot unmarshal number into Go value of type string” brought me to your blog to fix my port 80 problem. Thanks!

  3. Great it was helpful , there are not many docs which I could refer for concourse ,this helped .Thank you . If you could suggest few other links to read through ?

Leave a Reply

Your email address will not be published. Required fields are marked *