Canary Deployments and Route-Based Load Balancing in Cloud Foundry

,

·

cloud-foundry-canary-strategy-load-balancing

Cloud Foundry recently introduced two new features aimed at improving how applications are deployed and traffic is managed.

These features are:

  • CF push strategy “Canary”
  • Route-based load balancing algorithm 

Let’s walk through each one, starting with the new cf push strategy called “canary”. It offers a safer way to roll out app changes gradually. Until now, the only strategy available was “rolling”, which enables zero-downtime deployments by replacing instances one at a time. With canary, you can deploy to a single instance first, inspect its behavior, and then decide to proceed with the full rollout.

1. Canary deployment with cf push

Canary adds new possibilities for you now by simply configuring it via the CF CLI. Here’s how to use it:

cf push --strategy canary

Let’s look at a simple example using a Go web app:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/gorilla/mux"
)

const INDEX = `<!DOCTYPE html>
<html>
  <body style="background-color:blue;">
    <h1>Powered by anynines</h1>
  </body>
</html>`

func main() {
	router := mux.NewRouter()
	router.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(w, INDEX)
	})

	log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), router))
}

After we push the app successfully, we can check the result via curl:

Waiting for app cf-push-strategy-canary to start...

Instances starting...
Instances starting...
Instances starting...
Instances starting...

name:              cf-push-strategy-canary
requested state:   started
routes:            cf-push-strategy-canary.apps.anynines-testing.env
last uploaded:     Fri 01 Aug 10:49:59 CEST 2025
stack:             cflinuxfs4
buildpacks:
	name           version   detect output   buildpack name
	go_buildpack   1.10.36   go              go

type:            web
sidecars:
instances:       1/1
memory usage:    256M
start command:   ./bin/mod
     state     since                  cpu    memory     disk       logging        cpu entitlement   details
#0   running   2025-08-01T08:50:17Z   0.0%   0B of 0B   0B of 0B   0B/s of 0B/s   0.0%
 curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:blue;">
    <h1>Powered by anynines</h1>
  </body>
</html>

Let’s make a change to our code. Let’s say we made a mistake and the background color should be orange, not blue. Here’s the updated code:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/gorilla/mux"
)

const INDEX = `<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>`

func main() {
	router := mux.NewRouter()
	router.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(w, INDEX)
	})

	log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), router))
}

We’ll now deploy the new version using the canary strategy: –strategy canary

 cf push cf-push-strategy-canary --strategy canary

At this point, Cloud Foundry starts one new instance with the updated code, pauses the deployment, and waits for your confirmation to continue. During this time, traffic is split between the new and old instances.

Starting deployment for app cf-push-strategy-canary...
Waiting for app to deploy...

Instances starting...
Instances starting...

name:              cf-push-strategy-canary
requested state:   started
routes:            cf-push-strategy-canary.apps.anynines-testing.env
last uploaded:     Fri 01 Aug 10:56:13 CEST 2025
stack:             cflinuxfs4
buildpacks:
	name           version   detect output   buildpack name
	go_buildpack   1.10.36   go              go

type:            web
sidecars:
instances:      1/1
memory usage:    256M
start command:   ./bin/mod
     state     since                  cpu    memory          disk           logging             cpu entitlement   details
#0   running   2025-08-01T08:50:18Z   0.4%   14.2M of 256M   7.6M of 512M   0B/s of unlimited   57.3%

type:            web
sidecars:
instances:       1/1
memory usage:    256M
start command:   ./bin/mod
     state     since                  cpu    memory     disk       logging        cpu entitlement   details
#0   running   2025-08-01T08:56:27Z   0.0%   0B of 0B   0B of 0B   0B/s of 0B/s   0.0%

Active deployment with status PAUSED (since Fri 01 Aug 10:56:29 CEST 2025)
strategy:        canary
max-in-flight:   1
canary-steps:    1/1

Please run `cf continue-deployment cf-push-strategy-canary` to promote the canary deployment, or `cf cancel-deployment cf-push-strategy-canary` to rollback to the previous version.

We can now see two web processes: The Canary process, and our original. If we curl our app, we can also see that the result differs. Sometimes we receive background color blue, sometimes we receive background color orange.

curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:blue;">
    <h1>Powered by anynines</h1>
  </body>
</html>
MacBook-Pro:mod benjaminguttmann$ curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>

As this makes it hard to verify your new app version, there is a way to specifically get routed to your canary app. This can be done by setting a specific header which looks like this:

"X-Cf-Process-Instance":"<PROCESS_GUID>:<INDEX>"

Below you can see how to get the process guid and the index number to use for the header.

cf curl /v3/apps/$(cf app cf-push-strategy-canary --guid)/processes | jq .
{
  "pagination": {
    "total_results": 2,
    "total_pages": 1,
    "first": {
      "href": "https://api.system.anynines-testing.env/v3/apps/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1/processes?page=1&per_page=50"
    },
    "last": {
      "href": "https://api.system.anynines-testing.env/v3/apps/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1/processes?page=1&per_page=50"
    },
    "next": null,
    "previous": null
  },
  "resources": [
    {
      "guid": "50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1",
      "created_at": "2025-08-01T08:47:31Z",
      "updated_at": "2025-08-01T08:53:52Z",
      "version": "fdb95ace-ebc7-4d66-8808-472704440c13",
      "type": "web",
      "command": "[PRIVATE DATA HIDDEN IN LISTS]",
      "instances": 1,
      "memory_in_mb": 256,
      "disk_in_mb": 512,
      "log_rate_limit_in_bytes_per_second": -1,
      "health_check": {
        "type": "port",
        "data": {
          "timeout": null,
          "invocation_timeout": null,
          "interval": null
        }
      },
      "readiness_health_check": {
        "type": "process",
        "data": {
          "invocation_timeout": null,
          "interval": null
        }
      },
      "relationships": {
        "app": {
          "data": {
            "guid": "50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1"
          }
        },
        "revision": {
          "data": {
            "guid": "cd8ebedf-9357-4051-91dc-58751041f289"
          }
        }
      },
      "metadata": {
        "labels": {},
        "annotations": {}
      },
      "links": {
        "self": {
          "href": "https://api.system.anynines-testing.env/v3/processes/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1"
        },
        "scale": {
          "href": "https://api.system.anynines-testing.env/v3/processes/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1/actions/scale",
          "method": "POST"
        },
        "app": {
          "href": "https://api.system.anynines-testing.env/v3/apps/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1"
        },
        "space": {
          "href": "https://api.system.anynines-testing.env/v3/spaces/d5e1f384-3a3d-4882-bf21-dbd23bf2980f"
        },
        "stats": {
          "href": "https://api.system.anynines-testing.env/v3/processes/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1/stats"
        }
      }
    },
    {
      "guid": "e3838ce1-49a1-4c96-9b7e-111857181a70",
      "created_at": "2025-08-01T08:56:20Z",
      "updated_at": "2025-08-01T08:56:20Z",
      "version": "4385976a-d48b-49c8-9041-bb358fa73bd0",
      "type": "web",
      "command": "[PRIVATE DATA HIDDEN IN LISTS]",
      "instances": 1,
      "memory_in_mb": 256,
      "disk_in_mb": 512,
      "log_rate_limit_in_bytes_per_second": -1,
      "health_check": {
        "type": "port",
        "data": {
          "timeout": null,
          "invocation_timeout": null,
          "interval": null
        }
      },
      "readiness_health_check": {
        "type": "process",
        "data": {
          "invocation_timeout": null,
          "interval": null
        }
      },
      "relationships": {
        "app": {
          "data": {
            "guid": "50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1"
          }
        },
        "revision": {
          "data": {
            "guid": "b6186456-900f-4f49-b682-b85037061885"
          }
        }
      },
      "metadata": {
        "labels": {},
        "annotations": {}
      },
      "links": {
        "self": {
          "href": "https://api.system.anynines-testing.env/v3/processes/e3838ce1-49a1-4c96-9b7e-111857181a70"
        },
        "scale": {
          "href": "https://api.system.anynines-testing.env/v3/processes/e3838ce1-49a1-4c96-9b7e-111857181a70/actions/scale",
          "method": "POST"
        },
        "app": {
          "href": "https://api.system.anynines-testing.env/v3/apps/50a51f8f-3273-4a0c-a8b0-a67f7ef5bbc1"
        },
        "space": {
          "href": "https://api.system.anynines-testing.env/v3/spaces/d5e1f384-3a3d-4882-bf21-dbd23bf2980f"
        },
        "stats": {
          "href": "https://api.system.anynines-testing.env/v3/processes/e3838ce1-49a1-4c96-9b7e-111857181a70/stats"
        }
      }
    }
  ]
}

After collecting all of our information, we can set the correct header in our curl request to get routed to our canary instance.

curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL -H "X-Cf-Process-Instance":"e3838ce1-49a1-4c96-9b7e-111857181a70:0"
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>

Now that we verified that our new app version looks and works as intended, we can finish our deployment with:

cf continue-deployment cf-push-strategy-canary
cf continue-deployment cf-push-strategy-canary
Continuing deployment for app cf-push-strategy-canary in org system / space bg as admin-2024...

Waiting for app to deploy...

Instances starting...

name:              cf-push-strategy-canary
requested state:   started
routes:            cf-push-strategy-canary.apps.anynines-testing.env
last uploaded:     Fri 01 Aug 10:56:13 CEST 2025
stack:             cflinuxfs4
buildpacks:
	name           version   detect output   buildpack name
	go_buildpack   1.10.36   go              go

type:           web
sidecars:
instances:      1/1
memory usage:   256M
     state     since                  cpu    memory          disk           logging             cpu entitlement   details
#0   running   2025-08-01T08:56:26Z   0.9%   14.3M of 256M   7.6M of 512M   0B/s of unlimited   121.6%


TIP: Run 'cf app cf-push-strategy-canary' to view app status.


And, lastly, we can verify that only our new app version is available now. 

MacBook-Pro:mod benjaminguttmann$ curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>
MacBook-Pro:mod benjaminguttmann$ curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>
MacBook-Pro:mod benjaminguttmann$ curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>
MacBook-Pro:mod benjaminguttmann$ curl https://cf-push-strategy-canary.apps.anynines-testing.env -kL
<!DOCTYPE html>
<html>
  <body style="background-color:orange;">
    <h1>Powered by anynines</h1>
  </body>
</html>

The goal of canary deployments is to roll out a new version of your app to a single instance first: the canary. This gives you a chance to monitor how it behaves in production without impacting the rest of your users. If the canary instance performs reliably, you can then promote the new version to more (or even all) instances.

You can use the –strategy canary option in a weighted way by defining –instance-steps. This lets you gradually update more instances over time instead of jumping from one to full rollout immediately.

One practical tip from the official docs: if you include 100 as the final step value, you’re still able to revert the deployment even after all instances have been updated. This gives you a safety net in case an issue only appears at scale. However, reverting at this point does involve downtime, since all instances have already switched to the new version.

2. Route-based load balancing algorithm

The next feature I want to show you is the route-based load balancing algorithm. This is a great addition that allows you to change the default “round-robin” load balancing approach to a “least-connection” approach for your application without changing anything globally.

Here’s a quick video walkthrough of this feature:

To showcase this, we adjusted our sample app a bit to include a sleep function based on the index instance. This causes some instances to respond more slowly, and will help us better simulate actual scenarios where some instances hold connections longer than others. 

Our app looks as follows:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	"github.com/gorilla/mux"
)

const INDEX = `<!DOCTYPE html>
<html>
  <body style="background-color:blue;">
    <h1>Powered by anynines</h1>
  </body>
</html>`

func main() {
	interval, _ := time.ParseDuration(os.Getenv("INSTANCE_INDEX") + "s")
	var max_interval time.Duration
	max_interval, _ = time.ParseDuration("5s")
	router := mux.NewRouter()
	router.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(w, INDEX)
		time.Sleep(max_interval - interval)
	})

	log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), router))
}

We now push the app to Cloud Foundry:

cf push cf-route-based-loadbalancing-round-robin

Waiting for app cf-route-based-loadbalancing-round-robin to start...

Instances starting...
Instances starting...
Instances starting...

name:              cf-route-based-loadbalancing-round-robin
requested state:   started
routes:            cf-route-based-loadbalancing-round-robin.apps.anynines-testing.env
last uploaded:     Fri 01 Aug 12:59:44 CEST 2025
stack:             cflinuxfs4
buildpacks:
	name           version   detect output   buildpack name
	go_buildpack   1.10.36   go              go

type:            web
sidecars:
instances:       1/1
memory usage:    256M
start command:   ./bin/mod
     state     since                  cpu    memory     disk       logging        cpu entitlement   details
#0   running   2025-08-01T10:59:56Z   0.0%   0B of 0B   0B of 0B   0B/s of 0B/s   0.0%

Afterwards, we scale the app to 5 instances and use a small script (as seen below) to create a number of requests. While those requests are sent, we gather the logs of our app via CF CLI and pipe them into a file.

cf8 logs cf-route-based-loadbalancing-round-robin |  egrep -o "app_index:\"\d\"" >> /tmp/app_output_round_robin.txt

Based on the logs, we can later check how CF balanced our requests.

#! /bin/bash

for i in $(seq 1 1000);
do
  curl cf-route-based-loadbalancing-round-robin.apps.anynines-testing.env -Lk &
  # we do a sleep here so that CF has more time to balance the requests
  sleep 0.1
done

After our run is finished, we can check the logs we stored:

 cat /tmp/app_output_round_robin.txt | sort | uniq -c
   1 app_index:
 173 app_index:"0"
 173 app_index:"1"
 175 app_index:"2"
 178 app_index:"3"
 178 app_index:"4"

It’s a pretty steady distribution. 

Next, we deploy our app with the least-connection load balancing algorithm configured. This is done by specifying the desired algorithm directly in the app’s manifest file.

---
applications:
- name: cf-route-based-loadbalancing-least-connection
  routes:
    - route: cf-route-based-loadbalancing-least-connection.apps.anynines-testing.env
      options:
        loadbalancing: least-connection

Now, we push the app:

cf push -f least-connection-manifest.yml
Pushing app cf-route-based-loadbalancing-least-connection to org system / space bg as admin-2024...

Applying manifest file least-connection-manifest.yml...

Updating with these attributes...
  ---
  applications:
+ - name: cf-route-based-loadbalancing-least-connection
+   default-route: true
+   routes:
+   - options:
+       loadbalancing: least-connection
+     route: cf-route-based-loadbalancing-least-connection.apps.anynines-testing.env
Manifest applied
Packaging files to upload...

...
Waiting for app cf-route-based-loadbalancing-least-connection to start...

Instances starting...
Instances starting...
Instances starting...

name:              cf-route-based-loadbalancing-least-connection
requested state:   started
routes:            cf-route-based-loadbalancing-least-connection.apps.anynines-testing.env
last uploaded:     Fri 01 Aug 13:19:21 CEST 2025
stack:             cflinuxfs4
buildpacks:
	name           version   detect output   buildpack name
	go_buildpack   1.10.36   go              go

type:            web
sidecars:
instances:       1/1
memory usage:    256M
start command:   ./bin/mod
     state     since                  cpu    memory     disk       logging        cpu entitlement   details
#0   running   2025-08-01T11:19:32Z   0.0%   0B of 0B   0B of 0B   0B/s of 0B/s   0.0%

We then scale the application to 5 instances and run our script to generate requests again. Like before, we collect the logs to analyze how the requests were distributed. 

According to our setup, instances with higher index numbers have shorter response times. So, under the “least-connection” algorithm, we’d expect most requests to land on index 4. Let’s see if the results confirms this:

 cat /tmp/app_output_least_connection.txt | sort | uniq -c
   1 app_index:
  97 app_index:"0"
 109 app_index:"1"
 145 app_index:"2"
 196 app_index:"3"
 330 app_index:"4"

As expected, the majority of requests were routed to the instance with the shortest response time, confirming that the least-connection algorithm works as intended. This demonstrates how Cloud Foundry’s new route-based load balancing can optimize request handling with uneven workloads.

Conclusion

In this post, we explored two powerful new features in Cloud Foundry: canary deployments and route-based load balancing. Canary deployments allow for safer rollouts by introducing new application versions gradually starting with a single instance and scaling up in controlled steps. With the –strategy canary and –instance-steps options, you gain fine-grained control over how updates are rolled out, including the ability to roll back even after a full rollout.

We also demonstrated how to configure least-connection load balancing via the app manifest, providing a smarter alternative to the default round-robin behavior. This approach can significantly improve performance in scenarios where response times vary across instances. As long-time Cloud Foundry experts, we’re excited to see these enhancements make CF even more powerful for modern deployment workflows, and we’re here to help you make the most of them.

, , , , , ,
THE AUTHOR

Benjamin Guttman

Share this article
Recommended Topics
Popular Tags

Cloud Foundry (8) data service management (7) DEI (4) diversity (4) GoLang (6) hybrid cloud (4) inclusive tech (4) Klutch (4) Kubernetes (8) Tutorials (16)

Other interesting posts you may like