How to Schedule Background Jobs Using the Clockwork-Gem

First of all the most obvious question has to be discussed:

Table of Contents

• Why we can’t just use cron for scheduled jobs?
• Clockwork – A tool to schedule command execution
• How do i create a working schedule with clockwork?
• Ok, but how can we deploy the worker to Anynines?
• Further reading

Why we can’t just use cron for scheduled jobs?

An essential part of a PaaS is to isolate running application instances from another. Therefore all application instances run in so called Warden containers. Warden is an operating system level virtualization framework shipped with Cloud Foundry, the foundation of anynines.

Most Warden containers will run application instances that will never need a CRON daemon. Running CRON jobs on application servers is generally not a good idea as it creates a coupling between application delivery and background processing.

One of the major benefits of using a PaaS such as anynines is that scaling is easy.

In order to achieve this it’s necessary to keep the application server simple which goes down to the basic fact:

It’s much easier to scale application servers if all it does is to serve the application.

That’s the main point.

Background processing of any kind is intended to happen apart from serving incoming application HTTP requests.

Read more about how Cloud Foundry generally deals with background processing in a serious of interesting blog posts over at CloudFoundry.com.

Basically, the article explains that Cloud Foundry supports the execution of stand-alone applications, applications that do not respond to HTTP requests.
So CRON is basically a special case of background processing and therefore

all we need is a tool that allows to schedule the execution of given commands. 

Clockwork – A tool to schedule command execution

The ruby based clockwork-gem does exactly this. It is very useful, especially in the context of a Ruby application (with Rails, Sinatra, Padrino, etc) as it is possible to require the whole app-environment in the jobs. Of course, one should always be aware of the following design principle:

heavy lifting does not belong into a CRON (like) job.
CRON is about scheduling jobs, not doing them. 

For heavy lifting a background worker system is recommended. See [1] for more details.

How do i create a working schedule with clockwork?

First, add the following line to your Gemfile:
gem 'clockwork', require: false
Then create a schedule file to define the jobs and the scheduling:
# script/workers_schedule.rb

require 'clockwork' # Require the full rails environment if needed require './config/boot' require './config/environment' include Clockwork # Define the jobs handler do |job| if job.eql?('frequent.cleanup_sessions') Sessions.cleanup_expired() elsif job.eql?('mondays.send_news') NewsDispatcher.send_news_to_customers() end end

# Define the schedule every(1.hour, 'frequent.cleanup_sessions') every(1.day, 'mondays.send_news', :at => '01:00', :if => lambda { |t| t.wday == 1 })
More information about the scheduling parameters can be found in the parameters section of the clockwork-gem readme.

You can test your schedule by executing the following command in the shell:
clockwork script/clockwork_jobs.rb

Ok, but how can we deploy the worker to Anynines?

You have to add the new worker-application with a custom start command in your manifest.yml manually:

--- applications:

• name: Super-Useful-App memory: 512MB instances: 1 host: useful-as-hell domain: de.a9sapp.eu path: . services:
  • mysql-ush0815
• name: Super-Useful-Worker memory: 512MB instances: 1 no-route: true path: . command: 'bundle exec clockwork script/workers_schedule.rb' services:
  • mysql-ush0815

Now just push your application as usual (but with the –reset flag to accept the manual manifest.yml changes) and wait for your new shiny scheduling to trigger.

Further reading

[1] Rethinking Cron