How to Use the anynines Swift Service With Paperclip

Web applications often produce files such as images, videos or documents. For a long time it was standard to put those files into the filesystem. Today this is considered a bad practice.

Table of Contents

• Why is storing to filesystem bad?

• That’s where OpenStack Swift comes in.

• OpenStack Swift

• Paperclip

• Paperclip + Swift demo application

  • Gemfile adjustments
  • Paperclip Initializer
  • Active Record Initializer
  • .cfignore
  • Deployment manifest
  • Deploy the application to anynines.com

Why is storing to filesystem bad?

Using the filesystem as persistent storage breaks with rule number 6 of the 12factor manifest. This introduces two major architectural flaws:

1. Filesystems don’t scale well.
When storing a large number of files there is a point where a single server is not enough anymore. At this point filesystems become a problem as they don’t span across servers. Managing distributed filesystems is… painful.

2. A filesystem is a single point of failure.
Servers die and so will file servers. Making NFS servers, for example, redundant is an awkward task because it was never designed to be highly available.

Instead we want a storage service that is scalable and highly available by nature.

That’s where OpenStack Swift comes in.

Swift is highly scalable and can serve up to hundreds of petabytes. More than that it is also redundant. Data is replicated across a number of storage nodes. This allows Swift to survive the outage of one or multiple servers.

So let’s achieve our goal of creating stateless application instances by using a content delivery network based approach to separate our running code from the content data using OpenStack Swift.

OpenStack Swift

The OpenStack Object Storage project (codename “Swift”) implements an object storage system that can be accessed using a REST-ful web api. Swift provides online file interaction comparable to Amazon’s Simple Storage Service (Amazon S3):

You can upload and download files to or from the Swift storage cluster using standard CRUD operations.

The uploaded files can be organized in containers, so called buckets, for public or private access.

By using an object storage we avoid having state in our application instances. All instances can create, read and change the files stored in the Swift container bound the corresponding application.

Swift creates another advantage. By uploading files to Swift a URI is returned. This URI can be used in HTML documents so clients will retrieve assets directly from Swift instead of hitting the application stack. Think of it as simple CDN. Hence, the application  is freed from traffic caused by asset delivery being able to server more application requests.

The anynines Swift service forms the interface between the anynines Cloud Foundry and the Swift storage cluster. When binding the Swift service to an application it creates a user in the Swift cluster and provides the account credentials to the app instance as environment variables. You just have to read the credentials and start using the Swift object storage.

Paperclip

Paperclip is a widely used file attachment library for Active Record. It simplifies implementing file attachments for Ruby applications. For an introduction on how to use Paperclip please refer to this excellent RailsCasts screencast.

Paperclip + Swift demo application

Let’s dive into a hands-on example application. We will use the Paperclip-demo reference application to describe the usage of Paperclip with the anynines Swift service. The repository is a fork of Thoughtbot’s Paperclip reference application adjusted to the anynines use case. The app allows to add friends with attached images to a database. You can run the app locally by executing the following commands:
$> git clone git@github.com:anynines/paperclip_demo.git $> cd paperclip_demo $> bundle $> rake db:create db:migrate $> bundle exec rails s
A diff including all adjustments to the original app can be viewed here. The next paragraphs describe the adjustments in detail.

Gemfile adjustments

To enable Paperclip’s support for OpenStack Swift we added the fog gem to the Gemfile. The fog gem provides the api for accessing the object store to Paperclip. In addition we set the Ruby version to 2.0.0 and added the rails_12factor gem to adjust the application to the anynines PaaS environment.

Paperclip Initializer

Since anynines provides the Swift credentials within environment variables, we need to read the application environment and configure Paperclip to use the fog driver with the retrieved credentials. The config/initializers/paperclip.rb file implements the needed Paperclip configuration steps:

1. – Parse the Swift credentials from the environment
2. – Create a public Swift bucket to store our images if the bucket isn’t already present
3. – Configure paperclip to use the Swift credentials

Active Record Initializer

We added an initializer for the automatic execution of Active Record migrations at application startup in config/initializers/run_migrations.rb. This simplifies the app deployment process since we don’t need to execute the migrations manually when deploying the application for the first time.

.cfignore

The .cfignore file is used to exclude files and directories from the Cloud Foundry deployment process. Since we don’t want to upload our locally added images, logs and temporary files to the anynines servers, we added the according directories to the .cfignore file.

Deployment manifest

The application provides a manifest.yml example file which has to be configured to address your desired application identifier and route within the anynines system. Please copy the manifest.yml.example to app_root/manifest.yml and replace all occurrences of app_name with your desired application identifier.

Deploy the application to anynines.com

You can deploy the application to anynines by executing the cf push command from the applications root directory:
$> cf push
For further information on how to deploy applications to anynines please refer to our documentation.

When the command finishes deploying the app it displays the according url to access the application instances. Visit the url to test your newly deployed Paperclip example and try to add friends to your app by uploading their pictures!