mixd-deployment

Deploying WordPress using Git and Capistrano

UPDATE – March 2014: With the release of Capistrano 3, a lot of what is covered in this post is now obsolete. However, we recently released wp-deploy – a framework for deploying WordPress projects with Capistrano 3, which covers everything written here and a while bunch more.


This week at Mixd I've been working on completely overhauling our WordPress deployment process – something that we've been meaning to do for a long, long time. The entire process has been a huge learning curve for me, and despite some major banging my head against a wall moments I found it incredibly rewarding, and we're already seeing massive improvements in our workflow. This post gives an in depth overview of what I've learnt over the past week and explains how to best structure a WordPress git repository and deploy it to multiple environments using Capistrano.

Up until now we've deployed entirely using FTP which is troublesome for a number of reasons:

  1. It's slow
  2. It's insecure
  3. Theres no easy way to quickly deploy to different environments
  4. It encouraging "cowboy coding" and hotfixing which can easily be wiped or overwritten
  5. Recent changes can only be detected by modified date
  6. It's difficult to roll back to an older release

It's just a complete pain.

We started using git over a year ago and it completely revolutionised our development process. There are plenty of ways to deploy an application via git, but we were shamefully lazy; and just stuck with what we were comfortable with at the time. When we decided we were going to change our deployment process, we did a lot of research into which deployment method would work for us – not only which was the easiest, but which would work best with our current workflow.

Why Capistrano?

Having worked on a number of Ruby on Rails projects, I'd become familiar with Capistrano – a command line tool used primarily for deploying large scale Rails applications. It was completely new to me and I didn't fully understand what was going on behind the scenes but I loved how simple and straightforward it was. If I wanted to deploy the development branch of a repo to the production server, I'd simply run cap production deploy. If I wanted to roll the staging server back to the previous release, I'd run cap staging deploy:rollback. It just made sense.

My favourite thing about Capistrano is that it allows you to write your own deployment scripts. That means you can write scripts that deploy mysql databases, take backups, and most importantly, dynamically create environment specific WordPress configuration files.

So to get started, there are 3 gems to install:

gem install capistrano
gem install capistrano-ext
gem install railsless-deploy

capistrano-ext extends the core Capistrano functionality to allow for multiple stages (amongst other things), whilst railsless-deploy strips out a lot of guff that we don't need for deploying PHP applications.

A word on permissions

One of the most common issues I ran into whilst working with Capistrano was down to permissions and RSA key issues. Before proceeding, make sure you have SSH keys set up between your local machine and your server, and between your local machine and your GitHub account.

A better WordPress structure

For Capistrano to be most effective you need to keep all your WordPress files in the repo. Core files, plugins, and all. The best (and cleanest) way to do this is to sit your WordPress core in it's own directory and using a custom content folder for uploads, themes and plugins. I structure my repository as follows:

/content/themes/
/content/uploads/
/content/plugins/
/wordpress/
index.php
wp-config.php

Your uploads directory should be git ignored as you don't want to version control or deploy them. Your wp-config.php here should just contain your local db settings so you can git ignore that too – we'll create new config files for your production environment later.

You can keep your wp-config.php outside of the WordPress directory and WP will still find it, and index.php can be pointed to your new directory by changing a single line:

require('./wordpress/wp-blog-header.php');

Adding WordPress as a submodule

You can include WordPress as a git submodule which is super handy and helps keep your structure more modular. You can even update WordPress straight from git with a quick git fetch && git checkout 3.5.1 which helps for consistency accross environments and between developers. Adding WordPress as a submodule is easy:

git submodule add git://github.com/WordPress/WordPress.git wordpress
cd wordpress
git checkout 3.5.1

The WordPress repo uses branches for major releases and tags for minor releases, so a checking out a branch/tag will change the version your use. You can even checkout the master branch to keep on the latest WordPress beta – just make sure you don't do that for production sites!

Capifying your project

So now your repo is sparkling clean, we need to add a few configuration files so Capistrano knows what to do. That's easy, just cd into your repo and run capify. This will create two files, Capfile and config/deploy.rb.

Capfile is Capistrano's first port of call when you run any of it's commands. It's default content is tailored specifically for Ruby projects, so remove that and replace it with the following:

require 'railsless-deploy'
load 'config/deploy'

That requires the railsless-deploy gem we installed earlier, and loads in your deployment scripts.

Open up your config/deploy.rb and replace it's content with the following:

set :scm, :git
set :repository, "git@github.com:Mixd/your-git-repo.git"
set(:git_enable_submodules, true)

set :user, "sshuser"
server "www.example.com", :app
set :deploy_to, "/var/www/vhosts/example.com/httpdocs"

ssh_options[:forward_agent] = true
set :deploy_via, :remote_cache
set :copy_exclude, [".git", ".DS_Store", ".gitignore", ".gitmodules"]
set :use_sudo, false

So lets quickly discuss what each of these lines do.

  • set :scm, :git Tell Capistrano we're using git
  • set :repository, "git@github.com:Mixd/your-git-repo.git" sets the git repo where you're deploying from
  • set(:git_enable_submodules, true) Allows us to deploy our WordPress submodule
  • set :user, "sshuser" defines your SSH username
  • server "www.example.com", :app defines the server you're deploying to
  • set :deploy_to, "/var/www/vhosts/example.com/httpdocs" is the path on the server to deploy to
  • ssh_options[:forward_agent] = true forces SSH to use your local machine's public key (prevents previously mentioned permission issues)
  • set :deploy_via, :remote_cache tells Capistrano to keep a cached copy of your repo on the server to speed up deployments
  • set :copy_exclude, [".git", ".DS_Store", ".gitignore", ".gitmodules"] is an array of files you don't want to deploy but still want to keep in the repo
  • set :use_sudo, false don't use sudo for SSH commands. Best keep this false to security purposes.

Here's what your local directory should be looking like now:

/config/deploy.rb
/content/themes/
/content/uploads/
/content/plugins/
/wordpress/
Capfile
index.php
wp-config.php

Dealing with Capistrano's directory structure

Before you jump the gun and start deploying, you'll need to run cap deploy:setup from within your repo. This will tell Capistrano to connect to your server and set up the folders it needs ready for deployments:

/releases
/releases/20130605001122
/releases/...
/current => latest release
/shared

The releases folder contains a folder for every deployment you make, and is named with a timestamp of when the deployment was made. current is a symlink to the most recent release, and shared will house all other content that doesn't need to be version controlled and/or deployed in Capistrano – your Uploads directory for example.

Creating a symlink for your uploads directory

Because our uploads directory lives in content/uploads in our repo, but in shared/uploads on the server, we need to create a symlink between the two locations. In your config/deploy.rb file, add the following:

namespace :wordpress do
    desc "Setup symlinks for a WordPress project"
    task :create_symlinks, :roles => :app do
        run "ln -nfs #{shared_path}/uploads #{release_path}/content/uploads"
    end
end

after "deploy:create_symlink", "wordpress:create_symlinks"

What we're doing here is creating a namespace called :wordpress with a task called create_symlinks which (funnily enough) creates a symlink between the uploads folder in the release and the one in the shared directory. You could run this task manually using cap wordpress:create_symlinks but we want it to run after every single deploy, so the last line adds the task as a callback of the deploy:create_symlink task which runs at the end of each deployment – perfect.

Creating your WordPress config file

The next issue to deal with is how to manage your WordPress wp-config.php. When you're working locally you can just set up your wp-config.php in your project root as you normally would, but that's not going to work on your remote environments. You could keep your remote database details in your config file and keep that in your git repo, but you obviously won't want to publicise your database details to everyone with access to your repository.

A common way to deal with this is to have your wp-config.php pull in your database details from a separate file which only exists on your remote environments:

if ( file_exists( dirname( __FILE__ ) . '/wp-config-production.php' ) ) {
    require( 'wp-config-production.php' );
}else {
   // local database details
}

So all you'd need to do keep a wp-config-production.php in your production environment which has your WordPress database constants and you can happily keep your wp-config.php in your repo without worrying about publicising your database details.

Creating your config file on the fly

The above works great works great in most circumstances, but I wanted to make the process a little more foolproof so anyone in the development team can easily set up a new environment without the need of manually creating a new environment-specific config file. I used a Capistrano recipe to dynamically create a wp-config.php on the fly when you set up a new environment:

desc "Create files and directories for WordPress environment"
    task :setup, :roles => :app do
        set(:secret_keys, capture("curl -s -k https://api.wordpress.org/secret-key/1.1/salt") )
        set(:wp_siteurl, Capistrano::CLI.ui.ask("Site URL: ") )
        set(:wp_dbname, Capistrano::CLI.ui.ask("Database name: ") )
        set(:wp_dbuser, Capistrano::CLI.ui.ask("Database user: ") )
        set(:wp_dbpass, Capistrano::CLI.ui.ask("Database password: ") )
        set(:wp_dbhost, Capistrano::CLI.ui.ask("Database host: ") )

        db_config = ERB.new <<-EOF 
<?php
    define('DB_NAME', '#{wp_dbname}');
    define('DB_USER', '#{wp_dbuser}');
    define('DB_PASSWORD', '#{wp_dbpass}');
    define('DB_HOST', '#{wp_dbhost}');
    define('DB_CHARSET', 'utf8');
    define('DB_COLLATE', ''); 
    define('WPLANG', '');

    $table_prefix  = 'wp_';

    #{secret_keys}

    define('WP_HOME','#{wp_siteurl}');
    define('WP_SITEURL','#{wp_siteurl}/wordpress');

    define('WP_CONTENT_URL', '#{wp_siteurl}/content');
    define( 'WP_CONTENT_DIR', $_SERVER['DOCUMENT_ROOT'] . '/content' );

    if ( !defined('ABSPATH') )
            define('ABSPATH', dirname(__FILE__) . '/');

    require_once(ABSPATH . 'wp-settings.php');
    EOF

    put db_config.result, "#{shared_path}/wp-config.php"
end

This task is run on the callback of the cap deploy:setup command, and prompts the user for database details (and other variables used we need for wp-config.php) in Terminal and creates the file in the /shared directory of the server for you. It also retrieves the WordPress security keys from the API and adds those to the file.

Letting Capistrano deal with this extra legwork for you can save lots of time in the initial stages of a project and ensures the wp-config is set up exactly how we need it.

I also added another line to this task to create the /uploads directory in the shared directory for us, to save us a job later:

run "mkdir -p #{shared_path}/uploads"

All we need to do now is add a new symlink to the :create_symlinks task we created earlier to point the wp-config in your latest releases to the new one in your shared directory:

run "ln -nfs #{shared_path}/wp-config.php #{release_path}/wp-config.php"

Dealing with multiple environments

Another great thing about Capistrano is how easy it is to deal with and deploy to multiple environments. The typical project I work on has 3 environments: development, staging, and production. With Capistrano you can define different servers, paths and deployment scripts for each different environment.

You'll need to add 3 new lines to your deploy.rb file:

require 'capistrano/ext/multistage'
set :stages, %w(production staging)
set :default_stage, "staging"

This pulls in the multistage functionality from the capistrano-ext gem and sets up two environments: production and staging. We also set the default stage to staging so if we don't specify an environment to deploy to, we'll deploy staging.

From there you just need two new files /config/deploy/staging.rb and /config/deploy/production.rb. Any variables you set in staging.rb will override anything in your deploy.rb when you run cap staging deploy and likewise for production.rb. That means you can set a different server and for your staging environment by by editing staging.rb:

set :user, "sshuser"
server "X.XX.X.XXX", :app
set :deploy_to, "/var/www/vhosts/example.com/httpdocs"

You can also tell Capistrano to deploy a particular branch to each environment:

set :branch, "development"

Rounding up

With everything fully set up, your local repo should look something like this:

/config/deploy.rb
/config/deploy/staging.rb
/config/deploy/production.rb
/content/themes/
/content/uploads/
/content/plugins/
/wordpress/
Capfile
index.php
wp-config.php

So that's it! Well, the basics of it anyway – theres so, so much more you can do, and theres an awful lot I'm yet to learn. Capistrano is a great tool, and this article only touches the surface of what you can do. Backing up and deploying databases, for example, is possible with reasonable ease and could potentially make site migrations an awful lot simpler – I think I'll make that my next challenge.

As I mentioned at the start, this entire process was completely new to me a week ago and I'm by no means an expert, so if you disagree with how I've done something or have any questions, please do say so.

Leave a comment