Using CVS for collaborative website development

This document describes a method of using CVS as the centerpiece of a web development environment that enables multiple developers to work on the same or different parts of the website at the same time with a minimum of inter-developer toe-stepping-on.

Some Definitions

I'll assume you're working on a website called www.example.com. You also have, on a separate machine (perhaps inside your firewall), a staging server called staging.www.example.com. Both of these servers have a document root of /www/www.example.com. Additionally, the directory /www/support/www.example.com contains files and directories that you use for the website, but you don't want to be under the document root (.htpasswd-style authorization files, other included page parts, etc.)

The files at /www/www.example.com and /www/support/www.example.com on the staging server are a checked-out copy of the files in the CVS repository. After a developer commits changes, the staging server manager merges in the changes and updates the checked out copy that is under the staging server's document root.

Developers each have their own virtual server that points at a sandbox directory they can use. The virtual servers can be called something like username.dev.example.com. The document root of the sandbox server is a directory underneath the developer's home directory, say ~username/username.dev.example.com/document-root. It is helpful if this document root doesn't really point to an actual directory, but a symlink that the developer can use to point to one of several branches he may have checked out and be working on.

Developers

When a developer wants to begin work on a new feature or bugfix, he uses the cvs rtag command to create a branch for his work:

[1]$ cvs rtag -b -r staging-current my-feature-name example

where staging-current is the tag that the staging server manager applies to the latest functional, tested, set of files on the staging server, my-feature-name is the descriptive name of the developer's branch, and example is the CVS module alias for the support/www.example.com and www.example.com directories.

Once the developer has created a branch for his work, he can check out a set of files and begin editing them. To keep separate different branches the developer may have checked out at once, he should make a directory under ~username/username.dev.example.com, and then cd into it. Then, to check out the files in the newly-created branch, do:

[2]$ cvs checkout -r my-feature-name example

This will create support/www.example.com and www.example.com subdirectories of the directory he is in when he issues the cvs checkout command and populate them with the files from the repository. To point the sandbox server at these directories, he should symlink ~username/username.dev.example.com/document-root to the created www.example.com directory, and ~username/username.dev.example.com/support to the created support/www.example.com directory. (This assumes that the developer's sandbox server is using ~username/username.dev.example.com/document-root as the document root and is looking in ~username/username.dev.example.com/support for the support files.)

The developer can edit these files and the changes will be immediately visible on the sandbox server. When the developer has finished with the bugfix or feature addition, he needs to commit the changes. However, before he does, he should resynchronize the sandbox with any changes that may have been made to the staging server. He does this by typing, in the parent directory of the support/www.example.com and www.example.com directories:

[1]$ cvs update -j staging-current

This will scan the developer's files and attempt to merge in any changes made to the staging server while you have been working on your feature or bugfix. CVS will print out any unresolvable merge conflicts, which should be resolved manually (and re-committed) before notifying the staging server manager that the feature or bugfix is complete. If the developer uses

[1]$cvs -q update -j staging-current

to synchronize the sandbox with the staging server, CVS will use a terser output format, and just print the files that were changed or had a conflict, instead of listing all of the directories into which it recursed.

Once the feature or bugfix is complete and the branch is synchronized with the staging server, the developer should notify the staging server manager that the work is done and that it should be merged into the staging server for testing.

There may be times where the developer cannot avoid merge conflicts between his branch and the main trunk of the staging server. He should make sure to notify the staging server manager of the files in which this happens.

When a developer is done with a branch, he can just remove the directory from the sandbox that the branch is in.

Merging into the Staging Server

When the staging server manager is notified that a branch is ready for integration into the staging server, it should cd to the staging server's support/www.example.com and www.example.com directories and then attempt to merge the new branch in by running, in each directory:

[1]$ cvs update -j branch-name

where branch-name is the name of the new branch. CVS will attempt to merge in the changes from the branch. If the developer has properly synchronized its sandbox with the staging server before notifying the staging server manager that the branch is ready for merging, there shouldn't be any conflicts when merging in the branch. However, if there are conflicts, the staging server manager can

Edit the files with conflicts and resolve the conflicts
Request the developer resynchronize its sandbox and resolve the conflicts
Back out the staging server to the staging-current revisions by removing individual files and then restoring them with cvs update -r staging-current filename. Note that this sets a sticky tag that must be removed with cvs update -A later.

The staging server manager may find helpful CVS's -tn flags, which will show a trace of program execution and not execute anything that will change the disk. A command like

[1]$ cvs -tn update -j branch-name

will alert the staging server manager about potential conflicts without actually changing any of the files on the staging server.

Once the conflicts are resolved (or if there are no conflicts after the merge) the staging server manager needs to retag the staging server. (The following commands should all be executed from the parent of the www.example.com and support/www.example.com directories.) First, commit all of the changes from the merge and conflict resolution:

[1]$ cvs commit -m "some helpful log message" example
p

where some helpful log message is a helpful log message that indicates what's being committed (e.g. what branch, what feature, etc.).

Then, make a new tag to indicate a stable revision of the staging server:

[2]$ cvs rtag -r HEAD staging-YYYYMMDD-HHMM example

where YYYYMMDD is the current (4-digit) year, (2-digit) month, and (2-digit) day, and HHMM is the current (2-digit) hour and (2-digit) minute. HH should range from 00 to 23.

Next, the staging-current tag needs to be applied to this newly tagged stable revision of the staging server. This is done by first removing the old staging-current tag:

[3]$ cvs rtag -d staging-current example

and then applying the tag to the new stable revision:

[4]$ cvs rtag -r staging-YYYYMMDD-HHMM staging-current example

where staging-YYYYMMDD-HHMM is the same daily-revision tag used in step 2.

Note that instead of removing the staging-current tag with cvs rtag -d and then re-applying it with cvs rtag -r, the staging server manager could just force a new application of the tag with cvs rtag -F. However, using -d keeps the staging-current tag closer to the top of the list of tags on a file, making it easier to find. However, this has the drawback of causing problems for developers who attempt to make a new branch between the time that the staging-current tag is removed and when it is re-created.

Copying Content To The Live Servers

First, a tag is applied to the current version of the site on the staging server that marks what will be copied:

[1]$ cvs rtag -r staging-current live-YYYYMMDD-HHMM example

where YYYYMMDD-HHMM is a daily-revision in the same style as the staging-YYYYMMDD-HHMM tags.

Then, this new tag is passed as an argument to the shoveling program:

[2]$ shovel.pl --tag=live-YYYYMMDD-HHMM

This program will copy files tagged with live-YYYYMMDD-HHMM to the live web server(s). It will only copy files that have changed since the last time they were shoveled. This program will output each file that gets copied to each server. Pay close attention to the output -- if something goes wrong with the shoveling, the operation may abort partway through, but some files will already have been copied to the live server. Make sure that each file gets copied to each server. If you run the same shovel.pl --tag=live-YYYYMMDD-HHMM command more than once, nothing will break -- files that have already been successfully copied won't be recopied.