I’ll be speaking at WordCamp Austin later this month. My session is “The Query, the Whole Query, and nothing but the Query”. It will include everything you need to know about the WP_Query class and how to properly query the WordPress database for posts. I look forward to meeting some new people and seeing old friends. I’ll be updating this post with slides and additional code samples. Please comment if you have any questions or suggestions.
WordPress lead developer, Mark Jaquith, recently gave a talk at WordCamp San Francisco on Scaling, Servers, and Deploys that including some great advice on professional WordPress development using Git. Git is a free & open source, distributed version control system designed to handle everything from small to very large projects with speed and efficiency.
The session from WordCamp San Francisco:
The purpose of this guide is to show you how you can add Git to your WordPress development workflow and will include the basics of securely installing Git on a remote server, using your local machine for development and deploying your changes to a live production server using SSH.
This article assumes:
Step 1: Installing Git on the remote server
Install Git:
apt-get install git-core
Adding the git user. (You will be asked to enter a password for the git user. We won’t be using it but remember what it is just in case.)
adduser git
Setup SSH keys for the new git user. If you still have access to the public key you use to access your server you can add it to /home/git/.ssh/authorized_keys. If not you will need to generate a new public/private key pair and add the public key to your server. See my previous article WordPress Performance Server – Debian “squeeze” with Nginx, APC and PHP from the Dotdeb repos for more info on setting up SSH keys on your server. You can also use the ssh-keygen tutorial on GitHub.
su git cd /home/git mkdir .ssh chmod 700 .ssh cd .ssh touch authorized_keys cat id_dsa.pub >> authorized_keys exit
If you need to switch back to your normal user to add your public key make sure you chown the /home/git/.ssh back to the git user by running:
chown git:git /home/git/ -R
Now lets test our SSH connection using the git username on the remote server. If you use lots of different SSH keys for different servers it helps if you add instructions to your config file to identify the right key to use when logging in.
On your local machine create if needed and open the file ~/.ssh/config
Host remote_server_domain.com Hostname remote_server_domain.com IdentityFile ~/.ssh/id_dsa //Your private key that matches the public key we installed on the server User git
Now lets test our connection to the remote server
ssh git@remote_server_domain.com
If your able to connect let’s go ahead and create and initialize our remote Git repository. We’re calling this repo “project1″.
cd /home/git mkdir project1 cd project1 git --bare init exit
You just created your first remote Git repository!
For security we are going to restrict shell access for the git user. This will allow you to share development with other users by giving them SSH access to the repo without giving them shell access to your server.
SSH back into your server with the user with su privileges.
nano /etc/passwd
When you open the file find the line for the new git user we just added and change the end of the line from /bin/bash to /usr/bin/git-shell. (Do not change the user Id or group ID). It should look like this when your done:
git:x:1001:1001:Git,,,:/home/git:/usr/bin/git-shell
Test the new config by exiting and trying again to SSH into the server with the git user:
ssh git@remote_server_domain.com fatal: What do you think I am? A shell? connection to [remote_server_domain.com] closed.
Now we have a secure remote repository.
Step 2: Installing Git and setting up your local machine
Download and install the latest version for Mac (For Lion or Snow Leopard choose: Git Installer 1.7.6 – OS X – Snow Leopard – x86_64)
Making your first commit.
Now that we have our remote repo setup and Git installed on our local machine let’s make our first commit.
mkdir project1 cd project1 git init touch readme.txt git add . git commit -m "Initial commit - added readme.txt"
The first time we use our new repo we need to tell Git where to find it.
git remote add origin ssh://git@remote_server_domain.com/home/git/project1 git push origin master
Now we are going to delete the project1 directory we created on our local machine and check out the repo so we can start doing some WordPress development. I use the ~/sites directory for my local development so lets check the new project out into that directory. If you haven’t set up your local environment using hostname aliases in the ~/sites directory check out this tutorial on WPCandy: How to improve local WordPress development on a Mac
rm -rf project1 mkdir ~/sites/wp.dev cd wp.dev git clone ssh://git@remote_server_domain.com/home/git/project1
We now have checked out the latest version of our repo (currently only containing the readme.txt file) and are ready to start working but first we need to decide our version control strategy.
Next Page – Step 3: Setting up your repo for WordPress theme and plugin development
This is a step by step guide on how to set up your own unmanaged server or vps to run as a WordPress Performance server environment. Using this setup you will have the ability to host a large WordPress Multisite set up or numerous single installs. This guide assumes you have a basic understanding of using SSH via the Mac Terminal app or Putty for Windows and requires an unmanaged dedicated server or VPS hosting account with either the 32 or 64 bit Debian kernal available.
In my last guide I touted a Nginx reverse proxy cache with WordPress and Apache as the ultimate WordPress performance stack but since then I have found that there is really no reason to use Apache unless you have a specific need for it. Nginx has matured and works great with the WordPress rewrite engine and even W3 Total Cache now includes Nginx rewrite rules for page cache, minify and browser caching.
Choosing a web host with Debian “squeeze” images available
Debian 6 aka “squeeze” is considered the grandfather of Linux distors and Debian is known for relatively strict adherence to the Unix and free software philosophies. Debian is also distributed with access to repositories containing thousands of software packages ready for installation and use.
Most of the large managed hosting companies don’t give you the option of choosing your Linux distro so your going to need to find one that does. The reason for this is that it’s much easier for a hosting company to provide support using cPanel or Parallels Virtuozzo VPS virtualization. Virtuozzo allows them to squeeze many more VPSs on a box and control the resources and your limited to guess what – Cent OS.
Your going to need a provider that offers Xen virtualization. Xen is a powerful open source virtualization platform that supports a wide range of guest operating systems including Linux – Debian, Ubuntu, Free BSD, Cent OS, Fedora and more. When you set up your account have your provider load the Debian 6 aka “squeeze” image on your VPS or install it on your dedicated server. If you have over 2GB memory go for the 64 bit version.
I use Softlayer (formally The Planet) for my dedicated server because of the state of the art data centers and major broadband backbone it’s connected to. It also has a very powerful back end portal and allows you to connect to your box over a private VPN. Linode, Slicehost, and VPS.net are also very good choices. I would stay away from Media Temple and Rackspace as I’ve had problems with both and they’ve had well documented security vulnerabilities in the past.
Next Page: Connecting to your server the first time.
This post is in response to a write up entitled “Open Source Motivations” by long time WordPress developer, contributor and plugin author, Alex King.
Alex runs a successful and well known software and development consultancy firm, Crowd Favorite, along with WordPress Help Center, a specialized WordPress support center.
In the article Alex answers some tough questions about the motivations, financial gain (or lack of), and indirect financial benefits of writing, maintaining and supporting “free” WordPress Plugins. Alex’s response to the questions along with some of the reactions on Twitter suggest that the current situation is unsustainable for developers.
I disagree with this
I actually feel strongly that the current situation is unsustainable. Unless the WordPress community at large starts to better recognize and reward the developers that create the tools that they use and rely on, the developers won’t/can’t continue to provide as they have.
Why it might not be financially beneficial for large shops like his to continue developing and supporting free themes and plugins, it is very beneficial to individual developers like myself, and others whose release of free plugins and themes have brought them great financial gain.
How do you think developers like Alex and others got to where they are today?
I strongly believe that part of the success of Crowd Favorite was due to the contributions that were made to b2 and to the WordPress community. Others like Jason Schuller, Brian Gardner and Cory Miller attributed part of the success of their Premium Theme businesses to releasing free themes to the community. You can listen to the interviews and discussion about this very topic on Jeff Chandler’s WordPress Weekly episode 94.
Contributing to the WordPress community is a way for smaller developers to make a name for themselves, get experience and prove to potential clients that they know what their doing.
WordPress.org gives you the opportunity to put your product in front of millions of people. Currently I have only released one plugin, WP Coda Slider, on WordPress.org. My plugin is pretty simple and the only options available for it have to be put inside of a shortcode or through the use of a template tag but the response I have gotten has been great. Hosting the demo of and providing support for the plugin on WP-Performance gets me around 250 – 500 visitors a day.
In fact my simple little plugin has brought 8,267 visitors to my site since I released it back in September.
Website traffic may not be that big of a deal for the well known players in the WordPress development community but it has offered me the opportunity to expose my WordPress support and development services to a great number of people. In the last three months I have gotten around 4 or 5 new really good clients who found me through my plugin. I also find it very rewarding to help others and I am amazed that people are actually using software I created on their websites.
One of the most troubling parts of Alex’s article is his sentiment towards the users of his products:
In talking with other plugin developers, it seems fairly universal that the reward for a successful plugin is a deluge of support email that includes the worst kind of sense of entitlement, rudeness and ignorance. The community as a whole seems to expect to be able to pay nothing, yet received expert and individual help and support for free.
I completely disagree that the community as a whole has this type of entitlement.
I am sure there are those out there who don’t understand how much work and effort developers like Alex put into a free product but I have found that a majority of the community is very appreciative and also very polite when it comes to getting support for a free plugin. Any time I have offered free help or support to other WordPress uses it has been a very pleasurable experience. See for yourself on my plugins support page. Almost every single request contains a thank you and is very polite. The same is true when I answer questions in the WordPress.org support forums or on Stack Exchanges WordPress Answers site.
I used to get about $100-200/month in the way of donations through my website. Unfortunately due to changes in the way plugins are presented on WordPress.org that has dried up to about $5/month.
Ask any Plugin author about the “donations” they have received for their work and they will laugh because donations for WordPress Plugins have almost always been non existent. If your counting on donations to make it or to feel appreciated then the “FREE” Plugin business is not for you. This is not the fault of WordPress.org’s new plugin page design this is how it will always be.
It is very understandable that busy dev shops with payrolls to make and lots of clients to keep them busy don’t have the time, energy, resources or find it rewarding enough to contribute then thats fine. There are plenty others who do and are thankful for the opportunity to contribute and without an open source community like WordPress none of you would be in business today.
Thanks for taking the time to read this and I welcome your thoughts and comments below.
UPDATE: There has been a lot of discussion on this topic including a great post by Jeff Chandler
I recently answered a question over at WordPress Answers, Stack Exchanges new WordPress question and answer site, about best practices for using a caching plugin with a shared host.
Not all WordPress users are able to host their blog or website on a private server or VPS and need a way to speed up their websites. There are a few great WordPress caching plugins like WP Super Cache and W3 Total Cache. Both plugins are very popular, well supported by their authors and do a good job of caching.
WP Super Cache is developed and supported by Donncha O Caoimh who is a member of the Automattic team and a core WordPress contributor.
W3 Total Cache was originally developed for Mashable.com by their CTO, Frederick Townes .
W3 Total Cache is much more than a caching plugin. Along with page caching, W3 also does object caching, database caching, minifies and combines css and js files and also has built in CDN (Content Delivery Network) support. This is the only plugin available that tackles all the best practices of High Performance Websites
You will only be able to statically cache the html output from your pages. This is the fastest way to serve pages but you loose the dynamic aspects of WordPress like making comments and seeing the latest comments on posts.
There are disk caching options available in W3 Total Cache for objects and database but for object caching you will only see gains if your host is running fast drives and most sites have far too many queries per page (due to poorly written plugins) to see any advantage from database disk caching.
When the plugin is first activated the recommended settings will already be in place.
This article will go through all the steps of setting up the plugin to get the best performance gains on a shared host. If your on a VPS or private server you will have a lot more features to work with like using a PHP opcode cache and caching database queries in memory.
Check all the options for page cache
Turn this on and set the update interval at what ever is appropriate for your site. This will rebuild the page cache at the given interval.
Select Rewrite url structure and if your going to use the CDN check the auto upload so newly minified files will automatically get uploaded to the CDN. 
Enable and check remove line breaks, inline js and css minification. If your using adsense or another service that uses comment stems enter them here to avoid having them minified. 
In the file management choose your theme and add any css files you want combined and minified. There is also a help wizard that will search all your templates and add the suggested files for you.
W3 Total includes a tool that goes through your theme templates and finds Javascript and CSS files that are used and provides recommended settings. First try theses settings and if problems are encountered go back and modify as needed. Any files highlighted in red are files you have already included to be minified.
The minify settings give you the option of placing the minified files in different locations. After <head>, after <body> and before </body>. It is best to put as many as you can before </body>. If any plugins add inline js you won’t be able to use before </body> for jquery or the plugins js because it will need to load before any inline <script> tags. You can include any combinations of files in each location and for each template. For instance you can set your comment-reply.js to only load on single.php
This is the most important one to get right. If you properly cache your static content in your users browsers you can drastically reduce page load times and reduce bandwidth usage.
Check everything. Make sure and check “do not process 404 errors for static objects”. browser cache settings is a big win for shared hosting because invoking PHP and returning 404 pages to bots etc is a big drain on resources and this feature prevents that.
Check everything and set expires header lifetime to far future. 31536000 seconds is 1 year and what yslow recommends. If you make changes to your css or javascript you have to change the file names to prevent users from using the old version. If your using minify you won’t have to worry about serving outdated content because W3 takes care of changing the name of the minified file for you.
Set your cache Control policy to cache with max age.
There are two more browser cache setting sections. HTMl and Images. For Images use the same settings as CSS and JS.
For HTML don’t set expires unless your site is mainly static. You can use short lifetimes if you want (180 seconds) but I wouldn’t go higher. Enable gzip and you can check “set W3 Headers” so you can check the response headers to make sure they are working.
CDN is one of the largest performance wins for any hosting scenario even if self-hosted. W3 Total has built in support for popular origin pull and origin push CDN’s and a robust self hosted option that requires you to set up subdomains and cnames. Note: Do not check force override unless new files are being overwritten. Force override will constantly re-upload files even if they already exist. This wastes bandwidth and resources.
Self hosted CDN will let you take advantage of pipelining. Browsers can only download a few files at once, only 4 in some cases. Pipelining is a technique whereby aliases (subdomains for example) of your server are used to allow your browser to increase the practical limit of files that can be downloaded in parallel. Doing so maximizes the throughput of the your internet connection and allows the browser to render a page faster. W3TC takes care of managing these files transparently once DNS CNAMEs (aliases) and subdomains are properly configured.
W3 Total Cache is one of the best supported plugins on the WordPress.org plugin repository. The author responds to all questions tagged w3-total-cache and a support tab is also included in the plugin to use for reporting bugs.
You should always test your results and tweak your settings accordingly. I like to use WebPageTest.org. To compare my results and identify any potential problems.
I recently helped a client reduce his page load times from almost 10 seconds to 3 seconds by implementing these settings on his WordPress blog.
As you can see by the waterfall image he was loading javascript before css which causes a blocking condition. Blocking is when the loading of a script or file makes all other requests wait until it is fully downloaded. He also had 50 external file requests and some 404 errors.
These results are not perfect but they are a dramatic improvement over what they were. We reduced the total time to load the page from 10 seconds to 3 seconds. By combining and removing the javascript to the bottom we reduced the Start Render time from 3.5 seconds to .45 seconds. The start render time is very important because it indicates when the page starts becoming viewable in the browser. This progressive rendering effect gives the illusion of instant page loads because the user doesn’t notice there are still files being loaded in the background below the fold.
W3 Total Cache is the best choice for a caching plugin because it has so many more features than just caching. What has your experience been with W3 Total Cache or other web performance solutions? I would love to hear some of your best practice tips lets discuss this further in the comments.
Optimizing WordPress for Performance, my WordCamp Houston presentation.
How to add a re tweet button in an Iframe after the page loads for better performance
The Quest for Speed by Frederick Townes