Checking over the logs it became obvious that queries using SQL_CALC_FOUND_ROWS are a problem. Digging through code it appears that where a limit on results is used it uses this query modifier to allow simpler coding of paging. The problem is that since the main query to grab blog posts on the blog home page or any archive page uses this, on a large blog this becomes quite inefficient.

For now I have tweaked MySQL so that the MySQL Query Cache is now larger. These articles were useful:
MySQL Query Cache (at MySQL Performance Blog)
Turn on MySQL query cache to speed up query performance?

Looking at the log after this change query times have dropped dramatically (from over 0.5 seconds and up to milliseconds). This has immediately improved performance of the blog and overall load on the server.

I am currently looking at some WP Caching API and plugins to further improve performance.

If you are looking for some more ways to speed up WordPress check out this article as a starting point:
4+1 Ways To Speed Up WordPress With Caching

And … for myself – just so I remember:
Install Memcache onto cPanel running CentOS

Hopefully this saves someone the few minutes it took for me to find this – pretty much everything you need to know!

The easy (and fast) solution proved to be modifying the Nucleus RSS feed so that it exports all the blog posts in full. These can then be imported into WordPress.

Steps in brief (this might be different depending on your Nucleus version):

1. Edit the skin for xmlrss2 in Nucleus. Find this (or similar) <%blog(xmlrss2,10)%> and change it to <%blog(xmlrss2,1000)%> where the second number is larger than the number of posts in your blog.
2. Edit the template for xmlrss2 in Nucleus. It will probably look something like this:

   <item><title><%syndicate_title%></title>
    <link><%blogurl%>index.php?itemid=<%itemid%></link>
   <description><%syndicate_description%></description>
    <category><%category%></category>
    <pubDate><%date(rfc822)%></pubDate>
   </item>

   Change it to:

   <item>
    <title><%syndicate_title%></title>
    <link><%blogurl%>index.php?itemid=<%itemid%></link>
   <content:encoded><![CDATA[<%body%>
   ]]></content:encoded>
    <category><%category%></category>
    <pubDate><%date(rfc822)%></pubDate>
   </item>

3. Open the Nucleus RSS feed in a browser (it should be really long and include the content of all your blog posts). Save the source (nucleusexport.xml for example).
4. Reset the changes to the Nucleus templates and skins above so your feed is back to normal.
5. Import the RSS feed into your WordPress site (this used to be under the Manage tab, it is under Tools now).

Note: this won’t migrate comments and maybe other stuff that is important to you. This is purely the post content.

If you don’t have access to mod_deflate, i.e. if you are on a uncooperative shared host, you can use this method using .htaccess …
Speed Up Your Blog Using ZLIB

If that doesn’t work you may be on a phpSuExec server. Create a file called php.ini in the root of your WordPress install with the following contents …

zlib.output_compression = On

You can test here:
http://www.gidnetwork.com/tools/gzip-test.php

I had to look at the WordPress core code to work this out – thought I was going nuts. If you apply new rewrite rules using the rewrite_rules_array filter you need to ensure that you refresh the rewrite rules that WordPress has stored in a hidden option. You do this by simply resaving the Permalink Structure under the Options menu in the admin area of your WP install.

Not immediately obvious that this would be required, but makes sense that they generate the rules and store them at this point.

What is even better is there are plugins for CMS software, including this WordPress Pagepeel Plugin. Very easy to install and configure. Just upload a 100×100 image and a 500×500 image for the initial peel and the full display and set a URL.

Works well. Recommended.

Hint: if you want to use this for a private forum for your business check out these plugins …
Force Login to view the forum
Restrict Registration – only allow registration from people with an email address at particular domains (i.e. yourcompany.com)

for anyone who doesn’t know where to do that … the “Header Image and Color” link is under your presentation tab.