Helpful resources

Alex recently launched a DVD course on WordPress security that is available for FREE + shipping
Stop – I know what you are thinking – FREE + Shipping these days normally comes with lots of strings attached, forced continuity often hidden etc. Whilst Alex does cross-sell a few related products, the main offer is genuinely free.

Michael VanDeMar has a useful plugin to lock down your login process

SEO Egg Head offers a WordPress firewall

Donna has a useful script for monitoring your files

Of course you should also keep backups which you have total control over – this includes both database and files and you shouldn’t rely on claims that your webhost has a backup. With a VPS I find being able to “roll back” to a previous version useful, but backup with shared hosting plans supposedly made by admins isn’t a solution when you need to fix things in minutes.

Keep WordPress up to date, plugins up to date etc

Part of security is controlling what bots can crawl and index on your site, so some pamphlets would be useful as well

Getting URLs outta Google – the good, the popular, and the definitive way
Handling Google’s neat X-Robots-Tag – Sending REP header tags with PHP

Nasty Bots & Users

A lot of security relies on identifying nasty bots, detecting rogue activity such as failed logins or preventing access to all but approved users using an additional layer of password protection, or only allowing access to a server from a specific IP or range of IP addresses.

Also it is important to realise that different WordPress implementations require different levels of access control. With WordPress frequently being used for membership sites, you need to allow access to members. This reduces the number of security options available.

SEO Benefits

Lots of the pages you want to block from being crawled for security purposes also need to somehow be blocked or removed from indexation for SEO purposes, so tightening up security using the right methods will have natural SEO benefits.

Robots.txt isn’t the best option because you end up with lots of blocked pages appearing in search results and potentially indexed instead of pages you want in the index. As Sebastian explained, you have to let the bots in to crawl a URL before you can redirect them.
Not all bots can be identified, and not all bots obey robots.txt, though you can trap the naughty ones. If you are serious about your bot control you might also consider Fantomasters Searchbot Database.

User Agent Access Control For Total Lockdown

Lots of security and SEO methods rely on identifying various bots and kicking them somewhere else with 301 redirects, or denying them access to areas they are not wanted.

Far better would be to only allow access to one specific user agent, and globally kick out anything that doesn’t match – this is the user agent equivalent to restricting access to only a single IP address.

But how could this be achieved?

Many SEOs would already be familiar with User Agent Switcher for Firefox. This allows you to wander around the web pretending to be someone else, or something else such as Googlebot.

Unfortunately User Agent Switcher has a nasty problem – you often forget you have it switched to something different and then suddenly realise when a website starts misbehaving, refusing you entry, redirecting you to funny places etc.

If you created a custom user agent for security purposes, it wouldn’t be very secure if there was a chance you could broadcast it to lots of other webmasters by mistake. It is bad enough that user agent is broadcast “in the clear” unless you use SSL connections.

Then I came across an article discussing how to fake your user agent specifically for itunes but not other sites.

The Header Control Firefox plugin allows you to set your User Agent specific to a domain.

This would allow you to set a specific unique or relatively obscure user agent, and for it to only be used when accessing your own websites.

Even more useful this can be set up in multiple locations, work with variable IPs etc.

Experimental

This is something I am still experimenting with – I haven’t decided whether it is best to use .htaccess, php or a combination of both, and I am convinced the best option is to 301 redirect everything rather than deny access.
The best option might be to use a combination htaccess > php so you can do some enhanced logging.

The user agent doesn’t have to be unique, it could just be an obscure out of date version of Firefox or Chrome.

Example .htaccess to deny access

RewriteEngine on
#
RewriteCond %{HTTP_user_agent} !^RareUserAgent
RewriteRule .* - [F,L]
#

Example .htaccess to 301 redirect

RewriteEngine on
#
RewriteCond %{HTTP_user_agent} !^RareUserAgent
RewriteRule ^ http://WhereIWantPagerank.com/MyMoneyPage/ [R=301,L]
#

What I haven’t included are rewrite conditions based on specific paths as I haven’t worked out exactly what paths I need to block whilst using specific WordPress Membership Plugins.

Warning 1 – always have backups
Warning 2 – you can majorly mess up access to your website with htaccess it you get it wrong and can’t restore a working version

Disclaimer/License: GNU FDL – run with it, make it useful

It is quite likely that you have been developing other articles on similar topics for months, receiving very little online attention, and even more worrying, if you create follow-on articles providing important updates, they are less likely to be seen.

The majority of traffic will by default enter your site (the landing page) on the page that received the most links, and this traffic might continue for days, weeks, months and even years.

Ways to highlight other important information:-

• Create a new post to inform your subscribers

  This has a tendency of alienating at least some of your readers, especially those who were not totally convinced by your “blog storm” article. It is quite possible that further articles will be looked on as “milking it”, trying to take advantage of a situation.
  Whilst this might be partially true, the process of providing updated information when/if you are the centre of attention is vital. This is how CNN catapulted into the mainstream, being on the scene of major news stories and providing “up to the minute” news updates.

• Related posts

  Useful for the few people who go to the trouble of clicking them though if they are generated automatically, they can be a little hard to control.
  The problem is that very few people tend to use them, and they generally appear after someone has read the “blog storm” article. Initial reaction will be to the first article they read, and not to any updates, unless you can force them to read updates.

• Update the article with links to newer information

  This is quite a time intensive operation because over the period of a few days you might have to make multiple updates to multiple articles, and when the blogstorm has died down, you might need to optimize the links even further.

  • If all you do is create updates to a single post, whilst new visitors receive a relatively clear picture, your subscribers might only read the original article.
  • It is much easier in the flow of content creation to refer back to previous articles than to update previous articles with links to newer information.
  • If updates are drawn out over weeks or months, it can get very messy

The Simple Solution

One simple solution is to think of any news item as a series of posts, and to use a plugin designed to help you create a series of articles around a particular theme.

The In Series Plugin

The plugin is quite well documented, and will allow you to modify the order in which posts in a series are presented, and you can style the content such that it stands out.

The Advanced Solution

The ideal situation in many cases is to create a specific landing page for a series of articles, that can then be optimized for specific terms, and used to channel both humans and search engines towards the most important articles you want them to see.

This is actually fairly easy to achieve in a number of ways, and most of the skills are similar to the various ways you can perform siloing I described in my WordPress SEO Masterclass.

Here are the basic tasks that need to be undertaken:-

• How to create an optimized landing page

  There are a number of ways to create an optimized landing page. The ideal method will really depend on your existing site structure and the ways you currently highlight content.
  An additional concern is certainly your technical ability. It is much easier to create a round-up post or a page with related links than to create landing pages using more automated methods and get the page looking right.

  • Using a dedicated category with optimized template such as category-6.php in WordPress
  • Creating a dedicated page with manually selected links
  • Using a siloing plugin which presents posts from a particular category on a single page
  • Writing an update post with links to each of the previous articles

• Planning Content

  Work out which articles to link to from a landing page, keyword strategy, and linking structure both to the existing content on the topic, and to other pages you want to get a lot of link juice and attention.

• Planning Redirects

  Not all of this can be automated unfortunately. If you have made specific references to any of the articles that you are about to create redirects for, you need to make a note of them, because these might need to be adjusted so that they contain updated URLs.
  It is hard to do this with incoming links from external sources, and not always desired, but I think that internally where you might often have referenced 4 different articles from within an update page, it is best that those links remain pointing to the specific articles referenced, thus will need to be hand edited.

  Create a table of original URLs, and the updated URL which will soon house that article.

• Create Updated .htaccess

  Create… don’t upload yet

  Here is an example .htaccess entry

  <IfModule mod_rewrite.c>
  RewriteEngine On
  redirect 301 /2007/01/day-job-killer-review.html http://andybeard.eu/2007/02/day-job-killer-review.html
  </IfModule>
  

  In that particular example I was quite lazy and brought a review quickly to my front page by changing the publish date, but I also added a redirect because Google had indexed the old URL.

  I would suggest that these changes should be made right at the end of your .htaccess file.

  It is possible to manage 301 redirects with plugins, and WordPress 2.3+ is also meant to handle some things automatically, but I haven’t experimented with that yet and I don’t like being locked into using particular plugins for my site to function.

• Create Landing Page

  Depending on the method you will be using this may or may not include updated URLs automatically. If you are crafting the links by hand, you may need to refer to your previously prepared table of changes.
  Until the redirects take place, the new landing page will get very little attention from Search Engines or visitors, and whilst it is not ideal to have currently broken links on a page, it is probably better to have a few links that are broken than have lots of visitors get redirected to a page that doesn’t exist.

• Upload .htaccess or Modify Existing URLs

  This stage speed is of the essence, as it is a bit like a chicken and the egg scenario. If you are working with a large site you might need to work out some way to automate this process.

  If you upload the .htaccess first, then visitors will arrive at your landing page, and either click on links that return a 404 page not found error, or in the case of using categories or silo plugins, they will click on links that redirect to where they currently are on the landing page.
  However this is probably better than changing URLs before the .htaccess is in place.

  Once you have uploaded the .htaccess, it is time to modify existing URLs to those you have planned to use.

  I would suggest that “time is of the essence” at this stage, it is not something you can undertake at the end of a working day, though “more haste, less speed” should also be taken into consideration.

  Even with extensive planning, it is fairly easy to mess something up in your linking structure and 301 redirects.

This isn’t an article for SEO beginners, I leave those for the “experts” to write.

Your mileage may vary – I have written this article mainly as part of my own planning stage to make similar changes on this blog.
I have a number of topics that could benefit from using this method, including WordPress SEO, PageRank, Dofollow, Technorati, and even my “about page” which could take advantage of many of the blogging memes.

How much benefit you might gain from this may be marginal from an SEO point of view, or could make a substantial difference.

Fortunately the primary reason for doing this in many cases is to improve the browsing experience for users, so that they arrive at a landing page that provides them with a current overview of a topic, with possibly additional background articles that you feel are important.

There is a lot more to linking structure on a website or blog than just sticking nofollow on a few links to pages of less importance, or installing a wonder “do everything” SEO plugin.

Power Tip – once you get comfortable with this, you can actually plan your content with this strategy in mind, choosing your page titles and URLs carefully to maximise the benefit of redirects in the future.

In many ways this technique is the opposite to 3rd level push, though the concepts are not mutually exclusive, as whilst you are diverting link juice from a 3rd level document to one on the second tier, that juice then flows evenly (if you want) to your 3rd level.

Optimize your site for users not search engines ;) [cough]

In Search of the Ultimate Htaccess file

A couple of months ago Alister Cameron posted a simple solution to .htaccess such that you didn’t need to use a plugin to convert URLs using www to URLs without.

At the time I suggested a couple of improvements, and also mentioned I would post about it here on my blog, hopefully to help develop what could be looked on as the “Ultimate” .htaccess file for WordPress, something you could just drop in your root folder and be done with it.
For me the inclination was for multiple niche websites using WordPress as a CMS, so I really wanted to avoid anything that would make the content look dated.

Lets start off with the default .htaccess for WordPress once you turn on mod_rewrite for SEO friendly URLs

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

The first thing we want to do is get rid of the WWW if someone uses it. I know there are 2 schools of thought on whether URLs should have www by default or not, I prefer without and never type www unless I can’t access another site without it (broken htaccess).

Secondly we also want to get rid of trailing slash problems

The base rules that Alister first suggested were

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www.alistercameron.com$ [NC]
RewriteRule ^(.*)$ http://www.alistercameron.com/$1 [R=301,L]

RewriteCond %{REQUEST_URI} ^/[^\.]+[^/]$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1/ [R=301,L]
</IfModule>

However we want this to be the Ultimate htaccess code, thus we don’t want to have to enter the domain name. I am not sure whether this will work if you have multiple blogs in sub-folders.

In this code we are using HTTP_HOST rather than adding a URL manually to every .htaccess file you create. If you are setting up 50 blogs (niche marketers do things like this, and fill them with unique original content – not everyone creates splogs) then being able to use one default file is a major advantage.

# If subdomain www exists, remove it first
RewriteCond %{HTTP_HOST} ^www\.([^\.]+\.[^\.]+)$ [NC]
RewriteRule ^(.*)$ http://%1/$1 [R=301,L]

Can we improve on the trailing slashes code?

Possibly…

A while ago I was also reading a post over on Aaron Walls SEO Book blog. Within the comments were suggestions with improvements to the code Aaron suggested.
Finding the exact reference is a problem as it wasn’t on this thread

Searching on a phrase in the code these days only brings up a reference on Alister’s blog where I mentioned it in the comments, so I have no idea who to attribute this htaccess code to.

# If requested resource does not exist as a file
RewriteCond %{REQUEST_FILENAME} !-f
# and does not end with a period followed by a filetype
RewriteCond %{REQUEST_URI} !..+$
# and does not end with a slash
RewriteCond %{REQUEST_URI} !/$
# then add a trailing slash and redirect
RewriteRule (.*) $1/ [R=301,L]
</IfModule>

I am not a htaccess guru, but this seems to take into account more potential situations such as files to download.

If you put all this code together you end up with something like this

<IfModule mod_rewrite.c>
RewriteEngine On
# If subdomain www exists, remove it first
RewriteCond %{HTTP_HOST} ^www\.([^\.]+\.[^\.]+)$ [NC]
RewriteRule ^(.*)$ http://%1/$1 [R=301,L]

# If requested resource does not exist as a file
RewriteCond %{REQUEST_FILENAME} !-f
# and does not end with a period followed by a filetype
RewriteCond %{REQUEST_URI} !..+$
# and does not end with a slash
RewriteCond %{REQUEST_URI} !/$
# then add a trailing slash and redirect
RewriteRule (.*) $1/ [R=301,L]
</IfModule>

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

So can anyone offer any constructive improvements?

If you offer improvements, please provide code samples and explain exactly why it is an improvement so people can learn from it (as I said I am a newbie at this)
Code can be entered using code tags in square brackets.