Helpful resources

Alex recently launched a DVD course on WordPress security that is available for FREE + shipping
Stop – I know what you are thinking – FREE + Shipping these days normally comes with lots of strings attached, forced continuity often hidden etc. Whilst Alex does cross-sell a few related products, the main offer is genuinely free.

Michael VanDeMar has a useful plugin to lock down your login process

SEO Egg Head offers a WordPress firewall

Donna has a useful script for monitoring your files

Of course you should also keep backups which you have total control over – this includes both database and files and you shouldn’t rely on claims that your webhost has a backup. With a VPS I find being able to “roll back” to a previous version useful, but backup with shared hosting plans supposedly made by admins isn’t a solution when you need to fix things in minutes.

Keep WordPress up to date, plugins up to date etc

Part of security is controlling what bots can crawl and index on your site, so some pamphlets would be useful as well

Getting URLs outta Google – the good, the popular, and the definitive way
Handling Google’s neat X-Robots-Tag – Sending REP header tags with PHP

Nasty Bots & Users

A lot of security relies on identifying nasty bots, detecting rogue activity such as failed logins or preventing access to all but approved users using an additional layer of password protection, or only allowing access to a server from a specific IP or range of IP addresses.

Also it is important to realise that different WordPress implementations require different levels of access control. With WordPress frequently being used for membership sites, you need to allow access to members. This reduces the number of security options available.

SEO Benefits

Lots of the pages you want to block from being crawled for security purposes also need to somehow be blocked or removed from indexation for SEO purposes, so tightening up security using the right methods will have natural SEO benefits.

Robots.txt isn’t the best option because you end up with lots of blocked pages appearing in search results and potentially indexed instead of pages you want in the index. As Sebastian explained, you have to let the bots in to crawl a URL before you can redirect them.
Not all bots can be identified, and not all bots obey robots.txt, though you can trap the naughty ones. If you are serious about your bot control you might also consider Fantomasters Searchbot Database.

User Agent Access Control For Total Lockdown

Lots of security and SEO methods rely on identifying various bots and kicking them somewhere else with 301 redirects, or denying them access to areas they are not wanted.

Far better would be to only allow access to one specific user agent, and globally kick out anything that doesn’t match – this is the user agent equivalent to restricting access to only a single IP address.

But how could this be achieved?

Many SEOs would already be familiar with User Agent Switcher for Firefox. This allows you to wander around the web pretending to be someone else, or something else such as Googlebot.

Unfortunately User Agent Switcher has a nasty problem – you often forget you have it switched to something different and then suddenly realise when a website starts misbehaving, refusing you entry, redirecting you to funny places etc.

If you created a custom user agent for security purposes, it wouldn’t be very secure if there was a chance you could broadcast it to lots of other webmasters by mistake. It is bad enough that user agent is broadcast “in the clear” unless you use SSL connections.

Then I came across an article discussing how to fake your user agent specifically for itunes but not other sites.

The Header Control Firefox plugin allows you to set your User Agent specific to a domain.

This would allow you to set a specific unique or relatively obscure user agent, and for it to only be used when accessing your own websites.

Even more useful this can be set up in multiple locations, work with variable IPs etc.

Experimental

This is something I am still experimenting with – I haven’t decided whether it is best to use .htaccess, php or a combination of both, and I am convinced the best option is to 301 redirect everything rather than deny access.
The best option might be to use a combination htaccess > php so you can do some enhanced logging.

The user agent doesn’t have to be unique, it could just be an obscure out of date version of Firefox or Chrome.

Example .htaccess to deny access

RewriteEngine on
#
RewriteCond %{HTTP_user_agent} !^RareUserAgent
RewriteRule .* - [F,L]
#

Example .htaccess to 301 redirect

RewriteEngine on
#
RewriteCond %{HTTP_user_agent} !^RareUserAgent
RewriteRule ^ http://WhereIWantPagerank.com/MyMoneyPage/ [R=301,L]
#

What I haven’t included are rewrite conditions based on specific paths as I haven’t worked out exactly what paths I need to block whilst using specific WordPress Membership Plugins.

Warning 1 – always have backups
Warning 2 – you can majorly mess up access to your website with htaccess it you get it wrong and can’t restore a working version

Disclaimer/License: GNU FDL – run with it, make it useful

I was greeted with a nice message from Firefox in the morning

Firefox's Warning That A Site Might Be Harmful

If someone was searching Google and came across one of my results, there was a clear warning that my site was dangerous, and if they clicked through on a result they would be greeted with this.

Googles Hacked Warning From Search Results

What Affect Does This Have On Search Traffic

What do you think? Kills it dead…

Search Traffic Killed By Security Warnings

Over 90% of Google search traffic was wiped out

How Dare They Do That

The first reaction by many people is probably shock, horror, outrage… I mean how dare they take away all that free traffic.

My first reaction was to upload a new index.php file that shut off the blog, gave a warning, and a 503 header (that I checked to make sure that it was being sent correctly)

I don’t want anyone to suffer from visiting my site due to injected iframes for suspicious sites injected into my pages.

I renamed my existing index.php

I then uploaded a new index.php with the following code

<?php
ob_start();
header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Status: 503 Service Temporarily Unavailable');
header('Retry-After: 3600');
header('X-Powered-By:');
?><!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>Cleaning up a hacked server, might be a while</p>
</body></html>

This wasn’t a total cure… there were a few extreme situations where this wouldn’t have been effective such as a few static files, but it was a very good fast measure, and the next step would probably have been to use htaccess to redirect all traffic to that page that was outside WordPress.

Fixing Hacked WordPress Installation

Lorelle has a great recent compilation of how to diagnose a hacked WordPress blog, and how to fix it.

I determined that what happened to my site wasn’t the new worm based attack, and that my database wasn’t affected.

Here is the procedure I used:-

1. Backed up database – I already have backups sent daily to Gmail – I don’t store backups using WordPress plugins for S3 etc as these can actually introduce another attack vector.
2. Backed up server image – one of the advantages of using VPS Hosting is often the ability to create an immediate snapshot of the whole server.
3. Rolled back to previous server snapshot – I have daily, weekly and monthly snapshots of the whole server backed up, and the best option was the weekly one taken Sunday night.
4. Uploaded backed up database to server
5. Restored database using simple mysql command via terminal
6. Tons of passwords changed

The WordPress export is a useful alternative to what I did, but just imagine using that cleanup method suggested by Lorelle if you were running a busy membership site using WordPress, or have lots of SEO and other special tweaks not supported by WordPress export.
Ultimately solutions for cleaning a database would be a lot more appropriate.

Google Reinclusion Request?

The final step is a reinclusion request with Google which in theory might take 4 or 5 days for them to take a look at, but here is an interesting chain of events.

I filed a reinclusion request with Google possibly 8 hours after I discovered I had been hacked – I was a little busy with other offline events so fixing server took a little more time than I would otherwise expect.

Some feedback for the Google Webmaster team if they read this

• When you file a reinclusion request, currently a copy of what you send Google is not CCed back to you, even in the webmaster interface
• The form for filing reinclusion requests has some very wierd scrolling/focusing events going on, so it is impossible to use when filing a long request with the detailed information asked.
• 6 hours after I filed my reinclusion request, I recieved notification in Webmaster Tools from the Google Search Quality Team that my website had been spotted by Google as compromised. That is at least 14 hours after it happened.

Malware notification regarding http://andybeard.eu/ September 16, 2009

Dear site owner or webmaster of http://andybeard.eu/,

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com.

Below are one or more example URLs on your site which can cause users to be infected:

http://andybeard.eu/

http://andybeard.eu/1297/

http://andybeard.eu/1298/

Here is a link to a sample warning page: http://www.google.com/interstitial?url=http://andybeard.eu/

We strongly encourage you to investigate this immediately to protect your visitors. Although some sites intentionally distribute malicious software, in many cases the webmaster is unaware because:

1) the site was compromised

2) the site doesn’t monitor for malicious user-contributed content

3) the site displays content from an ad network that has a malicious advertiser

If your site was compromised, it’s important to not only remove the malicious (and usually hidden) content from your pages, but to also identify and fix the vulnerability. We suggest contacting your hosting provider if you are unsure of how to proceed. StopBadware also has a resource page for securing compromised sites: http://www.stopbadware.org/home/security

Once you’ve secured your site, you can request that the warning be removed by visiting this Webmaster Help Center article and requesting a review. If your site is no longer harmful to users, we will remove the warning.

Sincerely,

Google Search Quality Team

You would think if Google are notifying Stopbadware.org who in turn are notifying other sites such as I noticed Tweetmeme blocked my site very quickly, that somehow the notification to the webmaster would be listed in Webmaster tools at around the same time.
I can understand 15mins difference, even an hour, but attentive webmasters are going to have their servers rectified before this notification is sent.

You Might Not Need The Reinclusion Request Any More

Better safe than sorry, but here is the normal series of events for a reinclusion request.

1. You file it and get sent a message that you filed it (without the details of what you sent attached)

Reconsideration request for http://andybeard.eu/ {Date}

We’ve received a request from a site owner to reconsider how we index the following site: http://andybeard.eu/

We’ll review the site. If we find that it’s no longer in violation of our Webmaster Guidelines, we’ll reconsider our indexing of the site. Please allow several weeks for the reconsideration request. We do review all requests, but unfortunately we can’t reply individually to each request.

2. Some time passes
3. Google send you notification that they have looked at the request

We’ve processed your reconsideration request for http://andybeard.eu/ {date}

We received a request from a site owner to reconsider how we index the following site: http://andybeard.eu/.

We’ve now reviewed your site. When we review a site, we check to see if it’s in violation of our Webmaster Guidelines. If we don’t find any problems, we’ll reconsider our indexing of your site. If your site still doesn’t appear in our search results, check our Help Center for steps you can take.

But I am still at the “some time passes” stage, which I honestly expected to last 4-5 days, and felt was quite reasonable – even Google don’t have unlimited resources.

• I filed my reinclusion request sometime around 9am PST (6pm CET) Wednesday
• By 10am CET Thursday my site was no longer being blocked.

This is somehow now being automated.

There wasn’t any exceptional crawl activity, but Google average crawling over 800 pages of my site every day anyway. A big enough cross-section to detect anything unusual.

Other than the feedback items I noted above, Google are doing a great job with handling hacked sites, at least based upon the experience I have just undergone.

Effectively 24hrs from seeing my site being blocked, fixing the hacked site, notifying Google, and finally having my site back without horrible security warnings everywhere is amazingly efficient.
The data seems to have also been pushed out to those that use it extremely fast, as all my Tweetmeme buttons are already active again.

This post hopefully can act as a bit of a counter-balance to all the stories of dread you will find if searching of how hard it is to fix a site after it has been hacked, and what it takes to get Google to reconsider your site after it has happened.

Bravo Google Webmaster and Search Quality team & Stopbadware