How Google & StopBadware.org Handle Hacked WordPress

 

Yesterday I had the fun challenge of dealing with this blog being hacked. I live for challenges.

I was greeted with a nice message from Firefox in the morning

Firefox's Warning That A Site Might Be Harmful

Firefox's Warning That A Site Might Be Harmful

If someone was searching Google and came across one of my results, there was a clear warning that my site was dangerous, and if they clicked through on a result they would be greeted with this.

Googles Hacked Warning From Search Results

Googles Hacked Warning From Search Results

What Affect Does This Have On Search Traffic

What do you think? Kills it dead…

Search Traffic Killed By Security Warnings

Search Traffic Killed By Security Warnings

Over 90% of Google search traffic was wiped out

How Dare They Do That

The first reaction by many people is probably shock, horror, outrage… I mean how dare they take away all that free traffic.

My first reaction was to upload a new index.php file that shut off the blog, gave a warning, and a 503 header (that I checked to make sure that it was being sent correctly)

I don’t want anyone to suffer from visiting my site due to injected iframes for suspicious sites injected into my pages.

I renamed my existing index.php

I then uploaded a new index.php with the following code

<?php
ob_start();
header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Status: 503 Service Temporarily Unavailable');
header('Retry-After: 3600');
header('X-Powered-By:');
?><!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>Cleaning up a hacked server, might be a while</p>
</body></html>

This wasn’t a total cure… there were a few extreme situations where this wouldn’t have been effective such as a few static files, but it was a very good fast measure, and the next step would probably have been to use htaccess to redirect all traffic to that page that was outside WordPress.

Fixing Hacked WordPress Installation

Lorelle has a great recent compilation of how to diagnose a hacked WordPress blog, and how to fix it.

I determined that what happened to my site wasn’t the new worm based attack, and that my database wasn’t affected.

Here is the procedure I used:-

  1. Backed up database – I already have backups sent daily to Gmail – I don’t store backups using WordPress plugins for S3 etc as these can actually introduce another attack vector.
  2. Backed up server image – one of the advantages of using VPS Hosting is often the ability to create an immediate snapshot of the whole server.
  3. Rolled back to previous server snapshot – I have daily, weekly and monthly snapshots of the whole server backed up, and the best option was the weekly one taken Sunday night.
  4. Uploaded backed up database to server
  5. Restored database using simple mysql command via terminal
  6. Tons of passwords changed

The WordPress export is a useful alternative to what I did, but just imagine using that cleanup method suggested by Lorelle if you were running a busy membership site using WordPress, or have lots of SEO and other special tweaks not supported by WordPress export.
Ultimately solutions for cleaning a database would be a lot more appropriate.

Google Reinclusion Request?

The final step is a reinclusion request with Google which in theory might take 4 or 5 days for them to take a look at, but here is an interesting chain of events.

I filed a reinclusion request with Google possibly 8 hours after I discovered I had been hacked – I was a little busy with other offline events so fixing server took a little more time than I would otherwise expect.

Some feedback for the Google Webmaster team if they read this

  • When you file a reinclusion request, currently a copy of what you send Google is not CCed back to you, even in the webmaster interface
  • The form for filing reinclusion requests has some very wierd scrolling/focusing events going on, so it is impossible to use when filing a long request with the detailed information asked.
  • 6 hours after I filed my reinclusion request, I recieved notification in Webmaster Tools from the Google Search Quality Team that my website had been spotted by Google as compromised. That is at least 14 hours after it happened.

Malware notification regarding http://andybeard.eu/ September 16, 2009

Dear site owner or webmaster of http://andybeard.eu/,

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com.

Below are one or more example URLs on your site which can cause users to be infected:

http://andybeard.eu/

http://andybeard.eu/1297/

http://andybeard.eu/1298/

Here is a link to a sample warning page: http://www.google.com/interstitial?url=http://andybeard.eu/

We strongly encourage you to investigate this immediately to protect your visitors. Although some sites intentionally distribute malicious software, in many cases the webmaster is unaware because:

1) the site was compromised

2) the site doesn’t monitor for malicious user-contributed content

3) the site displays content from an ad network that has a malicious advertiser

If your site was compromised, it’s important to not only remove the malicious (and usually hidden) content from your pages, but to also identify and fix the vulnerability. We suggest contacting your hosting provider if you are unsure of how to proceed. StopBadware also has a resource page for securing compromised sites: http://www.stopbadware.org/home/security

Once you’ve secured your site, you can request that the warning be removed by visiting this Webmaster Help Center article and requesting a review. If your site is no longer harmful to users, we will remove the warning.

Sincerely,

Google Search Quality Team

You would think if Google are notifying Stopbadware.org who in turn are notifying other sites such as I noticed Tweetmeme blocked my site very quickly, that somehow the notification to the webmaster would be listed in Webmaster tools at around the same time.
I can understand 15mins difference, even an hour, but attentive webmasters are going to have their servers rectified before this notification is sent.

You Might Not Need The Reinclusion Request Any More

Better safe than sorry, but here is the normal series of events for a reinclusion request.

  1. You file it and get sent a message that you filed it (without the details of what you sent attached)
  2. Reconsideration request for http://andybeard.eu/ {Date}

    We’ve received a request from a site owner to reconsider how we index the following site: http://andybeard.eu/

    We’ll review the site. If we find that it’s no longer in violation of our Webmaster Guidelines, we’ll reconsider our indexing of the site. Please allow several weeks for the reconsideration request. We do review all requests, but unfortunately we can’t reply individually to each request.

  3. Some time passes
  4. Google send you notification that they have looked at the request

We’ve processed your reconsideration request for http://andybeard.eu/ {date}

We received a request from a site owner to reconsider how we index the following site: http://andybeard.eu/.

We’ve now reviewed your site. When we review a site, we check to see if it’s in violation of our Webmaster Guidelines. If we don’t find any problems, we’ll reconsider our indexing of your site. If your site still doesn’t appear in our search results, check our Help Center for steps you can take.

But I am still at the “some time passes” stage, which I honestly expected to last 4-5 days, and felt was quite reasonable – even Google don’t have unlimited resources.

  • I filed my reinclusion request sometime around 9am PST (6pm CET) Wednesday
  • By 10am CET Thursday my site was no longer being blocked.

This is somehow now being automated.

There wasn’t any exceptional crawl activity, but Google average crawling over 800 pages of my site every day anyway. A big enough cross-section to detect anything unusual.

Other than the feedback items I noted above, Google are doing a great job with handling hacked sites, at least based upon the experience I have just undergone.

Effectively 24hrs from seeing my site being blocked, fixing the hacked site, notifying Google, and finally having my site back without horrible security warnings everywhere is amazingly efficient.
The data seems to have also been pushed out to those that use it extremely fast, as all my Tweetmeme buttons are already active again.

This post hopefully can act as a bit of a counter-balance to all the stories of dread you will find if searching of how hard it is to fix a site after it has been hacked, and what it takes to get Google to reconsider your site after it has happened.

Bravo Google Webmaster and Search Quality team & Stopbadware

 

Liked this post? Follow this blog to get more. Follow

Comments

  1. says

    Andy,

    Thanks for this info. In the past few months, I have helped to clean three hacked WordPress websites. All instances included code injection into the WordPress core files and theme files- no database was compromised. My steps included all the steps you described except implementation of 503 – it is a good idea and I will definitely keep it in mind next time

    I wonder how quickly will your search traffic resume. I no longer get the warning message- is your traffic better today?

    • says

      Certainly signs it has picked up a little bit

      Rankings / Overall trust dont seem to be have been hit hard, just CTR I can’t blame people not clicking on a link that suggests a site is infested with any kind of malware.

  2. says

    How as the site hacked? Thru Gumblar/FTP Hacked or thru some WordPress plugins or something else?

    Was the WordPress table setup to be default wp_

    Usually hackers attempt SQL injections using wp_ tables

    • says

      I specifically avoided details as I am in no way a server or WordPress security expert, and WordPress is just one of the scripts I run on this server

      It is quite possible my server has more holes in it than swiss cheese

      The primary purpose was to give a little credit where credit is due – Google / Stopbadware.org seem to have a very efficient system set up both to spot defaced / compromised websites and unblocking them as soon as things are fixed.

  3. says

    I have never, ever, in my whole life, read a blog that’s as interesting as this one when it comes to WordPress, and all other tech-stuff. Thanks for a good blog Andy.

  4. says

    I am sorry Andy for the hack yet I am emphasised by the aftermath. As much as any blogger/site owner have backups for both ‘files’ and ‘database’, things can be fixed soon from the technical point of view however the impact of hacking itself is the keystone; in your case both the wipping out of 90% of traffic + warning placed in webmaster via Google.

    It’s great how the quality team responded quickly and I think they have done great job and like you, I do not think it was somehow all automated.

    Thanks for sharing this for many users who will learn from this esp the 503 redirection. My only note for the hacking itself is you should also invistigate the problem because if it turns to be a server-security problem, this is not good at all!

    • Brent2 says

      I’ve seen some pretty nasty WP hacks. Things like WP suddenly attacking other servers or base64, gzipped code used to inject random links into comments.

      As long as you stay up to date on WordPress and minimize plugin use, you’re generally fine. Obviously some of the hacks are going to actually happen before WP.org finds out about them and you might get hit. They’re usually VERY good at getting back on track though. If you pay attention when a new version is released (and upgrade) you’re generally safe.

      Now that WP has the auto-update option there’s not a lot of excuse. Some hosting companies even send you an e-mail when a new version is released.

  5. Craig M says

    I’m glad it was so fast for you. It took several weeks for them to look at one of my blogs several years ago… The first time I fixed it I missed a couple things and they said the next request will take x many months before they look again. Can’t remember the number but I believe it was 3 months.

  6. Marketing List Build says

    At last, i already found such interesting information about wordpress. Thanks for posting it, it’s a useful information.

  7. Katie (South France Immobilier) Radisson says

    I found that a site where I needed to collect some password protected pdfs had the dreaded warning and even though I said “ok but still let me in” Google would not show me the page regardless…

    Are they now the web’s police force whether we like it or not?

    Katie

    • says

      Careful with those footer links and if you work full time for RHF International then get them to add you to their about page.

      If not, please don’t use my blog for link building client sites

  8. says

    It always bothered me that proceeding to the site will lead you to another warning message anyway. Why even bother asking if I want to take chances if you won’t let me?

    • says

      Yeah annoying – the worst part for me was trying to diagnose things quickly, trying to view source and seeing the malware warning there too, without the ability to actually see the source code. It also happened to friends trying to see the source code as well.

  9. Georjina says

    Hello Andy,
    My first time here and now that you’ve scared me out my wits because I’m also a first time WordPress user, is there any way to prevent this happening or at least minimize the risk on my side of things?

    Thought I’d ask before I go any further into the chaos:)

    • says

      The #1 solution is to keep regular backups – there are lots of plugins to do that

      There are also various plugins to secure your site better, you should use long passwords etc

      Your hosting service will also do a lot to help with security if you use managed hosting

      Keep WordPress updated

      I have been using WordPress for 4 years without taking special security measures other than backups, and this was the first time it has happened to me to my knowledge.

  10. says

    Andy,

    Just remembered the question I meant to ask when I first saw your post. In your opinion, do you think Google penalizes for liking to “infected” websites even if your links are nofollowed?

    It happens often to me on few of my blogs. I get a trackback from a blog, but as I check where the links is coming from I sometimes faced with that dreadful “Warning” message. On some occasions I succeeded to contact the website owners warning then of the problem.

    I do let through trackbacks from some automated “quasi-splogs” when the offer a good clean link to my website. The problem with some of them is that their owners don’t always pay attention after initial installation and set up and they get hacked quiet often.

    Thought you might have some insight. Thanks in advance.

    • says

      In theory no, but hopefully Google will provide a way to inform website owners of sites they are linking to which have badware in the webmaster tools soon.

      They have the APIs to do it

  11. says

    We had our site hacked a few weeks ago though we weren’t on wordpress. The hackers uploaded a new index.html file with a “You have been hacked by…” message on it. We were on shared hosting and they had hacked the server and taken everybody down. Worst thing was it was the best part of 36 hours before our site was back up and running and we lost a hell of a lot of search traffic, no signs that it has cost us search rankings though.

    Definately an argument for getting a dedicated server.

  12. AutoSpector Inspections says

    Seeing this makes me glad I stick with static pages when ever I can get away with it.

      • Brent2 says

        Static sites are harder to hack but, if you have a form of any kind, it’s very doable. If your static sites are HTML, read up on HTML injection attacks (Wikipedia, Google, etc). They’re actually surprisingly easy to defend against but can be used to drop a dozen porn links on the bottom of your page; each nicely just out of sight until Google notices.

    • says

      There are some reasons why I wasn’t using a number of hacking tools, but they are pretty technical and it would take a lot of time.
      Ultimately my server has a more conventional (but highly optimized) config now, so some security aspects will be easier to manage (and a lot of it is out of my hands with my server management team)

Trackbacks