Here is the official proclamation of best practice in the eyes of Google

I should note that Google classifies their post as “Webmaster Level: Beginner”

I am classifying this post as “Webmaster Level: Here We Go Again”

1. It is effectively a case of “Do As We Say, Not As We Do”

You see, every time I leave a comment on the Google Webmaster blog (something I avoid doing too often), I get pummelled with email spam.

(edit: added a smaller image without thumbnail that doesn’t use WordPress gallery)

If you are using any kind of subscribe to comments feature it can absolutely kill your email delivery rates if you let through spam like this, especially if you don’t have effective blacklist control and a feedback loop.
Somehow Gmail always lets through spam like this, but blocks legitimate comment threads I have subscribed to. True I have subscribed to the comments on the Google Webmaster blog because I want to read legitimate comments, but I don’t want the comment spam in my email box.

Maile Ohye did eventually close the comments on the original blog post, but they almost certainly need better controls.

2. What can I do to avoid spam on my site?

Whilst Google warn about cleaning up bad comments you might have left elsewhere, they don’t give a hint that linking to dodgy places, maybe even using nofollow links could have a negative impact on search results.

Maybe they should have taken this opportunity to warn webmasters about Page Level Penalties for Comment Spam

At least they didn’t pimp Recaptcha which they bought not too long ago

Don’t misunderstand this, I think it is a terrible user experience to have to wade through poor quality comments trying to pick out the gems – one of the reasons I take a harsh line on comment spam is because I respect my readers. I want my comments area to be worth reading.

3. What Happened to Google’s Old Legal Slant?

At the height of the paid links crackdown 2 years ago, there were lots of warnings about the potential problems with disclosure and paid links – in August 2007 I even suggested that the FTC should take a look at Matt’s blog shilling for Google…
Matt now has a disclosure policy, but he doesn’t practice what he preached for so long, very clear in post disclosure which is probably what the FTC would hope for.
Here is an example, his most recent post on Google OS does not state he is an employee of Google within the content.

Shill comments from a legal perspective are probably just as bad as shill blog posts

But I am not a lawyer, and Google employs 100s

In many ways I support the actions Lord Matt has taken on recent comment spam, contacting the supposed company who left comment spam, Rapunzel Rapunzel to see if they were actually aware of the dangerous SEO strategy employed by their contracted SEO company, or maybe a marketing employee.

BTW – Jim Edwards interviewed Mr. Rich Cleland, Assistant Deputy covering topics such as FTC, Disclosure, Blogging, Testimonials etc
Whilst there are tons of kits on sale prepared by laywers, I would strongly recommend this as a first base of call on anything to do with the new FTC advisory on disclosure.

4. Similarities Between Paid Links & Comment Spam

Almost none other than their effect on Google rankings and the vehemence with which service providers defend them, such as when I changed my comments policy, or wrote about Wide Circles and Wide Circles Comment Spam.

• Comment spam generally leads to poor quality sites – paid links generally lead to a site with commercial focus, but it would be rare for someone to invest lots of money in paid links trying to rank a malware site.
• Paid links are generally vetted – comment spam generally isn’t
• Paid links from a human perspective historically were often more disclosed than comment spam

What will possibly become the biggest similarity going forward is the way Google punish site owners who host comment spam, just like they punish people who host paid links. They can’t easily punish sites that are gaining links in this way if they have an otherwise normal link profile, because ultimately the comments could just be a competitor trying their hand at “Google Whacking”.

WordPress Comment Solutions

Shaun almost a month ago released a modified version of Lucia’s Linky Love that removes links from comments rather than nofollow them as a partial solution to Google’s changes to PageRank distribution in regards nofollow.

Dave Naylor is also doing something similar

I also now need to take you back to a comment I left over on SEOmoz

Vladomir Prelovac has come up with what I would regard as a partial solution to this problem

I am glad someone has done this as I have been dropping hints about taking this approach for the last year in various places, back to the old Webbbs days, though it needs taking a few steps further to be truly effective.

There are however some big monster bugbears that need to be considered with this approach, just like with tag pages.
The benefits you will see on his site, with a huge amount of PageRank to play with from the release of WordPress Themes are potentially significant, whereas with a smaller site it can lead to complications, and you might for instance want to noindex the newly created comment pages ;) Vladomir doesn’t use tag pages extensively, more selectively.

For the last six months or so, my understanding of how Google ranks pages has changed significantly, in part due to studying the way Google handles huge sites such as Blogcatalog & Technorati, but it would be wrong for me to publish details without clearance from Tony at Blogcatalog because I had access to their analytics.

Whilst a lot of it would be speculative… almost like a fairy story, for some it might be more akin to a lightening strike than a light bulb moment.

Imagine you have a choice between having a tag page or a comment page in Google’s index

• A tag page you can specify the exact title tag
• A page created with SEO Super Comments you can’t, in many ways the comment is about as optimized as a Tweet on twitter.

Twitter’s upstream based on Alexa is only 10% Google.com, so maybe 20% overall – a large proportion would be navigational queries & Twitter account holder names.I am not 100% confident about Alexa upstream numbers, but they might be more accurate for Twitter than many sites.

You would get an occasional tweet ranking for very long tail terms, but it is not significant.

So if you are creating new pages for comments, you would want them in the index only under specific circumstances.

• Existing flat site architecture with all original content in primary index
• Your categories are indexed and viable landing pages
• You have your tag pages sufficiently indexed which may require various techniques to make the content more unique and useful.
• Have ways to use comment data on tag pages
• Have ways to create tag pages based purely on comments and 3rd party content ;)
• The permalink for a comment from comment feeds points to the new pages, not to an anchor/fragment/”#name” – this has been something that needed fixing anyway, because permalinks on WordPress posts with lots of comments are currently broken, because comments can move from page1 to page2 – there are lots of ways to then use this RSS feed pointing to unique URLs on your site ;)
• Rewrite rules for comment URLs
• Link to a comment should use anchor text based on the title
• Link from a name should provide all comments from that user on a single page
• Extensive use of Ajax – this gets a bit complicated, and it would be experimental, but why have the whole comment on the post permalink at all? At least from a spider perspective. A representation of the comments can be pulled in as pre-cached page fragments. Comments could also be pulled into member profiles if a person commenting is also in some way a site member, and maybe in that situation an individual commenter page should be totally replaced by a member page.
• Integration with social media – if you are pulling in tweets, friendfeed etc, give those a page as well, and then allow people to even comment on those directly from your blog, and push the data back out to whichever service.
• Pull more data from trackbacks/pingbacks – grab an excerpt and host it on your site on a unique page. If someone comments on it from your site, send a pingback

The Future Of Commenting And Aggregation

An even more radical approach would be to totally get rid of “comments” as a unique entity, and many other social sites for that matter, and have only unique personal streams of media, long or short form, video, pictures, text or a mixture, and what appears on other sites, whether on a blog as a comment, or on Twitter, Youtube or an social site would just be a syndicated copy of your original content. Just one permalink for the original content, with full ownership and privacy controls over who could see it.

In many ways Youtube is just a video feed reader where you syndicate your unique video, and you should link back to the original source, and get the original source ranking :)

What I am suggesting is a somewhat reverse approach to “Echo” recently launched in private beta or the Friendfeed aggregation.

A single source that you push out to other sites, rather than a multitude of aggregators. More like Tumblr or Posterous, but with much more control.

As a marketer however, it makes it difficult to reward comment participation without some kind of additional registration process.

The Complexities Of Syndicated Comments & Social Mentions

What really turns your mind upside-down is when you have a situation where you have a private blog post in a members area, and someone leaves a comment which is specific to the private content.

The commenter needs to maintain access controls, but at the same time the owner of the site with private content needs control as well, which can possibly be overridden. Who gets priority and ultimate control?

Who would have moderation rights? If moderated, would it be only the syndicated copy on a publishers site, or the canonical version maintained by the commenter.

Do you really want to mention in an “echo” on your blog that you cross-posted the same content to 100 social media sites?

Disqus?

I recently highlighted Disqus as a solution, it still is, but my reservations are increasing after using the service for a month – whilst the WordPress integration is clearly stated as being beta, I am going to call it a very raw beta – there are tons of problems with synchronization and comment moderation leaves a lot to be desired. My last support ticket to them was 6 days ago, with a second full comment export to try to get sync sorted out remains unanswered.

I have informed them already that I am pulling the plug – the synchronization attempts are hopefully to fix problems that might prevent others having problems in the future.

There are other issues that I don’t feel should be aired here on the blog. I am giving them some time to hopefully get them fixed.

Other WordPress Plugins?

A big shout out to 4 other plugins I have been using recently

Autooptimize – so far it is the best CSS / Javascript optimization plugin I have used (and I have used quite a few) and the author has been highly responsive with fixes to various plugins and widgets. It sets expires and gzip correctly too.

What I have also done is hacked things so Disqus uses local CSS and images, that I will eventually be able to migrate to a CDN, though the Disqus CSS brings up all kinds of horrible warnings in Yslow and Page Speed Firefox Plugins.

Wp Supercache Plus – I am currently using it with Eaccelerator – I am using the “bleeding edge” version from SVN, and am in the process of implementing fragments with thematic – I had a few problems using it with memcached WP Supercache combined with Memcached object-cache.php.
Fragment caching with comments especially will reduce server load each time a new comment is added to a blog under heavy load, such as a product launch.

Probably also long overdue is a mention of Tim’s WordPress Membership Plugin. On the surface most plugin offerings look the same, it is only when you look at the code and how they have overcome hurdles that some solutions shine – I was involved a little with the early stages over a year ago and many features have since been copied, and other offerings have leapfrogged Your Members in more obvious ways, but at its core I still believe Your Members to be the most flexible solution (oh and you can see the support in public). There are lots of useful hooks and ways you can extend the platform, relatively easily, though a little PHP knowledge goes a long way. It is also pretty secure.
The full reasons deserve a lot longer post, but other solutions get promoted extensively without extensive research into alternatives – I need to spend another $500 on alternative solutions before I can realistically write a comprehensive review.
With Your Members it is possible to control access level to comments as well as the posts themselves. If you have a private post, you also want to selectively keep the comments private.

FAQ-Tastic – Zain now has both a free light version (that is very flexible) and a pro version – it is a serious solution for anyone looking to leverage their audience to create new product or content offerings. I am frequently asked to add an “Ask Andy” section here on the blog, but I will most likely do it in a more private area.
Comments on custom areas of WordPress is something I don’t think 3rd party systems will ever handle effectively.

This post has been a little bit of a mixed bag, but hopefully you find something useful & worth sharing with others.

I now feel that 95% of bloggers should switch to using Disqus, though I have some reservations.

These are some of the reasons why:-

• Matt Cutts today confirmed that nofollow links can reduce the amount of PageRank that flow to internal pages. The easiest current solution to solve this problem is to use Javascript for comments.
  It is an external javascript file, which Google can’t really handle currently, and even if they did, the chances are it might only count as a single link to your disqus discussion.
  Blogstorm has gone into the problems facing comment links in more detail, something I highlighted when Matt first mentioned this clarification at SMX.
• Matt Cutts in the same post highlighted again who you link to matters, and I think Google is going to place more and more emphasis on this. It is a lot of work for the average blogger to keep control of user generated content, and even the best comments sometimes come with spammy links. I have always maintained that “dofollow” isn’t for everyone because of the time commitment.
• Disqus is universal – it can be installed on every major blogging platform – many SEO solutions won’t be universal or easy to implement
• Can Spam & Email Deliverability – this is 50/50 – I have highlighted in the past that emails being sent from your own domain can be a significant liability
  

  It is your choice based on your own research and the legal advice you have received whether you think emails being sent from your domain which are not totally under your control could represent a problem.
  I honestly don’t know if Safe Harbour rules might apply to email delivery. I don’t know of any blog owner who has had problems either from a legal perspective, or with their hosting or domain registrars, but then I personally only know a few people who have been killed in car accidents – I know a lot more people who drive cars than publish business blogs.

• Invalidated Cache – this is a major consideration for high traffic blogs, and potentially product launches. The javascript doesn’t change on each new page load, thus your cached content also doesn’t change (if you just use their javascript on your page) – this can represent a major reduction in server load, even if you are using some kind of Op Cache (eaccelerator APC Xcache), RAM based page cache (Memcached) or more advanced techniques using page chunking. Forget conventional WP Cache / Supercache for product launches, it just can’t cope on its own.
• Social Media Viral Effect – the social media viral effect of using Discus is significant. Not many people are exposed to services such as backtype, but tons of people use Facebook – implementation of Facebook, Twitter and other logins for commenting whilst possible with WordPress isn’t trivial, and that is more plugins to deal with, more server load etc.
• Get to visit Disqus more often – I sometimes leave comments on other blogs that are using Disqus, and sometimes the comment notifications don’t get to me (deliverability issues) – I just noticed on my Andy Beard Disqus profile that Scott Rafer responded to something important 2 months ago, and I didn’t see it.

Negative Side of Disqus

• The SEO of the site needs some major work – it is almost insulting that the link to Twitter on my Disqus profile is followed, yet the link to my blog isn’t. The anchor text from a conversation on Disqus back to a blog isn’t exactly ideal. This is how Google sees my Discus Profile
• Google is very bad at indexing content on Disqus – this is partially due to the Disqus SEO problems – certainly a conversation I took part in 4 days ago isn’t indexed.
• I have heard reports that managing spam can be an issue, though I haven’t tested it, I rarely see spam on highly popular blogs
• It isn’t suitable for private content – you would have to use alternative commenting on private posts if you are running your blog as a membership site. That is something that can be worked around.
• It is hard, maybe impossible to market to people after they have left a comment. With standard WordPress comments, after someone has left an email, you could present a page offering site membership, a one time offer, or an affiliate product – even suggest related content of interest.
• If you have lots of niche blogs, you will hardly want to include all of them on a single Disqus profile if you want to stay under the radar of your competitors. How would a blog network cope? Retain ownership? I can’t see B5 Network with a profile of 300+ blogs in Disqus and managing who can moderate comments.

The SEO problems with Disqus are fixable, and in the current Google climate could offer significant advantages – plus Disqus could conservatively gain at least 400% traffic even without new adoption.
It is possible even if they also gave much more SEO friendly links throughout the whole site.

I am recommending Disqus above competitors JSKit and Intense Debate because both of these services the email subscriptions don’t stack in Gmail – a nightmare if you subscribe to comments on a popular blog, plus I believe there is more chance of Disqus fixing problems for a win/win solution.

Who knows, we might even get Twitter to remove nofollows too… eventually

Disclosure:- I am recommending Disqus even though part of my startup plans would have involved an SEO, product launch and membership friendly system with refined marketing funnel. If I ever get it off the ground, there are ways to migrate back from Disqus.