This post was originally posted Nov 7, 2006 – I have added a screenshot as the original subject site is no longer online – it is still just as relevant today as it was over 3 years ago. In places I have added some additional commentary or expanded on original ideas.

References to PageRank should be looked on as synonymous with Google Juice & overall site authority, and not just green pixels in a toolbar, though that can be a good visual indicator at times.

How a Blogroll can kill your PageRank

I followed a link from Digg a few minutes a go, read the story, and as I frequently do on any site I visit, I snooped around a little.

I actually do exactly the same every time someone writes a blog post referring to me and pings my blog. It is the polite thing to do, and maybe I can add something to the conversation. It also allows me to relate any comment to the person who is writing about me, either positively or negatively. Everyone is entitled to opposing views. What is often important is why they have an opposing view, and it isn’t always obvious.

Now about the site in question:

1. I have only read half of one article there, so I don’t know much about the site contents
2. The reason I am linking through to the site is purely from an SEO point of view
3. This is a very common problem, very easy to make, and honestly not too hard to correct.

With that said, here is the site Kerry Fox Live (Archive.org link – the site seems to have been offline for 2 years.)

Initial Site Analysis

So the first thing I notice is that it is a PR3 site.

The internal categories are mainly PR2

The archives are mainly PR2

The individual post pages are generally PR1 or unranked

A large proportion of the content is duplicate syndicated content from services such as Associated Press, without any wrapping

But the site has been around for 16 months

You can make a splog, chuck duplicate content at it, and get a PR4 or PR5 after a few months.

So what is wrong with the site?

Take a look at the sidebar

At a guess (I am seriously not going to count them all), only 20% of the links on any given page point to an internal page. (yes that sidebar is on every page)
There are 2 blogrolls, one of which seems to be websites and blogs he likes, and the other is a whole load of press related sites… news sources.

Every single one of those links is a live external link leaking PageRank to other sites. Those other sites are not reciprocating in any way.

Now I am sure someone is likely to point out that those links provide a service for visitors, and maybe add a little authenticity to the site.

What visitors?

We are looking at a news site with an Alexa rank of close to 2M – not 2k, but 2M

Now there are times you want to sacrifice a little page rank to other sites, especially if they are reciprocating, sharing visitors, or in the case of my blog, I like visitors commenting and joing my “community”. You might also do it in a carefully controlled way from a mininet to one of your own sites.

Solutions

Get rid of the blogroll on all internal pages. It is giving away too much traffic to other sites, not to mention PageRank.

Add nofollow to all the blogroll links that are not reciprocating, or you don’t want to be overly friendly with. (Update August 2010 I would probably tend to use javascript in some way)

Increase internal linking to compensate for all the leakage.

How to Increase Internal Linking

• Recent posts – 10 links
• Top Posts – 10 links
• Recent Comments – 5 – 10 links
• Tagging + Tag Cloud – 50+ links
• There wouldn’t be a need for as much internal ball linking if there wasn’t so many external leaks. The site is gaining very few comments.
• Related posts – 5-10 links
• Related reviews – 5-10 links
• Glossary links

Emphasis should be placed on the links you wish visitors to traverse

External Linking

The site has 2 visible external links to the front page. I am not sure how many to internal pages, but even if it did have external links, any PR given would immediate leak.

Just syndicating one article will generate loads of backlinks, far in excess of what you can achieve with a single blog post (unless you have 100k+ readers). Based on my analysis of “A” list bloggers, their average blog post might normally generate around 10 backlinks (showing in Google).

(update August 2010: – whilst many of the bloggers I analysed in 2006 have 10x as many subscribers by RSS & email now, the number of links they receive, other than from splogs & social media probably hasn’t increased)

Conclusion

It is not rocket science, just simple maths.

If you have 100 external links on every page of your site, you need lot of internal links to retain some (hopefully most) of your PageRank, and it would certain help if those people you give a link to on your sidebar reciprocate in some manner.

(please note that includes me – don’t put a link in your blogroll to my site – sure I appreciate the links, but I would much prefer just an occasional mention in your blog)

If you can’t get a reciprocal link, use nofollow, (August 2010 – blocked external javascript), or stick them on their own seperate page so they don’t suck your own site dry.

And finally… this site structure plagues a huge proportion of blogs. Other blog owners who do not have this problem, quite likely don’t even realise why.

Further reading:

Revenge of the Mininet | 3rd Party content | Blog Comments | No Follow

Update: whilst I am still a fan of article marketing, I no longer recommend any service that doesn’t provide a way to have unique passwords for each distribution site.

It has been described as lipstick on a pig and others have just offered tips on site indexing.

What Happened To My /* Search Query?

Before this change, one of the tools still available to SEO practitioners to give an indication of the depth of site indexing was to add /* to the end on a site: search query.

Thus you would use

site:andybeard.eu/ – this would return the total pages supposedly indexed
site:andybeard.eu/* – this would return the number of pages in the primary index

By subtracting one from the other, you get some indication of the number of pages that might be in the supplemental results.

The numbers changed on a daily basis, and by datacenter, and were a little unpredictable, but it at least gave some indication.

Some sites I routinely checked for comparison were seo book, Michael Gray, Seomoz, Sphinn, Search Engine Land, Search Engine Journal, Bumpzee, Blogcatalog, Mybloglog and a number of others.

Typically Seo Book and Michael Gray had very deep index penetration, with above 90% of their pages in the primary index as indicated by /*

Many of the higher output SEO blogs faired far worse, with sometimes less than 50% primary index penetration, but far more total pages in the primary index.

Site structure makes a huge difference – if you have lots of duplicate content pages such as extensive use of tagging, you might end up with a fair number of those pages in the Supplemental Index because they do not receive a huge amount of links from external sources, and many receive a very small percentage of internal linking.
They would however still rank well for long tail search queries because of the different title tag, and combinations of content presented.

A wider net sometimes has bigger holes…

/* Now Returns Significantly Fewer Results

SEO Book now returns less than 50% of pages with /*
Michael manages 50%
SEOmoz has less than 20%
Search Engine Land seems to be fairing well with close to 75%, actually more than previously (I seem to remember less than 50%)
Search Engine Journal also manages close to 75%, again more than previously (again it was less than 50%)

I have dropped down to around 30% what was previously 85%, though my site structure isn’t quite the same as I had it before WordPress 2.3.1 (I still need to get some plugins modified)
Site structure might have had as much as a 10 or 20% difference, but not more.

I still see very little difference in Search traffic or positioning

I Don’t Know What It Means…

• It could be some datacenter issues that mean that only a small percentage of sites have so far been updated
• I am currently discounting this being some kind of penalty
• Rather than reducing the size of the supplemental index, Google might now be storing the majority of pages in Supplemental, and effectively have a “super primary” index instead.
• The tool could just be broken or set to give random results

I can understand the removal of supplemental results from the primary SERPs, but /* was actually quite useful. If supplemental results have disappeared, you would expect it to return the same number of pages as the search query without /*

Why do we have a useful tool fubarred?

Takeaways

I am sorry to say not a lot, it reinforces Google’s position not to really say a lot, though apparently there is no real penalty for duplicate content, and not to worry about how pagerank flows in your site.

So if you ended up with some of your internal pages PR5, and your home page PR2, don’t worry, you won’t have a problem selling extremely below market price text links… cough

About the reporting of paid links Adam had this to say:-

One of the most important points to make of this is that this is used to augment our data, and it’s not used to specifically take actions against sites.

.. plus quite a bit more about how this is really just to “augment” existing data.

On snitching…

But, on the reassuring side, we’ve had some people voice concerns “what happens if my competitor decides to report my pages, even though they are completely following the webmaster guidelines. Is some action going to be taken?” The answer to that is definitively not, because this does not lead directly into an action pipeline, but rather it is used only for review. It is not going to directly or adversely affect any pages or sites that are following the webmaster guidelines.

I did leave a comment and have exchanged emails with Eric over something related. We will see if what I have discovered is related or just a bug.

I did give Eric’s interview a Digg, as it is very suitable for a home page. It is not really SEO, but “Google” after all.

Digg Story

Go on, give it one, I need to get my Digg account a little more credible, and that helps you too.

Read the full interview with Google’s Adam Lasnik

Update: The Paid Link Meme

I thought I would add this as some counter-perspective on Adam Lasnik’s position, and that of Matt Cutts and Google in regards to paid links.
They have always argued that paid links are always detectable. Graywolf blew that theory apart.

Major Websites and Press Agencies Seriously Don’t Give a Damn About Conventions

Vipul Arora wrote a story today entitled Yahoo plagiarizes content – does not give credit!!

But was it plagiarism?

Lets go into detail and see what happens, and how this all works.

Museum of Hoaxes publish a great piece of linkbait “Top100 April Fool’s Hoaxes of All Time”

The Missing Pointer

AFP Picks up the story

Now unfortunately AFP doesn’t include a link through to the original source of the material it used. Links provide attribution on the internet and are used by visitors to read the rest of the story (only the top10 was published, not all 100 items), and by search engines in their calculation of the original source.

I am not sure where the law is on this, they only used 10% of the material, and they did provide attribution in textual form. They also didn’t use the content word for word.

Then again in value to readers the Top 10 is probably worth more than positions 20 – 100

Here are 10 of the top April Fool’s Day pranks ever pulled off, as judged by the San Diego-based Museum of Hoaxes for their notoriety, absurdity, and number of people duped.

You might think this is a minor issue, but Google currently shows the title of the story on 10,900 websites.

Yahoo Plagiarism?

Yahoo was the most prominent site that picked up the story – they have permission from AFP to use the content that legally probably isn’t plagiarism, but bloggers if it happened to them would be very upset, unless they gave specific permission.

Yahoo are in many ways the innocent party in this, but ultimately the site that is benefiting from the content the most. It is a very popular story that has already garnered over 900 votes on Yahoo. That is a lot of votes as not every reader would click the vote button.

Just imagine if there was a link through to the original source

Effect on Traffic

The Museum of Hoaxes should have had a tidal wave of eager readers visiting their site. It is possible they gained a small surge from search traffic from people who were interested in finding the rest of the Top100.

The story should have also done insanely well on social bookmarking services.

This is actually an extreme example of what happens when correct attribution (in internet terms) isn’t given by the press to the original author of an article.

Printed Media

Printed media in the UK, if referencing a website, typically use a service such as TinyURL.com to provide a link from an article so that they don’t have to print a long URL – that is a very ethical method, though I would question why they don’t spend $20 and buy their own redirect script that can be branded.

A Similar Example

This is actually a similar situation to what happened with Colleen who was recently interviewed by the LA Times regarding PayPerPost.

The followup video interview was a cracker, and you really should read the comments for a discussion on ThisNext vs PayPerPost, and whether you should have to use a disclosure when commenting.

I should also point out that I have seen a number of blogs that have also written about the LA Times story, and linked to their syndication partners ;)

Pot Calling the Kettle Black

The press shouldn’t attack the ethics of bloggers if they don’t provide correct attribution for the medium in which they are working. On TV it is verbal, in print it is written, and the best print even provide short links.

Online correct attribution is links.

Even Wikipedia provide a clickable click, although it does use nofollow

Note: Screenshots were used with permission from Vipul, it saved me 5 minutes preparing this. He has contacted AFP questioning their method of attribution.

For a long time my blogs have performed amazingly well with Google Blog Search. I always appear in the relevant results quickly, and the results I obtain have some reasonable longevity, even when I am not the original source of a story.

Considering how much competition I often have for certain search terms which everyone seems to be writing about because of common interest, I must have been doing a number of things right.

Bill Slawski of SEO By The Sea a few of days ago broke the news of Google’s Patent Application for Ranking Blog Documents.

SEO Round Table posted a synopsis lifted from the Cre8asite Forums that had been posted by Bill, and seems to be the easiest to understand.

I am going to do a little bit of mix and match here, and inject my own commentary but my interpretation of the patent is actually slightly different to those that I have read so far.

It should be noted I am working my way through the patent itself, and not recompiling the summaries of others.

Relevancy & Quality – Blog | Blogpost

It should first of all be noted that in the patent Google doesn’t differentiate between individual blog posts and whole blogs.

The phrase “blog document,” as used hereinafter, is to be broadly interpreted to include a blog, a blog post, or both a blog and a blog post. It will be appreciated that the techniques described herein are equally applicable to blogs and blog posts.

Later on in the patent, they also mention that feeds are also included within the documents that are compared and rated.

two distinct sets of data are used to determine a score of a blog (or blog post) in response to a search query–the topical relevance of the blog (or blog post) to the terms in the search query and the quality of the blog (or blog post), which is independent of the query terms. The quality of the blog (or blog post) may positively or negatively affect the score of the blog (or blog post)

Relevancy – this applies to the search term, thus Google will analyse the blog page, and they will also in some way determine the relevance to the whole blog.
Quality – this is irrespective of the search term, so think about factors from outside your niche

Google Blog Search – Positive Factors Affecting Search Quality | Relevancy

• Popularity of the blog document

A number of news aggregator sites (commonly called “news readers” or “feed readers”) exist where individuals can subscribe to a blog document (through its feed). Such aggregators store information describing how many individuals have subscribed to given blog documents. A blog document having a high number of subscriptions implies a higher quality for the blog document

This patent was first of all applied for 13th September 2005, with Google Blog Search launched 13 September 2005. At the time they were logically not basing this on numbers available for Google Reader subscribers. The Google Reader blog was launched October 21, 2005 with a post saying they had been up and running for 2 weeks.
Maybe there is a coincidence between the 2 events.

So which data were Google basing this part of their patent on? Some services such as Technorati and Bloglines do provide readership data, as does Feedburner, though most services report readership data as they are collecting new blog posts to a service like Feedburner, who aggregate the statistics.

It seems there might be some value is collecting Technorati favorites (my reciprocation policy might be well worth it) beyond limited bragging rights. Google of course through Google Reader now have access to lots of usage data, so maybe other sources will eventually be phased out.

• Implied popularity of the blog document

This implied popularity may be identified by, for example, examining the click stream of search results. For example, if a certain blog document is clicked more than other blog documents when the blog document appears in result sets, this may be an indication that the blog document is popular and, thus, a positive indicator of the quality of the blog document.

Click data from search results, possible from Google Toolbar users.

• Existence of the blog document in blogrolls

The existence of the blog document in blogrolls may be a positive indication of the quality of the blog document. It will be appreciated that blog documents often contain not only recent entries (i.e., posts), but also “blogrolls,” which are a dense collection of links to external sites (usually other blogs) in which the author/blogger is interested. A blogroll link to a blog document is an indication of popularity of that blog document, so aggregated blogroll links to a blog document can be counted and used to infer magnitude of popularity for the blog document.

Everything I have ever read has suggested that for normal search, blogroll links that are site wide carry diminishing value. Just because it is listed here as part of the calculation does not necessarily mean that everyone should start building up huge blogrolls… well unless they want to game Technorati and have a blog network.

• Existence of the blog document in a high quality blogroll

The existence of the blog document in a high quality blogroll may be a positive indication of the quality of the blog document. A high quality blogroll is a blogroll that links to well-known or trusted bloggers. Therefore, a high quality blogroll that also links to the blog document is a positive indicator of the quality of the blog document.

Another revelation, links on high quality pages are worth more than links on low quality pages.

Remember that “blog document” can mean both blog page and blog site.

Can blogroll just refer to a list of links on what is identified as a blog. Thus a column of links to related pages might also class as a blogroll, whether in the sidebar or below the content.
Thus a list of links to related documents on the same site could be looked on as a blogroll on a blog document.

Related links plugins are very powerful, especially if you also include them in content that gets syndicated by design, or by sploggers.

• Tagging of the blog document

Tagging of the blog document may be a positive indication of the quality of the blog document. Some existing sites allow users to add “tags” to (i.e., to “categorize”) a blog document. These custom categorizations are an indicator that an individual has evaluated the content of the blog document and determined that one or more categories appropriately describe its content, and as such are a positive indicator of the quality of the blog document.

Well some sites do allow you to tag in a meaningful way, maybe Google uses shared tags from Del.icio.us and other sites, but many of those use nofollow extensively.
It is my own belief that self tagging content heavily with plugins such as Ultimate Tag Warrior helps a huge amount. I have given lots of examples before, but more recent examples include

toolbar pagerank
google reader feedburner
feedburner google reader
compete toolbar
duplicate content supplemental results

Yes, I am just going down the inbound traffic results looking for likely candidates that rank well in both blog and normal search and aren’t totally obscure. These are subjects that sites in my niche have also talked about, with the keywords in the title, and which you would expect to rank higher than my own content.

This doesn’t just affect blogsearch, Google have been using it for some time with the main results as well.
Here are my observations regarding tagging from back in November, especially how they could relate to LSI calculations.

• References to the blog document by other sources

Wow revelation again, god links are worth having either to pages or blog.

• Pagerank of the blog document

Pagerank is still relevant, who knows for how long and how much.

It will be appreciated that other indicators may also be used.

What seems to be missing, at least at time of application?

• Domain age?
• Trustrank?
• Page Titles?
• URLs?
• Growth rate of link popularity

Plus lots more that also factor into it, but general search patents probably also cover blog search.

Google Blog Search – Negative Factors Affecting Search Relevancy | Quality

• Frequency of new posts on the blog document

The frequency at which new posts are added to the blog document may be a negative indication of the quality of that blog document. Feeds typically include only the most recent posts from a blog document. Spammers often generate new posts in spurts (i.e., many new posts appear within a short time period) or at predictable intervals (one post every 10 minutes, or a post every 3 hours at 32 minutes past the hour). Both behaviors are correlated with malicious intent and can be used to identify possible spammers. Therefore, if the frequency at which new posts are added to the blog document matches a predictable pattern, this may be a negative indication of the quality of the blog document.

Make sure there is some variation when you publish your content for the day, especially with future dated posts.
Most spamming tools are actually fairly sophisticated, thus I am not sure this measurement is very accurate. It most likely indicated a blogger who is very organised these days.

• The content of the posts in the blog document

The content of the posts in the blog document may be a negative indication of the quality of that blog document. A feed typically contains some or all of the content of several posts from a given blog document. The blog document itself also includes the content of the posts. Spammers may put one version of content into a feed to improve their ranking in search results, while putting a different version on their blog document (e.g., links to irrelevant ads). This mismatch (between feed and blog document) can, therefore, be a negative indication of the quality of the blog document.

This is actually a very significant and interestingly worded item. Google are stating that they are comparing the content of a feed with the content on your pages to ensure it matches.

Based upon this:-

• Don’t use a content spinner on your feeds to avoid duplicate content
• Allow Google to index your feeds
• If you use related links on your blog, make sure you use them in your feeds too
• Duplicate Content, especially in feeds

Also, in some instances, particular content may be duplicated in multiple posts in a blog document, resulting in multiple feeds containing the same content. Such duplication indicates the feed is low quality/spam and, thus, can be a negative indication of the quality of the blog document.

I can’t say I have noticed a problem having a lot of straggling RSS feeds on categories and tags.
This could also be referring to things like the large footer I have on each post, though I haven’t seen a problem with that either.

After the last toolbar pagerank update I spent some time studying Matt Cutts’ blog, and also looking at how pagerank was being transferred around my own site. Pagerank is only slightly useful as a guide, and only immediately after an update.
Rather than repeat myself, you can read about my organic garden approach to this site.

• Collective Intelligence

The words/phrases used in the posts of a blog document may also be a negative indication of the quality of that blog document. For example, from a collection of blog documents and feeds that evaluators rate as spam, a list of words and phrases (bigrams, trigrams, etc.) that appear frequently in spam may be extracted. If a blog document contains a high percentage of words or phrases from the list, this can be a negative indication of quality of the blog document.

Google invest a lot of research analysing spam, detecting various word matching patterns, and use that to identify other documents.

• A size of the posts in the blog document

The size of the posts in a blog document may be a negative indication of quality of the blog document. Many automated post generators create numerous posts of identical or very similar length. As a result, the distribution of post sizes can be used as a reliable measure of spamminess. When a blog document includes numerous posts of identical or very similar length, this may be a negative indication of quality of the blog document.

This might be of special interest to those that use out-sourcing for articles, you need to ensure the article length changes.

• A link distribution of the blog document

A link distribution of the blog document may be a negative indication of quality of the blog document. As disclosed above, some posts are created to increase the pagerank of a particular blog document. In some cases, a high percentage of all links from the posts or from the blog document all point to ether a single web page, or to a single external site. If the number of links to any single external site exceeds a threshold, this can be a negative indication of quality of the blog document.

In some ways this debunks the benefits of blogrolls mentioned as a benefit, but as previously quoted, Google are using blog document in multiple context, and comparing the context, thus it could just refer to multiple spam links always pointing to a single domain within the content.

• The presence of ads in the blog document

The presence of ads in the blog document may be a negative indication of quality of the blog document. If a blog document contains a large number of ads, this may be a negative indication of the quality of the blog document.

Remember this is just a patent, and Google recently relaxed the rules about having ads from other networks along with Adsense. As long as a page is of a reasonable size to support the adverts, I don’t think there is a problem. If you just have a heading and 5 words, with 10 advertising blocks, you might want to add a few more words.

However they go on to say this

Moreover, blog documents typically contain three types of content: the content of recent posts, a blogroll, and blog metadata (e.g., author profile information and/or other information pertinent to the blog document or its author). Ads, if present, typically appear within the blog metadata section or near the blogroll. The presence of ads in the recent posts part of a blog document may be a negative indication of the quality of the blog document.

Thus if you are using blocks in the content for all your ads, you might not rank as well, especially if you use multiple networks. You can probably get away with 3 in the content, or maybe 1 or 2 per post.

• It will be appreciated that other indicators may also be used

Conclusion

The feed stats information is very useful, and looking at the timing, my conclusion is that Google might have been using Bloglines and Technorati Favorites data, with Google Reader in its infancy, or maybe though less likely, when blog search was introduced, they weren’t using that part of the patent

For me the most significant information was tagging, but just linking though to Technorati with your tags isn’t a great idea.

Remember that Google have their own blogging system, and they have archives and labels, and they are not going to create a system to generate duplicate content and then penalise you for it. Google wouldn’t have added such a system unless they intended to benefit from the enhanced data.

You don’t have to build your blogs in a 1990s era tree like structure to rank well.

Subscribers and Technorati Favorites may help you rank.

Does your mind wander when you are doing a physical job that doesn’t require much other thought? Mine certainly does.

Duplicate Content of Biblical Proportions

I have been doing some research of “Top 10 Lists” for a new site. Whilst no list of the top 10 best selling books is exactly the same, there is one common feature. Every single top 10 list states that The Bible in all its variations has sold more than 6 billion copies.
That is close to the current world population, estimated by the CIA to be 6,525,170,264 (July 2006 est.)
Thus there are probably as many bibles in the world as people

That is an awful lot of duplicate content

Gideons

Gideons is an interesting organisation, whose work is evangelism and distribution of the bible. You will see Gideons bibles in every hotel room, and I was given a Gideons bible whilst at school in addition to a larger bible I was issued for Religious Education lessons.

With the help of the Gideons organisation, whenever you need a bible, there is one close at hand.

Spread the Word

This is a fundamental concept in all religion, not just Christian. It is not enough to have the knowledge and faith, you are also meant to pass it onto other people. Lots of the time the knowledge will fall on deaf ears. If the knowledge is out there, and easy to find even without specifically looking for it, such as in a hotel room drawer, it has a benefit.

Worthwhile Duplicate Content

I am sure many of those 6 billion bibles are in dusty attics, unused drawers, or buried at the back on shelves, long forgotten. Every now and then one copy gets rediscovered, unearthed or brought out of hiding.

The Bible and Duplicate Content

A church is not the only place you can find a copy of the bible. I am sure many religious people read more of the bible while in church than the cumulative total of all other places, but that doesn’t make it pointless to have multiple copies in your home, school, library and even hotel rooms. All those copies of the bible are seeds, and every now and then one of those seeds gets picked up and begins to germinate, wherever the seed is located.

Your Articles and Duplicate Content

I recently wrote a mammoth blog post on my philosophy with article marketing. Many marketing experts believe that article marketing no longer works, or you should be highly selective where you publish your content.

I mentioned a strategy, that as long as you link back to the original source for your content that people can follow, it makes sense to spread your article as widely as possible.
I don’t discount special publishing deals that will bring your article to the forefront of thousands of people, but that opportunity is often reserved for established writers.

Articles Are Seeds of Knowledge

Spread your articles far and wide. Sometimes they will land in fertile soil, and germinate and grow almost immediately. Whilst I wouldn’t dare liken EzineArticles to a church, it certainly is very fertile ground for your seeds to grow.

Some of your articles will almost certainly end up landing on less fertile ground, or buried deep in a website long forgotten. It is a seed that hasn’t germinated yet, but one day it might be uncovered and grow into a tree bearing fruit.

Sometimes your article may end up on even more unsavoury territory, with a link to you not being provided, or even worse plagiarised. It will happen, such is the nature of people. If the purpose of your article was to help people more than market yourself, then it really doesn’t matter if occasionally the link to you is broken. A person reading your article is still going to look for further knowledge on the subject, and eventually find their way to respected sources of information, such as yourself, just as someone picking up a Gideon’s bible might eventually find their way to a church.

If you don’t sow your seeds, they don’t have a chance to grow

As I have discussed in the past, when discussing blog internal linking, for this blog I am doing “massive ball linking”.

Links come into the blog from various places and to specific content. By using tagging and related posts on each page (maybe a few too many now), whatever pagerank, trust and relevance that a page is given, get passed onto other relevant content.

It also gets passed onto people who leave comments or trackback. When you link to me from your blog posts, the juice flows back – sometimes more, sometimes less, but it is all relevant links – just what the web is all about.

I suppose you could look on this as a “organic garden” approach. Just let things grow wild, and only interfere if something needs drastic change.

I haven’t been overly worried about the multiple forms of duplicate content that are typical on a WordPress blog.

Exactly Why I Don’t Worry About Duplicate Content

First of all I discussed this a little not so long ago, in particular about duplicate content and supplemental results.

Matt Cutts isn’t worried about honest duplicate content

Proof – Matt Really isn’t worried

First of all lets take at look at Matt’s robots.txt file

User-agent: *
Allow:

Then you can look at the metadata on all Matts duplicate content pages

Category archives – I am going to link to his seo glossary, I am sure it doesn’t get many links.
Date Archives
Sequential pages – lets go back in time a few pages

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Matt Cutts: Gadgets, Google, and SEO &raquo; Movies/Videos</title>
<meta name="generator" content="WordPress 2.0.7" /> <!-- leave this for stats -->
<meta name="robots" content="noodp">

Basically he lets the search engines work out what is going on.

There are simple plugins out there to prevent these pages being indexed and followed in various ways, but Matt Cutts doesn’t worry about it.

The Wayback Machine

Yep, more duplicate content – Matt doesn’t block them

Matt Doesn’t even block the login page to his WordPress installation – that same page appears over 2 million times on the web.

Let it Flow

OMG I am wasting all that Google Juice that I worked so hard for!

Or maybe not…

Google Juice is liquid… it flows, it you give it a direction to flow in.

Whilst you may give a page with duplicate content some juice, that juice flows back out to other pages. Just because that page might end up in the supplemental results, doesn’t mean the Google juice is being wasted.

Some blog designs unfortunately place too much emphasis on the most current content. They don’t use tags, their date archives are accessed by a silly calendar widget on the front page, and they don’t have any other routes for pagerank to flow.

Structured Linking

It is possible to use only a very structured linking system for a blog, but it is important to think about relevance. In Revenge of the Mininet there are some great linking structures defined. They would be ideal for highly “niched” sites.

Many blogs, including this one, cover a wide multitude of subjects. A carefully constructed linking structure would be much harder to define and you might lose the benefit of giving relevance to specific pages.

Structured Relevance?

Some duplicate content pages give great, maybe even enhanced relevance to content.
Date archives in many blog designs are not very helpful. If date archives and previous pages are used as the primary navigation to previous content, and path for search engines to take, you really aren’t giving your content any justice or longevity.

Content Longevity

Lets look at some of my internal pages for longevity

How a Blogroll can kill your Pagerank – 2 month old post, linked to a few times in posts, but it appears a lot on pages due to tagging and using related posts – currently PR4

NoFollow and Pink Boxes – also PR4

Some posts on UTW Tips – both PR3

Ultimate Tag Warrior SEO Tricks (pt 1)
UTW – Tagging SEO Tricks (pt 2)

From browsing around, I have discovered that the Google Toolbar Pagerank isn’t completely comprehensive, as I have posts that should have PR4 that don’t. One of them was my highest ranking page in November and December.

It was hard to find a post that hadn’t been later referred to in a more recent post but this is a good example:-

No NoFollow & No-Nofollow.com

That post is again PR3, and didn’t receive any specific direct love from outside links or internal links in posts. It did benefit from tagging, and related posts.

This time around I didn’t tag and categorize my old content from my blogspot domain. Almost all of that content didn’t receive much pagerank love unless it had been linked to directly.
I am going to see if for the next update it can be revitalized.

How much of your old content retains visible pagerank?

It is actually still difficult to compare, because so many pages still haven’t received pagerank they probably deserve. Even Matt has some pages that are linked to from PR6 categories that show a toolbar pagerank of zero such as http://www.mattcutts.com/blog/starving-to-death-review/

I would link to it directly with a live link, but I don’t want to ruin the example

For one of Matts posts that one probably received a lot less inbound links

Yes that is a Yahoo link – wouldn’t it be nice to have a “-all” flag on Google searches to list everything below Google’s threshold as well, even if it is just an aid for webmasters to find splogs.

Conclusion

Matt is a busy man, maybe he hasn’t had time to optimize and avoid all these duplicate content issues but I really believe it is nothing to worry about, and Matt might just be demonstrating this with his own blog.

You can aim for a more focused structure on a blog, using various forms of dynamic linking but it is not something easy to achieve yet with any platform.

He is blogging various sessions from the Chicago Search Engine Strategies conference.

Here are the bullet points from just one of the speakers on the session (Shari Thurow), and I bolded one of them

Ways that search engines filter out redundant content:

* Content properties – removes boilerplate elements and reviews what’s left
* Linkage properties – Is a press release on the wire service and the press release on a web site duplicate content? No, because the linkage properties are distinctly different, even though the content is the same
* Content evolution – most content does not change. High average page mutation – news site. Low would be a manufacturing site
* Host name resolution – If the hostname resolves to the same domain name, one version of the redundant content will be selected as the canonical url
* Shingle comparison – References Andrei Broder. Shingles are word sets. The more shingles that a document has, the more likely it is duplicate content.

There is very little difference between article marketing and press releases.

The big difference is in the other navigation elements on the page.

If you are using one of the very popular article site scripts, work hard in differentiating your site from other sites using the same script, especially if you are receiving the articles from a similar source such as Article Marketer.

The same would be true of people using RSS feeds from Article directories. If you are not doing anything to change the linking structures of a page and relationships between content, you are not going to achieve the results you want.

Do take the trouble to pop over to Lee’s blog and read the full article:- Duplicate Content and Multiple Site Issues

My original article was Keywords, ranking and a little SEO fun. It is over 16 months old but still spreading.

Take a look at this website in Chineese (I am not sure which Chineese)

First impression, you can’t understand a word of it… correct?

Try moving your mouse over the article content – up pops the original English phrase.

I am not sure exactly how they are doing this, and technically translation can be looked on as an unauthorised modification of the original work, but honestly I don’t care.

The way they have done this is extremely smart, especially if they have used some form of machine translation. For any phrase that wasn’t translated correcty, there is an immediate reference to the original document.
In addition Google are picking up the live backlinks surrounded by English text.

This is a feature that translation services such as Google Translate need to make their translation more useful.

It is one of the most innovative uses I have ever seen for duplicate 3rd party content.

Categories are for storing your documents into nice managable chucks on related topics.

Tagging can and should go far beyond categories in relating content to specific keywords.

Here is a good example:-

WordPress Training
Wordpress Tutorials
Wordpress Tuition

I would suggest you wouldn’t want to have all 3 phrases as a category in your sidebar. You could however be quite justified in tagging a related article with all 3 phrases, if they were appropriate.

There are several reasons for this:-

1. Tag searches do not use LSI. They do however quite often display related tags. The more data you provide related to your article, the more accurately it can be classified, and later found. In many ways your tags should be very tightly related LSI keywords.

2. MYSQL Search – lets face it, it isn’t highly effective. If someone is searching your site, the chances of them finding what they are looking for are remote unless you provide a lot of keywords on a page.

3. Folksonomy provides a better way to search when you use tag clouds to provide visual suggestions for what a reader may be looking for. In my example above, “training”, “tuition” and “tutorials” actually provide different stimulai

4. Better references to related content

Lets say you have a niche blog on dogs.

You might have categories on dog training, dog health, dog accessories etc

You might use tags to interrelate articles on dog training at feeding time, dog food, and dog bowl.

5. Making your website wider – making a better mouse-trap

Does Google really give penalties for duplicate content, or duplicate content that doesn’t have a specific purpose?

It is my belief that Google doesn’t give duplicate content penalties for content that has a specific purpose, that they can easily identify.

Technorati is all duplicate content, but it has a purpose, helping people find what they are looking for. I noted recently that tag pages at WordPress.com which are also full of duplicate content, give great backlinks for the blogs hosted there.

Google Likes Tag Pages

I think Google in some way within their algorithms are allowing duplicate content if it is on a page that is clearly a tag page, whether on an authority site, or on individual websites.

It is easy for people to spot such a page, they normally use tag or tags in the URL. If they wanted to algorithmically wipe out all that duplicate content they could with a wave of their hand.

But they haven’t… in fact pages that Google identifies as a tag page, it might be argued, are sometimes given a boost in search engine results.

I am sure you want some proof

I can’t give you total proof.

This is based upon my observations on multiple domains I own or monitor closely. I am not going to openly reveal all my niche sites (people try all kinds of ways to search for them every day, and they find old sites that have been allowed to be found).

I will give you one example though, which I noticed today.

This example is based upon the following:-

This is quite a new domain, but I do have an existing domain andybeard.name redirecting here with a global 301. I also have an existing blog on blogspot redirecting here with a meta refresh.
I have a few “exposed” niche sites linking back to this site.

That being said, Google doesn’t seem to be showing any backlinks (there are quite a few of them), though other search engines are.
It has been reported recently that Google are giving enhanced results to blog search within their main listings, especially in Europe which is a frequent test location. This isn’t related, because those results were appearing at the bottom of the Google listings.

I am located in Poland, and my IP address is Polish. Someone located in the US might see something different, using the same search URLs.

Here is a screenshot of a search result I noticed today. This one is on Google.com, but I have also tried Google.co.uk, Google.at, Google.pl, Google.de and a few more.
The results were similar. On the foreign language sites I generally held the position for the first English language result.

Google Search for WordPress Training

It isn’t a highly competitive keyword, and the results for “WordPress Tutorials” which is more competitive are different.

Now I do have to admit the tag page that is gaining the result is fairly junky looking. It is a result of the way my content is laid out, and I do need to get it fixed.
I include tags at the start of my content, and those tags appear at the start of any excerpt. This is true for excerpts, whether created by WordPress or by Ultimate Tag Warrior.
As standard there isn’t an easy way to adjust excerpt length unless you create a custom excerpt for every post.

In my defense it is not deliberate keyword spamming, it is something I am aware of, and it will be fixed. I am not a programmer, I just tinker with PHP, and the tag pages and archive pages on this blog are currently “out-of-the-box” configurations.

Specific to Google

This weighting of tag pages seems to be specific to Google.

For instance for the same search on MSN my original content is ranked around 26th position, but they still have lots of tag pages showing, and it is obvious the search is not very LSI based, as lots of dog and horse training pages appear.
Yahoo has fairly similar results to MSN, I have a result in 23rd place, but it is a totally different page.

Folksonomy and LSI

Folksonomy is in many ways the same as LSI.

With folksonomy the relationship between keywords is created by humans.
LSI is meant to be based upon algorithms, but is a lot more complicated. It is however based on words, and it is much easier to start with a list of human word relationships and to then apply additional relationships.

It is my belief that Google is in some way using folksonomy as part of its LSI calculations. It is even possible that they have recently increased the relevance of that part of their calculation.

Going Forward

Tagging is important for folksonomy based search. Tag pages on your own domain seem to be gaining a lot of relevance in Google.

Over the coming weeks I am going to be working to improve the quality of my tag pages, so that they provide unique content, and enhanced value for my readers.

Using tags for all keyword links might add enhanced relevance.
If you were buying textlinks, paid posts etc, would the value of the textlinks be enhanced if they had rel=”tag” ?

I have heard bloggers argue that creating links with relevant keywords is keyword spamming. In my opinion it is actually “best practice” if you are linking to someone to use a relevant keyword to the content you are linking to. It isn’t Google bombing, it is helping the search engines relate different content. If you then include rel=”tag” in the link, you are also helping folksonomy search.

You should maybe think carefully about your tagging strategy for 2007