Nofollow Killed Google Social Graph API 3 Years Ago

Lets face it, I have social profiles all over the web and I am or have been an active participant in tons of conversations on blogs and forums.

3 years ago Google introduced rel=”nofollow”, very similar to rel=”me” , rel=”friend” and other XFN and FOAF standards.
Today Google are championing their new API with the slogan “URLs Are People Too

3 years ago that was true, now URLs on many (most) places where people congregate on the web are no longer “people” because in a supposed attempt to combat comment spam, Google encouraged major sites to use rel=”nofollow” making the links null and void.

I did some initial testing with the example tools Google provides, and the results were less than encouraging, because lots of the data they probably need has nofollow links. There is data out there that is usable, but maybe I need to give Google more help.

Socila Graph API example

Fairly poor results considering how interconnected I am.

People who have better results have them primarily for 2 reasons

  1. They gave Google a lot more data to work with on the form – I only gave this blog URL
  2. They have extensive blogroll links to their social media profiles, and use a blogging platform that uses XFN extensively (as Jordan demonstrated on Marketing Pilgrim)

The Google system currently doesn’t seem to make good use of dispersed information, even when it can be used because there isn’t nofollow, and it includes rel=”me”

As an example take a look at my MyBlogLog profile

Andy Beard MyBlogLog Profile

If you look at the source code on my MyBlogLog profile, you will notice that it uses rel=”me” on all the profile links, so lets see what happens if we use my MyblogLog profile as the source.

MyBlogLog used as source

Lots more results – but they certainly didn’t pick up all my associations from my Technorati profile as I would expect to see a few more of my public blogs, and Twitter.

Lets see what happens if I just use my Stumbleupon profile, which is connected from both my Technorati profile and my MyBlogLog profile.

Here is a direct link to the search with the API

Just one Degree of Separation

Google currently is taking data you give it, and finding profiles it can associate with that data, but they don’t then extrapolate out to further connections. With the Stumbleupon test they did find my MyBlogLog profile, but then didn’t use the extensive data on that page to find more results.
To use the API extensively, it will be necessary to perform multiple recursive calls to find useful data, and some connections won’t be found because some of the hops won’t have rel=”me” associations.

It is interesting to note that they are picking up Technorati claim code in some of the searches, rather than actual profiles that link to the pages in reverse in many instances.
Nofollow is probably why they could pick up my MyBlogLog profile from Stumbleupon data, but couldn’t find it declared in Technorati.

Ultimate Proof That Nofollow Kills This Dead?

The most marked up pages regarding XFN and Microformats that I know of are Technorati, who have been champions of microformats for as long as I can remember.

Almost every single link on my Technorati page has some form of microformat data, but the page also has meta index nofollow in the header.

Now take a look at a very empty page when you use your technorati profile as your source data.

Google Scraper API

Ultimately this isn’t far removed from simple scraping scripts easily created by your average Blackhat SEO, or an averge PHP programmer familiar with using CURL and regular expressions.

Such a programmer could easily create something similar in a couple of days, though it would probably be much easier using Yahoo search results rather than Google’s – Yahoo have an Open API and report links whether they have follow or not.

In fact it is quite likely that such a 3rd party tool would give you much better results than Google currently provide.

Currently I don’t intend to help Google by providing them with more direct links to my social profiles with specific XFN – it can help with reputation management, but at the same time that would push all the great things readers have said about me further down the SERP. That isn’t necessarily an advantage.

I would also be interested if data starts showing up which could only be derived from links that are nofollowed which has numerous SEO implications if the data ever makes it into Google’s main search algorithms.

I would be more excited if Google announced support for additional XFN alternatives (maybe extended from XPN proposals), as an alternative to using nofollow, although the majority of people wouldn’t use them – I am sure paid review services would encourage support.

Currently Google shouldn’t be winning any prizes for this one, why is everyone talking about it?

Update

In this article I linked through to my Technorati profile. Technorati extensively uses meta nofollow, thus Technorati profile pages are not being naturally indexed – you have to link through to them for Google to index them, and whilst I have linked through to various aspects of my technorati profiles in the past, such as the opml of my favorites, I hadn’t linked through to my specific profile.
Since this article was published, Google has indexed it, followed the links, and indexed my Technorati profile as well – the cache data for my profile page is 2nd February.
If you now check my connections using Technorati as a source, you get a massive amount of data from multiple services.

Liked this post? Follow this blog to get more. Follow

Comments

  1. says

    Youre right, I didn’t think of it like that. Anyway, I think in order to get these options widely spread, they must also concentrate on user-friendliness. The wider public doesn’t have a technological affinity with code.
    Anyway all this openness of data can evolve to an important privacy hazard for someone who doesn’t know how to protect himself.

  2. says

    Nofollow has been an absolute disaster since day one. It has completely and totally failed to deliver on any of its promises but Google continues to goose-step down the road of imposing self-emasculation across the Web.

    People need to stop adopting every nofollow tactic and trick that is proposed. Instead, Web application developers, service providers, and SEOs need to stand up to Google’s bullying and say, “We’re sick and tired of your screwing up the Web with Nofollow and we’re not going to take it any more!”

    • says

      Michael you are the anti-nofollow advocate who still uses nofollow on comments even though he requires people leaving comments to register.

      Then in the comments of your recent post about Rand Fiskin’s nofollow results you state this

      There are some legitimate reasons to use rel=nofollow within internal content in a Web that has been mutilated by link spam. All personal profile pages should probably be nofollowed just to discourage people from signing up for profiles (not that a relatively orphaned profile page on the average social media or forum site will pass value anyway, but the perception of possible value stimulates the bogus link economy).

      If you are active on social media sites, those pages can build up a considerable amount of link equity and provide a valuable hub, and with this new initiative from Google could prove to be valuable.

      I would love to know how you determine your site is better indexed than most other SEO blogs, considering before Christmas Google decided to totally break /* reporting which was previously just inaccurate.

      Sure you could do extensive analysis on crawl frequency of every page on my site, but that might become a little excessive, and also is affected by temporal changes.

      It can certainly be improved but even what I would look on as fairly unimportant pages such as infrequently used tag pages still seem to be indexed fairly frequently.

  3. says

    Andy my question is related to the topic, but not necessarily to Google Social graph.

    As you know I have been long time supporter of removing “nofollow” from blogs comments. However lately I was thinking if what had taken place in “no nofollow” movement was all worthless? Don’t take me wrong, I have learned a lot from bloggers like yourself. I have discovered some outstanding blogs in dofollow movement. I am just asking this in terms of the no nofollow movement’s future? Is there any?

    • says

      If you read my comments policy, you could look on the links I allow as those that could be used as rel=”me” links

      Sometime in the future I might tweak the theme and add that, because that is what I regard as the purpose of the links.

      This actually makes the dofollow movement more valid than less, because blogs and discussions on blogs are part of the social graph, a part that without dofollow gets destroyed.

      • says

        Thanks Andy. I think it explains it. I never included rel=”me” into the equation. I guess I have some work to do as well with my theme. Why do you write a rel=”me” plugin? If you have time that is.

    • says

      Rel=”me” is purely informational

      rel=”nofollow” is the only attribute that currently has negative connotations which is why the W3C purists are so negative about how it was introduces, though apparently it is included in new draft specs.

  4. says

    Another great write up Andy! It is unfortunate that Google has limited its own abilities to find the data that they are now championing.

    Yet another example why the nofollow attribute has had large unintended and unnecessary effects on the relationships of linking on the web.

  5. refinance says

    Google has obviously really tried to put a wrench into the blogging communtity but I do think it is important to have some recourse for fighting spam. no follow is ok but you want to fight spam most of all. I dont mind giving someone some link juice as long as they are doing it right.

  6. says

    I actually find that sticking my Technorati profile in there makes a complete mess of the results for how my sites are connected, since I write for some group blogs so I end up seeing data for other people who share those blogs. It actually works OK for my own connectedness – in fact, as you say, it is quite effective based off of the Technorati profile, but that’s likely because I have things like Twitter in there as well. So I guess the net is that it’s an interesting experiment, but not very useful. I also can’t easily instrument links as being “me” on WP.com, unless I go and hack at my pages I suppose. Oh well.

    • says

      Andy it is very interesting that it is working for you, maybe there is a hole in my testing that I overlooked.

      I hadn’t linked to my profile on Technorati recently, and due to nofollow even though my blog is quite prominent on Technorati as a top100 favorited blog, it wouldn’t be indexed naturally.

      Since this article was written, and I linked to my profile within the article without nofollow, Google has now cached my profile page, and the results are a lot more noisy

  7. says

    That is so funny! Google should have realized how a bad an idea it was for them to try to get webmasters to fix a fundamental flaw in their algorithm.

    They should really do away with the nofollow tag, it is SO being misused these days that it’s not even funny. Oh well…

  8. says

    Hi Andy,

    Great posts and interesting comments !
    Just one idea I would like to submit to you.
    What do you think if you have a coComment API that returns to you a list of “social” links that you can insert in your page ?
    The links can contain your claimed blogs in cocomment, blogs where you are commenting or tracking conversations, blogs of your favorites, friends and neighbors in coComment….
    Then, if Google do its job correctly, it should build a more complete social graph of your activity from those links.

    • says

      Honestly I think the biggest help for Google would be for all the social sites to remove nofollow from profile links, and the same to happen with blog comments on blogs that have active moderation.

      I haven’t been using cocomment for a while due to browser problems and a few complaints regarding compatibility, I do need to take another look.

      APIs for this stuff is useful, but only if people create tools that make a real difference.

  9. says

    Yes, true. But also this assume that Google is following the links properly to build the network.
    If you are using coComment browser extension, we can track all blogs you are commenting on or just reading and then include those blogs in your social network. This is an information that Google might have some troubles to build ;-)
    What I mean by API, could also be a line of JS code that insert all the links in your blog page (or at least the most recent ones).

  10. says

    Andy,
    Heard that the new w3c standard will have “nofollow” as a standard element.? Is it so? or is it already in practice?

    (Is everyone gonna be an idiot?)

  11. says

    Great post Andy! This is probably THE most intelligent and personally relevant post regarding the Google Social Graph API that has come out thus far. The idea that the nofollow tag is the ultimate impediment to the Google Social Graph API was masterful!

    Thanks again for the insights!

  12. says

    The most important thing is that blogs like yours encourage participation because you don’t use nofollow. Google’s nofollow element will just make them irrelevant if no one uses the tag!

  13. says

    Great reading, this introduced me to the whole XFN-Thing and the implications. Didn’t know yet that there’s such a huge impact from the nofollow… I just switched my blog to do-follow because of that.

Trackbacks