How I Kicked Myself Out Of Google Blogsearch For Months or Years

By Andy Beard | Published: September 5, 2010

I’m either a total idiot or a raving lunatic or both.

I can’t even pinpoint within my data as to when this foolish or unfortunate incident occured because well… just look at the data.

Google Blogsearch referrers

That is nigh on 5 years of Google Blogsearch referral data, though because of the long time range it is listed as sampled data. There may be some traffic sources I have missed – variations of url depending on how Google were displaying blogsearch pages, but that is referrals from http://blogsearch.google.com

  • In the past when I have checked at various times (though admittedly it has been a while) I had ranked well in blogsearch.
  • There have always been so few referrals that I have more or less ignored Blogsearch
  • The only blogsearch I have used are for links to me – in the WordPress interface, and occasionally to grab more results than WordPress displays – it has always been unreliable
  • Blogsearch picks up links from all kinds of things
    • blogroll links
    • comments
    • if you send a pingback and a blog displays them, it will come up in blogsearch too

Because of all of these factors I had always assumed that the idea of using noindex on a feed of any kind was to prevent that feed appearing in Google’s primary organic results.

A pretty Feedburner feed isn’t a terrible landing page, but it is possble to do better. I have even written about using my feedburner URL when leaving blog comments in the past, as in some ways it immediately signals you want people to subscribe more so than linking to a blog.

Other RSS search engines were indexing my feed content – Technorati, Blogcatalog, Icerocket – my feeds were being read by my readers, picked up by various Twitter robots etc.

And of course my content remained indexed in Google’s primary organic index.

But then a few days ago I was browsing a little and looking for additional sources for a story followon, and noticed I wasn’t listed for previous coverage. I hadn’t been specific in the title that I was related… but there wasn’t a lot of competition.

Then I discovered this:-

Blogsearch for andybeard.eu

My first thought for 5 minutes was somehow for some crazy reason I had become penalized in Google Blogsearch – then I rationalized it in thinking it must be something to do with noindex settings in Feedburner.

You see I had never equated noindex with a blog search engine – every other blog search engine which sent me traffic was still picking up my content and sending me traffic.

Google Indexing RSS Feeds

There is still a very real need for a way to tell Google…

“Hey Google, this is my RSS feed – you can index it for Google Blog Search, but I don’t want it to appear in the organic search results.”

3 years ago Google were saying they were working to remove RSS feeds from organic search.
3 years later feeds from Feedburner are still appearing in organic search results.

http://feeds.feedburner.com all the results seem to have been removed
http://feeds2.feedburner.com there still seems to be plenty of feeds within the search results

Information About & Help With Feedburner Since Google Acquisition

On a scale of 1 to 10 Google Feedburner Support gets a 2 – it is a free service, Google monetize it providing Adsense for feeds, but don’t expect anyone to answer support queries in the Google groups from Feedburner.
Documentation is sparse – hardly updated in the over 3 years since Google bought Feedburner… but then there haven’t been too many visible changes other than adding Adsense. I am sure there have been changes to help with scaling, especially how it eventually was made easier to integrate with Blogspot, but very little for anyone else.

Feedburner Noindex Controls

So this I believe is the culprit

Feedburner RSS Noindex

This is the code that gets added to the RSS feed.

<xhtml:meta xmlns:xhtml="http://www.w3.org/1999/xhtml" name="robots" content="noindex" /><meta xmlns="http://pipes.yahoo.com" name="pipes" content="noprocess" />

That data is still not transferred to feed items that are shared within Google Reader or feeds such as tags created there – which can get fed to other places and indexed.

The left hand doesn’t know what the right hand is doing

I have explained my whoopsie, but somewhere in the Googleplex they are a little confused over what they are doing as well.

Blogsearch isn’t the only search for my Blog posts

For instance there is Google Buzz

Now remember – Google is treating the noindex on my RSS feed as being an instruction to not include my content in Google BlogSearch…. so you would expect that instruction to be universal for the RSS content.

Google Buzz Indexed Fulltext

Those were taken from the PUBLIC timeline of Buzz. That is content that Google isn’t indexing on Blogsearch due to a noindex in the XML.

I also have my full content being fed into Facebook and being indexed and made searchable within Facebook, but at least that is my choice.

The only way to prevent content being shared and indexed is currently to block Google Reader from accessing feeds. I have been trying for over 4 years to get Google to introduce more publisher controls for sharing… as it would be easy to share private content from Google Reader by mistake… with Pubsubhubbub it can be broadcast by mistake to your 1000s of Buzz subscribers instantly.
This is possibly why Google have never introduced support for http authentication.

With their current stance for sharing freedoms, it doesn’t make sense for them to treat the current xml declarations as an instruction not to index the content in Blogsearch, as the content is in Buzz anyway. It should be treated as just a noindex for the page.
Alternatively they should add support for x-header noindex, then noindex in the XML would be for search engines, and it should travel with each content item, even to Buzz, possibly with no way to share the content.

  • Share/Bookmark
Posted in Google, SEO Blog, blogging tips, web 2.0 | Tagged , , , , , | Leave a comment

Google Experimenting With 20 Search Results Per Page?

By Andy Beard | Published: September 4, 2010

You have always (well for as long as I can remember) been able to select the number of search results shown by Google.

The default is 10
Then they add in various other things like news, video, images, local results etc.

Those additions are generally known as universal search

Then you might have results included from your social circle – I have quite a large social circle because I follow lots of people on different social networks, so I almost always see them in the search results.

Google 20 Results Per Page

This isn’t unusual in itself, you could select 20 results before, but now even if I go into advanced search and select 10 results, or manually add &num=10 Google is still giving me 20 search results.

20 search results for lawnmower

Pseudo Site Search 20 Search Results

A couple of weeks ago Google rolled out what I coined pseudo site search. I spent so long trying to gather data, work out exactly why it was happening and come up with a name for it that Malcolm got the drop on me writing about it.

20 links forced in search results pages

I should point out pseudo site search isn’t brand – there are more prerequisites – there has to be a very specific indication in the search query that the intent might have been to perform a search query for a specific domain, but either the searcher was unaware of syntax or just lazy.

Commercial Reason For Google To Make Change

A week or so back Dan Taylor of Destination 360 mentioned in a comment on the travel technology blog Tnooz that for the search query [bellagio las vegas] he was seeing 7 out of 10 results from Bellagio.com – travel is a competitive even for hotels as a specific destination and Dan happens to have a page that might rank even higher for Bellagio Las Vegas now (thanks for the great example search result)

I was an idiot and didn’t grab a screenshot then… sigh.

So here is a current 20 page result of which bits were being pushed around. It isn’t now showing a pseudo site search.

bellagio result was pushed down

Everything below the current #5 result for Trip Advisor was being pushed onto the 2nd page of results, and trip advisor itself was at #10 with Vegas.com 8 & 9

Now imagine that everyone was forced to have 20 search results like I currently am for US searches… I still get 10 in the UK & Poland.
Whilst those results for hotel review sites would be pushed down, they would still be on the page, and a reader might be more tempted to scroll down a 20 result page.

Significance

For some search queries I think this would be useful, though it might cost Google revenue as there are no ads at the bottom half of the search results if people did start scrolling.
It may also result in the 2nd page of results being totally worthless, rather than just very low traffic, for all search queries.

This doesn’t mean you should ever be happy with a page #1 search result – the fold line – the part of the page immediately visible when opened gets most clicks, thus unless someone is using a very high resolution monitor anything below result #3 is frequently severely hampered.

Also with personalization and region there are significant variations in what is seen in the results.

You will also tend to have a lot more double search results in Google.

I would assume this is just a test for now, though Google did roll out the pseudo site search to everyone at once.

  • Share/Bookmark
Posted in Google, SEO Blog | Tagged , , , , , , | 7 Comments

ASA To Oversee UK Websites & Social Media

By Andy Beard | Published: September 1, 2010

The IAB in the UK is reporting that from 1st March 2011 the Advertising Standards Authority (ASA) will now have its coverage extended to include additional forms of online marketing and communication.

It was thought that this might come come into play in September (or at least 3rd Quarter) along with other revisions to existing coverage in the CAP codes which come into effect today, September 1st 2010.

The FAQ (pdf) provided by the IAB (Internet Advertising Bureau) states:-

However, this is a major step for advertising and the internet which will potentially impact upon everyone doing business online. There needs to be a period of grace to raise awareness and therefore long-term compliance.

Why is the money to police this coming from?

The ASA and the self-regulatory system are advertiser-funded: that means that advertisers themselves foot the bill. This is currently achieved via a 0.1% levy on ad spend collected at the agency level. To meet the expected additional costs, this levy is to be applied to PPC search marketing, mobile marketing and affiliate marketing (none of which was previously levied). Google has also agreed to supplement this.

Bold added for my PPC and Affiliate marketing readers benefit.

What Is Covered

  • advertisers’ own marketing communications on their own websites
  • advertiser controlled marketing communications in other non-paid for space

This includes marketing communications on advertiser-controlled ‘pages’ on social networking sites, such as Facebook, MySpace and Twitter.

User-generated content (UGC) will not fall within the extended remit unless it is incorporated – solicited or unsolicited – into an advertiser’s marketing communication. For example, only the advertiser’s marketing communication on a social networking page will come within the extended remit.

Thats mean it will also probably cover things like paid tweets and affiliate marketing on social networks, even if users are allowed to word their promotions themselves.

I have no idea how you can comply with Twitter – there isn’t even sufficient characters in a profile to include some disclosure, let alone things like company registration information in an accessible manner.
The same information would also have to be available through Twitter clients – you can’t really use a background graphic due to accessibility problems, and it doesn’t get seen by client users.

I also hope the ASA will follow through with their investigations into the activities of UK marketers using Strategic lawsuit against public participation tactics against bloggers who cover their announcements of decisions.

  • Share/Bookmark
Posted in Top, marketing, web 2.0 | Tagged , , , , | 4 Comments

Twitter Lists (part 1) – Twitter Wants Your Brand

By Andy Beard | Published: September 1, 2010

It has been 6 months since I left Twitter – plenty of time for Twitter to clean up their SEO act and if they needed/wanted some direct feedback I am not hard to contact directly.

What I am about to reveal in some ways is akin to publishing 0-day security stuff, and some SEOs will look on this as revealing things that they have been happy to exploit for months – thus some of this is fairly widely known, but maybe you don’t.

I will also include some thoughts on “the reset vector” and “evaporating PageRank” in later parts (my views radically differ to those widely publicised – even at recent conferences)

Lots of this article series is not based upon verifiable facts or even poor attempts to construct tests that would be ripped apart anyway. I am just throwing my thoughts onto the interwebs.

I have decided not to do this as a video – some of my loyal readers have accessibility problems with videos and I don’t want to exclude them.

This is a little technical and some aspects would be highly dependent on how Google treat internal linking within Twitter. It is hard to exclude external factors as there are very few ways to create a controlled experiment and there are so many possible ways to filter trust based on influence within the Twitter ecosystem and the whole social graph.

Robots.txt

#Google Search Engine Robot
User-agent: Googlebot
# Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl
Disallow: /*?
Disallow: /*/with_friends

This code prevents Googlebot crawling your historical tweets
Individual tweets if they get some juice from elsewhere can be indexed, but twitter aggregators tend to nofollow links to the content they repurpose.

You would think as Google crawl robots.txt on every visit, they would always have a copy of the robots.txt available… well unless you used x-robots in some way to nocache it.
You would also expect it to be reasonably up to date.

However here is a link to their current robots.txt as visible to humans, and it matches the cached version, and the version I have on this page above.

Interesting Factoid

Twitter doesn’t curently rank in the top100 Google results for Microblogging
http://www.google.com/search?q=microblogging&pws=0&hl=en&gl=US&num=100

So What Does Twitter Want To Rank For?

Quite probably Twitter’s only goal is to rank for your name or your brand.

In fact if you don’t have a Twitter account for your brand that is actively used (it is fairly safe to squat), it is highly likely that you will never have Twitter appear with realtime results for brand searches in the main SERP unless you are a trending topic.

Once you invite social media into your house, it is hard to get rid of.
(it isn’t as bad/good as it was a few months ago)

I noted to some friends when Ikea was having some social media unrest, and had just announced something so lots of press attention, they still didn’t have a scrolling realtime search box appearing in their brand search engine results, whilst it might only take a mention of a popular Twitter user’s name a few times, maybe a few retweets of a blog post for it to normally appear.

I assume Ikea own their trademark account or someone is being very careful squatting it (maybe even Twitter management) – it has PageRank – it has some followers – but it isn’t active.

The Twitter account currently ranks ~181 in the US search results on Google, just behind quite a recent article on Huffington Post – oh and I just linked to it.. oops.

Ikea Twitter Account

Considering the strength of the brand, that is a pretty strong ranking without any further support such as the link I gave it.

Real Time SERPs

There have been some changes to realtime SERPs recently which Dave covered.

Here you can see that the people attending the Mozinar went to bed fairly early (must believe in the witching hour on the NW coast of the US, or the bar closed)

realtime results

However unlike a few months ago it is very hard to get realtime results to appear in the main search results pages.

Mozinar SERP

Stay tuned for part 2 – we are going to look at the internal linking structure, and how to leverage it to maximum effect either for your own brand, or someone else’s.

  • Share/Bookmark
Posted in Google, SEO Blog, web 2.0 | Tagged , , | 1 Comment

Reddit.com Hostile Takeover of Digg.com #diggrevolt

By Andy Beard | Published: August 30, 2010

Digg is a community of millions of users but it only takes a few hundred to stage a significant protest – a few thousand active diggers working in a coordinated manner and Digg might have to rethink their current plans… after a year+ of development.

Digg 4 Is a New Aggregator

At least that is what it is meant to be, based upon my understanding – the site now has content from major publishers being fed by RSS to the site.

Most publishers seem to be opting to submit all their content
I added a feed for selected posts to be fed there which might be suitable for a wider audience

But there are things which really shouldn’t be there.

Techmeme

I love Techmeme, I think it is a great news aggregator, but nearly all the stories – especially the lead stories are duplicates of existing content that is being fed to Digg.
It is just the way it is with Techmeme – sure there are the occasional stories that make it to Techmeme by way of a tip on Twitter, but those are only occasional, and even then often are from a large site that would have made it there anyway.

Techmeme on Digg

I am not going to single out internet and tech cebebrities but for instance I have seen Google Buzz RSS feeds submitted.

On one such post I saw this

Digg Bury Spam

Pretty clear instructions, and later down there were instructions on how to find posts from Reddit.com

http://digg.com/search?q=site:reddit.com

Sure enough that brings up lots of Reddit stories

Reddit Takes Over Digg

And these stories are making it to the front page – even blindly

Effect on Digg Home Page

7 of the 10 stories on the home page are links to Reddit

Digg - Homepage

Will Kevin Rose Offer an Olive Branch?

I don’t think so… at least not yet.

  • There is no such thing as bad publicity
  • Digg is gaining a lot of good publicity with the people it wants to attract – the people who were not Digg elite
  • Digg will end up with their button back on tons of blogs and news media sites that previously abandoned them for Twitter – free advertising as soon as a story is published.
  • Casual users will either get
    • News from their friends and personal selection = dumbed down crippled feed reader
    • A front page that updates a few 1000 times per day.
  • The Digg elite (who grew Digg) will either get used to the idea, be thrown some kind of bone or will defect to Reddit or Mixx or other communities (many ex-Diggers such as Zaibatsu are huge on Twitter.

Digg On Reddit

Nice welcome message for new mmembers who had never heard of Reddit
A little gloating about owning the fron page of Digg – (they have as much right as Techmeme to be there)
Some interesting traffic stats

Digg On Reddit

Digg Inbox Zero

Over the last few days I have been maintaining “Inbox Zero” in my Digg account, just hiding every story (some after I Dugg them)
Most of the people I still follow are fairly inactive on Digg, so the amount of stories was quite small.

Digg 4 Inbox Zero

Today for some reason I can’t get the “hide” button to work – almost like it has been disabled.

Will I be adding Digg buttons here?

No idea… maybe I will have a vote for it when things settle down a little. I also need to do the same for Sphinn.

I am also evaluating what to do with 3 Reddits I set up 2 years ago
WordPress (lowercase P dammit)
Blogging
Internet Marketing

Oh and just to be a little ironic – more on Techmeme

  • Share/Bookmark
Posted in Top, web 2.0 | Tagged , , | 4 Comments

Google Pay Per View – How Will They Handle First Click Free?

By Andy Beard | Published: August 30, 2010

Google like searchers to have a good user experience.

This means that if they show a search result for something, if a user clicks the result, they should be given the information they expect – what is promised in the headline and snippet in the search results.

Given as in free

There should be no exceptions.

This is especially true for news results in Google news and mainstream media – if they want search traffic based upon their content that is behind a paywall or other restriction, they have to give Google and their users free access for that first page view.

It is quite possible to get stubs of a story to rank, but that is likely to result (long-term) in a poorer quality rating of your site contents by Google (more coming on that in future blog posts).

This is technically called “First Click Free”

First Click Free In Action

There is a story on Techmeme from the Financial Times about Google possibly looking to publish Pay Per View Videos on YouTube.

This is the link they provide

http://www.ft.com/cms/s/0/e638714e-b396-11df-81aa-00144feabdc0.html

If you click that link you hit the FT.com paywall – well freewall as you can register for free to read a certain number fo stories per month – any more and you have to pay.

Financial Times Paywall

However if you are accessing the article from Google, you skip the barrier

Financial Times No Paywall

Wanna Watch Movies Online?

Lots of people want to watch movies online, but most of the people want to watch the latest box office movies online, and quite possibly many of them want to do it for free.

At the end of June U.S. Immigration and Customs Enforcement (ICE) cracked down on online video streaming sites in “Operation In Our Sites“.

ICE - Movie Sites Shut Down

More commentary on this in a July 1st post by Torrent Freak.

So that shows the demand might be there… but Google have a problem.

First Click Free With Movies

Hollywood face a constant struggle to keep full (illegal) feature length movies off the internet, or at least out of the search engine results and the internet is international as are search results.
There is nothing more frustrating for an international user than discovering content through a search engine, or even manually whilst navigating Youtube, and being constantly presented with content they are not allowed to watch.

Current movies on Youtube whilst not available to all countries, are available to many locations – even to me here in Poland. Some of them are even in colour.

For online video you might compare a site search of Youtube vs Hulu

Youtube – http://www.google.com/search?hl=en&safe=off&gl=US&tbs=vid%3A1&q=site%3Ayoutube.com
Hulu – http://www.google.com/search?hl=en&safe=off&gl=US&tbs=vid%3A1&q=site%3Ahulu.com

Hulu videos are not being indexed as videos… possibly because of the streaming method they use.

But they do appear in normal search

Hulu In Google

Clicking that result brings me to this page

Hulu Access Restriction

What happened to “First Click Free” Google?

Here is an alternative which is even less helpful

Hulu Access

Those searches were using Google.com & US geo-location, but it is just the same with Google.pl and PL geo-location.

http://www.google.pl/search?q=Fringe+-+Full+Episodes&pws=0&hl=en&gl=PL

Fringe Full Videos

Google With Vevo

Google already have some funny content deals with Vevo

Custom branded channels
Custom Player with clickable link to Vevo
Custom Navigation Bar

YouTube - Taylor Swift

Custom restrictions on access to some content internationally. Recently I have been going through my 3 year old son’s Youtube playlists removing videos that have recently had access restrictions added to them.

Pay Per View vs First Click Free / Freewall & Paywall vs Geotargeting

I am bundling all of these technology solutions together because ultimately they are all appearing in my search results. If I go to click one of the results and I don’t have access, I feel jilted… by Google.

Barry (from Visual Script in Belfast)recently posted an article over on the Dojo/Huomah SEO Training blog about The Death of SEO.

His point was that search is moving to other platforms, and in particular search on other devices.

People are watching movies on their iPad – they will soon also be watching movies on their new Apple iTVs or whatever they end up calling them and paying money to Steve Jobs rather than Google.

You search for a movie, find it, pay and have access and it is much easier to handle regional access controls with regional catalogues.

Currently the results being returned for video searches in Google are little better than search spam – in many cases the spam results are actually better as they often lead to what people are looking for, at the risk of malware infestation.

Have you ever tried explaining geotargeting access restrictions to a 3 year old?

  • Share/Bookmark
Posted in Google, SEO Blog, web 2.0 | Tagged , , , , | Leave a comment

You Too Can Debase Yourself Or Your Business For A Fiver

By Andy Beard | Published: August 27, 2010

The FTC doesn’t like fake testimonials – they can also get you in trouble in the UK.

I have covered disclosure in blog posts extensively but fake testimonials just irks me.

There is a story in the New York Times over a settlement just reached for some fake testimonials on iTunes (via Techmeme)

The story reminded me of something I saw earlier on one of the new “job” sites, Fiverr earlier in the day. Some of the stuff posted there is quite legitimate, but all these offers for testimonials for whatever you want are just wrong.

Fiverr Testimonials Reviews

Now I have been a supporter of paid reviews in the past

For me they are legit when as an author you get to review something, even if just the website and if you offer some kind of opinion, then it has to be your honest opinion without any restrictions, and preferably without any oversight – free to publish honst criticism if justified. I have also supported clear disclosure.
You could also have advertorials written by a 3rd party as long as they are clearly marked.
You would also be wise to nofollow or otherwise block search engines from counting the links or they might get upset.

I can see video reviews being totally legitimate if they use the same criteria though how you tell Google that a video is a paid review I have no idea, especially if it is posted to Youtube.

What I can’t sanction, and to be honest it makes me feel all creepy and dirty is flat out paying for a positive video testimonial.

This is high risk not only for the person doing the review, but also for any business owner using the testimonial. I have no idea how this would also affect Fiverr who probably profit from any transaction.

  • Share/Bookmark
Posted in Top, marketing, news | Tagged , , | 14 Comments

New Digg 4 (Basic) SEO Score 30/100

By Andy Beard | Published: August 26, 2010

This is just a quick look at basic SEO factors and how Digg screwed them up.

I am not going to get into complicated internal linking sturctures, or even discuss things like nofollow and robots.txt

This will be just SEO 101 that any webmaster should know.

1. Title Tag

Used on Digg

<title>Digg - Topsy Search &amp; Twitter Backups</title>

Most SEOs think that keyword prominence is important for both ranking and click-through rate thus this title would be much better as:-

<title>Topsy Search &amp; Twitter Backups @ Digg</title>

5/10 but no banana

2. Meta Keywords

The keywords meta tag is only used by Yahoo of the major search engines currently, but should be specific to a page.

<meta name="keywords" content="Digg, pictures, breaking news, entertainment, politics, technology, headline news, celebrity news, offbeat, world business, sports, funny videos">

Having the same keywords for every page of the site is spammy and pointless thus 0/10

3. Meta Description

The description is not used as a ranking factor by major search engine, but might be used on other websites such as Digg so can be a secondary ranking factor.
Lots of sites these days don’t set a description because they concentrate on Google, and often just decide to let Google decide on which words to use as a description within the search results for each article.

The primary benefit of having an enticing description is to boost click-through rate.

Digg Description

Grabbing the whole article content and slapping it into the description is not cool – 0/10

4. RDFa

RDFa is geeky SEO stuff – I don’t even really want to discuss it as most sites aren’t using it, but damn.. I can’t ignore this junk.

Dig RDFa

It looks like we have a problem – they got the title much better here (no mention of Digg) but they have taken the whole article contents and included it a second time.

RDFa if it wasn’t there wouldn’t be an issue, but as it is there, and buggered up, they get 0/10

5. URLs

Digg have always rewritten their URLs so they are nice for humans rather than a bunch of parameters, but they should really fix the word seperators. Google does not treat an underscore as a space.

http://digg.com/news/technology/topsy_search_twitter_backups

No improvement thus 5/10

6. Links

Google like links – links are what powers the relevance in their search engines. If Google can trust the links then having them on a page is often a good thing. Linking to good resources has been mentioned by Google Engineer Matt Cutts as a positive ranking factor.
Some websites don’t trust their users to post good links so they stick rel=”nofollow” attributes in the code to tell Google and other search engines that the links can’t be trusted.

<div class="columns  group">
                <input type="hidden" id="item_id" value="20100826002508:946ca73a-0f8f-4d8f-8493-b4cbffbc9be9">
    <div class="column full group" id="main-column">
        <div id="permalink-story">

             <div class="story-item item-20100826002508_946ca73a-0f8f-4d8f-8493-b4cbffbc9be9 group" ><div class="media group"><div class="digg-btn has-tooltip item-20100826002508_946ca73a-0f8f-4d8f-8493-b4cbffbc9be9"><a href="http://andybeard.eu/3001/topsy-search-twitter-backups.html" target="_blank"><span class="digg-count"><span>1</span></span><span class="digg-count-label">diggs</span></a><a class="digg-it group""><span>digg<span class="digg-btn-icon"></span></span></a><span class="digg-btn-bottom"></span></div>

It seems that Digg now trust their users to post good links… or that they trust me to post good links, or they have decided my domain is trusted and this link was generatd from my official feed import. This could also be oversight.

The link itself uses the anchor text “diggs” and gets rewritten as a voting widget. That isn’t so good both for the site receiving the link, and for Digg themselves because it is hidden… though the anchor text is relevant to the widget.

It would be nice if it stays like this but don’t hold your breath. 7/10 for now

7. Canonical Part 1

Digg does not use canonical tags or 301 redirects to clean up messy URLs with extra parameters
This is an example:-

http://digg.com//news/technology/topsy_search_twitter_backups?link=12345

To fix this issue in the header Digg should use

<link rel="canonical" href="http://digg.com//news/technology/topsy_search_twitter_backups/" />

That can cause lots of issues 3/10

8. WWW – Canonical Part 2

There is a difference between these 2 URLs

http://digg.com

http://www.digg.com

They might serve the same page, but it can cause the “Gogle Juice” to be split between pages.

For years this is an issue Digg didn’t fix, but I am happy to see it has now.

Finally! 10/10 despite how long it took to fix.

9. Email Sharing Link

Digg Email Sharing Link

We seem to have another copy of my article.

This causes two problems:-

  • It is more junk repeated content on the page
  • Someone is going to receive an article without the images – that is a very poor reader experience

For hiding my article in code at every opportinity 0/10

10. Related Links

I am a big fan of linking to related content, either the latest on the specific story or similar topics. Whilst I am not going to pick Digg up on their internal linking structure too much, I am thinking of this from a general perspective that it is good for users and search engines. Related posts plugins and widgets are hugely popular with readers, and the equivalent on Digg would be great for content discovery.

They don’t have it at all, so no points 0/10

Total

Going into this I didn’t realise I would find so many basic issues – I am sure the Digg engineers are aware of many of them so this is just adding them to the bug reports.


Digg (Basic) SEO Score 30/100

I am sure someone is going to chime in about relative importance as I have valued each item the same. If I took importance into account I would probably be rating this more like 20/10

Updates

Apparently there is a lot of unrest amongst Digg users. Do check out the podcast by the Drill Down 42:30 onwards. More on Techmeme.

  • Share/Bookmark
Posted in SEO Blog, Top, web 2.0 | Tagged , , | 19 Comments

Topsy Search & Twitter Backups

By Andy Beard | Published: August 25, 2010

I am a SEO geek – if there were 140 characters on Twitter that have meaning – I might want to be able to find it, or help other people to find it.

I know some people look on SEOs as spammers – some are – I look on my job as helping people find content they are looking for. I am passionate about it. I wouldn’t class myself as white grey or blackhat, but I also try not to do anything that would pollute the web.

It is one of the main reasons I left Twitter. The SEO strategy in place at Twitter was holding my content hostage from normal indexation by Google, and Google have so far failed at indexing historical content.

Topsy Search

I welcome the news yesterday that Topsy have improved their search & Danny has more in-depth coverage.

Topsy has some great search functions – here are some example searches

Influencers

It is very easy to work out who are your biggest Twitter fans, though there is no way to sort by number of tweets. They have some kind of # tweets x influence calculation.

Andy Beard Supporters

I really haven’t promoted my posts that much compared to many, considering @andybeard was always my official channel.

This method of sorting hides sock puppet bot accounts run by you or others (I don’t have any pimping this site)

Sphinn Pimps

Often people are complaining about excessive self promotion on social media – Topsy makes it fairly easy to work out who is pimping what on sites like Sphinn.

Topsy Sphinn Pimping

Historical Tweets

How many tweets does Topsy have in their archive from me?

Topsy History Andy Beard @andybeard

1075 Tweets (you will see why this is significant later)

Friend Feed

I have a lot of content piped into Friendfeed since it launched but you can’t trust Facebook to keep Friendfeed online, especially the search which for me was one of Friendfeed’s best features.

Friendfeed Search

That was only a temporary glitch – Friendfeed search is working again now, but I didn’t see lots of screaming and blog posts about the demise of Friendfeed search

Finding My Tweets

Can I find the tweets I want to find? No

The alternative is I can back up my own Tweets and then do something with them…. well at least some of them

Twitter only allows your last 3200 tweets to be backed up in any way.

Here are some options

Online Twitter Backup Services

Online services in theory once you join will continue to backup future tweets, and if Twitter ever lift their limits might backup tweets beyond the 3200 limit.

Tweet Backup

http://tweetbackup.com/
So far has managed to retrieve 2080 of my historical tweets which are available in RSS (XML), txt & html format.

Tweet Backup

It works, but with almost 1000 historical tweets not retrieved, they possibly have some bugs to fix.

Tweet Scan

http://www.tweetscan.com
Claims to have retrieved tweets all the way back to Nov 2007 but there seems to be lots missing.

Backup your Twitter account. Your last 1000 messages are downloaded from Twitter and combined with results from our historical database. Tweet Scan can retain your tweets to improve later backups.

What I ended up with – 1076 Tweets

Tweet Scan

That is a screenshot of the html output in TwiddlyWiki format – they also provide CSV – the biggest failing? No dates.

My guesstimate is that of what were retrienved maybe 500 were from the front of my queue, and 500 or so from the history. Some were from earlier periods than found by other services.

Just a sec – 1075 tweets from Topsy search – 1076 from Tweetscan?

It wouldn’t surprise me if all Tweet Scan is doing is a backup of the tweets Topsy have available via API, or just scraping them.

Tweetake

http://tweetake.com
I couldn’t get this to work, it kept throwing up errors suggesting I tried again.

Twitter Safe

http://www.twittersafe.com/
Defunct

Backup My Tweets

http://backupmytweets.com/
Requires you to tweet for access for a year, or pay. Fair exchange, but I am not tweeting… yet.
They also highlighted on the home page that they could only manage 3200 tweets thus they don’t backup any more tweets than anyone else, though with a business model may last longer.

Twitter Backup

Twitter Backup
Allowed me to back up close to the full 3200 tweets (2993 to be precise) in XML format flawlessly in about 1 hour – whilst it asks for a password in the interface, it seems you can enter some random junk and it will use Oauth, thus the programmer is on the ball, though maybe needs to update that interface so it doesn’t even ask for a password.

Twitter Backup

With a desktop app you are not storing any information on a 3rd party server, and as this app is written in Java, you might be able to get it to work on any desktop.

MK Twitter

http://sourceforge.net/projects/mktwitter/
This seems to be fairly early stage – doesn’t use login at all and gets 3000 or so tweets which is all you can really expect with Twitter’s restrictions. Output stored in a .mdb file for further use.

MK Twitter

Close to 4000 Tweets

Between various combinations of backup service and possibly scraping Friendfeed or Topsy I probably have close to 4000 of my historical tweets. That is over 80%.
I have heard that Twitter have all of them – hopefully one day they will all be available within a searchable interface whether from Twitter, Topsy or Google (who knows Microsoft or Yahoo might surprise us)

But in the 6 months since I left Twitter, they still haven’t fixed their SEO

Soon… maybe even tomorrow I will tell you how to use and abuse Twitter’s current linking structure… for fun or profit.

  • Share/Bookmark
Posted in SEO Blog, Top, web 2.0 | Tagged , , | 4 Comments

5 Reasons Buzz Is Google In Startup Mode

By Andy Beard | Published: August 23, 2010

I don’t know whether to call the last few days “Leo Gate” or “Social Media Gate” or “Buzz Gate” but one thing it certainly highlights is how different the Google Buzz Team is:-

1. Responding rapidly to situations

Not just with a social media celebrity such as Leo, but with all users with problems. Quite often a user familiar with Buzz will just @message one of the Google Buzz Team if something needs some kind of interaction with an engineer to resolve, but quite often the Google Buzz team members get involved without being prompted.

2. Direct interaction

In many ways it reminds me of Friendfeed who were already in many ways looked on as underdogs compared to Twitter and only gained significant traction when Twitter was down. Just like Friendfeed the creators of Buzz are participating in the conversation and in many cases they are leading the conversation as well.
They are users, and creating a platform they want to use.

3. Scale & Community

Obviously Buzz isn’t yet at the same scale as Twitter – it is quite possible it is at a similar scale to Friendfeed. The funny thing is I saw lots of people writing smart comments in replies to Leo’s anguish such as “What’s Buzz?” as if it is insignificant.
There was also this comment in a post on Cnet

Only a few social-media services truly matter at the scale at which Google likes to operate, and Buzz is clearly not yet one of them.

The thing is Cnet isn’t relevant to Buzz – you will rarely see any Cnet content there. Actually you only see Techcrunch content on Buzz because people share it in a fragmented way, or syndicate tweets of Techcrunch to Buzz. Mashable has a much more focused strategy in place and you will regularly see 100+ comments to a Mashable post on Buzz, and they will often be far more interesting than the comments on the blog itself.

Ultimately with any software platform you don’t attempt to scale until you are ready for it. Buzz may not even remain within Gmail long-term, but I have a feeling it will have significant longevity, because Google are not siting on their hands.

4. Speedy of Implementation

Bugs seem to get fixed rapidly on occasion

Google Buzz Team – @Leo Laporte – Thanks for reporting this issue — and sorry we didn’t get to the bottom of it until today. You helped us uncover a very rare bug that has existed for a while, one that only someone with a ton of followers was likely to uncover.

Here’s what happened: If one of your followers deleted their Google Account (this probably happened around August 6th), Buzz failed to deliver your post to all of your followers. Your post still existed in your Buzz stream, it just wasn’t sent properly to the people who wanted to see it.

We’re in the process of fixing this bug now, and it should be resolved in the next day or two. We’re really sorry that you had this experience and really thankful that you reported it to us so we could fix it.9:00 pm

How dare Google ship a product with bugs – how dare they have bugs even 6 months later?

5. Free As In Open

This was pretty much the ethos of Friendfeed as well, but Google have a very much open approach with Buzz. This comment from DeWitt Clinton to me just typifies the approach.

DeWitt Clinton – @Andy Beard Also, we’re going out of our way to make it easy (and free) for anyone who wants to index or archive Buzz posts and comments to do so. We’re using standard Atom and ActivityStreams and PubSubHubbub — if someone wants to build a search engine around Buzz, just subscribe to the hub and off you go, no contract required:

http://code.google.com/apis/buzz/v1/using_rest.html#firehose

Stayed tuned for more features like this coming soon. This type of decentralized, open approach is central to the core values we have in mind for Buzz.

To be honest I am a little freaked out by the concept of a Google Engineer suggesting I build a search engine on top of their platform APIs and mentioned that a little thater in the conversation thread, but the concept intrigues me.

Update

Good to see that the new features in the search API have now been officially announced. (via Techmeme)

  • Share/Bookmark
Posted in Google, web 2.0 | Tagged , , | Leave a comment