This is all pretty basic technical SEO

Here is a very simple cleaned up Google search query that will bring up some interesting results

https://www.google.com/search?q=site:twitter.com+inurl:andybeard&filter=0

It doesn’t exclude a subdomain such as www.twitter.com or fr.twitter.com or api.twitter.com

Twitter splits their domain authority between lots of different subdomains which have no business being indexed.

It also doesn’t include http and https – typically those should also be canonicalized (think of Highlander “There can be only one!”)

The &filter=0 tells Google not to ignore some of the URLs it might otherwise due to no content, low PageRank etc. It is especially useful for picking up URLs which are blocked by robots.txt

Like these

The first red arrow is what Google sees when it decides not to follow the funky javascript redirects that Twitter does. It is possible Google sees that a lot – it is like having a door slammed in your face.

The second is the bouncers at the door to the nighclub… you get to see all the cool stuff entering into Twitter’s archives. but it won’t let Google through as they aren’t willing to tip the bouncers enough money, or haven’t got the right friends.

That barrier prevents Google crawling deeper into your content, so whilst if they were very observant they may have seen a piece of content once, it may eventually drop out of the index unless other sites in the Twitter ecosystem maintain links directly to that content that Google can follow.

So for instance sites like Topsy & Tweetmeme if crawled by Google and they link directly to a tweet, it is possible for Google to find content… but that is far from perfect.

What controls Google in this way is Twitter’s robots.txt file https://twitter.com/robots.txt

The line of that file in the Google section that is causing a lot of the issues is this one
Disallow: /*?

Effectively any URL on the whole of Twitter that contains a “?” or query parameter Google is not allowed to look at.

Here are 2 more queries for you

https://www.google.com/search?q=site:andybeard.tweetglide.com/blog&filter=0&start=991

https://www.google.com/search?q=site%3Atwitter.com%2FAndyBeard&filter=0&start=991

Those are 991 searches… Google will only list up to 1000 items for a search, so that kind of query will show the end of the search results – if used on huge sites, then you might have to refine things down to a subset of pages where possible, but in this case my Twitter account only has 5100 tweets and Twitter should easily be able to get all of those indexed. I am restricting the search to a folder /andybeard

I haven’t linked directly to either my twitter account or my archived copy on Tweetglide for some time so here they both get a link with a screenshot of current indexation over 2 years… not so many tweets over that time due to a long hiatus.

http://andybeard.tweetglide.com/blog

http://twitter.com/andybeard

341 vs 321

The winner here seems to be Tweetglide, and it seems fairly close until you examine all the URLs for Twitter that google crawls needlessly as duplicate content on different subdomains, and remove all the junk pages (and some good stuff) that are blocked by robots.txt

Such as this

You also have to understand that in the last 2 years due to my hiatus I have only created 280 tweets as archived by Tweetglide (there may be a few early ones missing), and the additional tweets in that deep search result are the archived Tweets of the people I have conversations with.

If we removed that filter parameter things are drastically different

https://www.google.com/search?q=site:andybeard.tweetglide.com/blog&start=991

https://www.google.com/search?q=site%3Atwitter.com%2FAndyBeard&start=991

341 vs 253

That is when Google filter out lots of the duplicate junk from Twitter, and none from Tweetglide.
That filter removal isn’t perfect.. there are still some duplicates – if Twitter retains half the indexation of Tweetglide I would be amazed.

Is there a crawl limit?

It vastly depends on juice.. my Twitter profile at one time had enough juice that if Google had been allowed to crawl, they probably would have picked up 25-50% of the 5100 tweets, but Twitter doesn’t allow you to paginate that far into it’s archive (even if it wasn’t blocked), and even the API is still limited and can only pick up around 3000 historical tweets.

My good online friend Vlad Zablotskyy has more tweets than me achived on Tweetglide.

https://www.google.com/search?q=site:vladzablotskyy.tweetglide.com/blog&filter=0&start=990

He actually has less pages indexed than me possibly because of juice, so lets give him some, and see if we can crack the 500 barrier.

This has been a small introduction to Twitter’s SEO woes to demonstrate hopefully to laymen that all is not well with Twitter, and any claims that they can be indexed normally are false. Any competent SEO could have found all of these issues with the site, and fixing them would reduce the load on Twitter’s servers caused by Google, and maybe allow Google to index more content.
I have avoided additional complications with rel=”nofollow”, and potential cloaking issues with their impelentation of #!hashbang URLs, and funny javascript redirects and haven’t touched on some additional nuances with the way they feed juice to list pages.

Disclosure: When Tweetglide launched I offered some SEO tips (pro bono) over a few days to the owner and one of his engineers, and had 100% “buy in” to follow my recommendations for internal linking structure on the site. I would possibly change the structure of the pagination links at the bottom but otherwise I think the site from an SEO perspective is doing great… damn… no difference between with and without filter=0 actually amazes me.

At least if you look at it from a search engine perspective, who should assume that every visitor isn’t a member of the site they are referencing in the search engine.

It should also be understood that both Facebook & Twitter are bursting at the seams with former Google engineers & execs – they can’t claim they were unaware of what Google is looking for from content owners on the web, webmaster guidelines etc.

Twitter

You can’t look at the Google cache and see exactly what Google sees, because they do some sneaky redirects which are very akin to cloaking.

I have written about this before.

Video Exclusive: Has Google Given Twitter a Cloaking Penalty?

This is what Google sees based upon the preview

The little piece of text at the top of the page is what amounts to your profile… you can’t count the background image if any because it can’t be read by Googlebot unless it works really hard using OCR, and certainly can’t be read by people with disabilities.
The links within the content of the page are mostly nofollow, and the links in the sidebar get blocked by robots.txt.
The link at the bottom of the page to access more content… which may be of interest to search is also blocked by robots.txt.

I am not the only one who has spent considerable time trying to get Twitter fixed. A great example is this post by Vanessa on Search Engine Land.
How Twitter’s Technical Infrastructure Issues Are Impacting Google Search Results

Facebook

Facebook is worse

There is nothing there of any real value… it isn’t the timeline a logged in user might see.

First Click Free

If you want to have some kind of membership wall for users, then Google have special arrangements where you are required to show content for the first click.

Cloaking

Google over the years have published lots of content about what they think of cloaking.

I can still think of a few cases where some kind of cloaking would be justified. As an example on uQast we serve RTMP video with flash and use javascript “cloaking” to provide mp4 for iPhone. We could even serve that video to Googlebot’s mobile crawler without breaking Google guidelines as “cloaking” to serve content to specific browsers is allowed. But we can’t serve Googlebot which crawls for the main search index something it understands, as the Google guidelines require you treat Google as a normal desktop user browsing from California in the USA.
So Googlebot is served flash based RTMP within the webmaster guidelines rather than something it might like to see which we would be quite happy to give it.

That doesn’t prevent Google sometimes (though rarely) indexing the mobile video by figuring out the javascript, but it would be so much easier to give them something they understand.

Google Isn’t Playing Fair

One area that Google isn’t necessarily playing fair is that I don’t seem to be able to view Google+ profile pages in their own cache, and they don’t give a preview of the page that Googlebot sees.

This is my Google+ Profile

You can normally search in Google for cache:https://plus.google.com/102279602913916787678/posts or any url to get a cached version of what the crawler sees.
It is possible for every site to tell Google and other search engines not to store a cached page, so Google are well within their rights not to do so… but it prevents comparrisons.

Compare
cache:andybeard.eu – brings up a cached result
cache:https://plus.google.com/102279602913916787678/posts – does not bring up a cached result, just a 404 error

FTC Complaint over Search Plus Your World

The blogoshere love a good witch hunt, but I can’t see that Google is treating Twitter or Facebook unfairly. Eric Schmidt was quite right about some of the nofollows, but there are bigger technical restrictions in place on crawling.

I actually quite like a Google profile as a default profile and identity on the web, but Google need to live up to the promise of salmon and make it a viable endpoint for all activity, or as an alternative use it for identity, and allow me to define my own default profile.. which if I choose might be Twitter or Facebook.
I can also understand why you wouldn’t undertake the complex engineering to make such flexibility possible for your first itteration, especially with partners who are unwilling to do something similar themselves.

Just ask Twitter how many content partners they now support on the new Twiter for embeds. (I wrote them a letter a year ago and never received a response)

This post ignores what a logged in and fully javascript supporting human might experience, but in many ways Google’s profiles whilst now having a social element for years have generously linked out to any other online destination of your choosing, and provided the necessary markup to claim them as being part of your personal social graph.

Update – Google Profiles Now Cached

Michael VanDeMar left a comment showing a way to get the cached page to show by including the https protocol at the beginning of the url to query.

However when I posted I had tried lots of different variations all resulting in a 404 error.

This unmodified link was previously bringing up a 404 error
cache:https://plus.google.com/102279602913916787678/posts

It now returns what appears to be a blank page – as Michael points out if you switch off the CSS in your browser you can see the complete cached landing page.

Click to view full size without CSS

This appears to be a recent change, though they still need to fix the canonical – the canonical changes as you navigate between tabs and between the first 2 urls on this list there is effectively a redirect loop with /posts claiming / is the canonical, but humans are redirected to /posts

https://plus.google.com/102279602913916787678/

https://plus.google.com/102279602913916787678/posts

https://plus.google.com/102279602913916787678/about

All the different URLs show all of the same content, so should set whichever canonical a human is redirected to which currently is /posts

Not Total Fix

It seems some other pages are still giving 404 errors – maybe due to all the funky redirects going in circles with the canonical on occasion (this query is with HTTPS)

If you have difficulty understanding the concept of canonical, it is just like Highlander… “There should be only one” page with the same content in Google’s index, especially on the same domain.

But one thing I haven’t read about anywhere are the technical implications of +1 and the way it deals with canonicalization of votes, which currently is significantly inferior to both Facebook and Twitter.

There are all kinds of reasons why a publisher or ecommerce store might want to include parameters in the URLs people bookmark.

• Various kinds of tracking data
• Geotargeting
• Dynamic landing pages based upon advertising campaign (even with Adwords)

A user lands on a page, and you want to ensure that whatever they bookmark or share with friends or family is the exact same content/offer that they saw themselves, or that the initial source for the traffic is correctly counted to evaluate advertising campaigns.
This is especially important with affiliate marketing if you are a vendor who wants to ensure that affiliates get attribution for the traffic/customers they send.

• You see people use multiple browsers on one PC, and cookies are not necessarily shared
• Then you get people who have multiple computers in their home
• Then you have tablet PCs… someone might find a product on their desktop PC, but when they want to share the product with a significant other before making a purchase, they might do so on their iPad, loading a page from shared bookmarks.
• As mobile browsers become more powerful, a phone could also be the device used for bookmarking.

I don’t think it is possible to get everything perfect, to get correctly attributed tracking, ensure bookmarks are handled correctly etc (especially when paramters that remain visible can affect conversion), but I personally feel it is important to do whatever you can to ensure valid tracking data and attribution.

Facebook & Twitter – 2 Different Methods

Both Facebook & Twitter allow you to define tracking parameters or links for their share buttons, but they do it different ways.

With Twitter it is defined within the Twitter button by defining the url and counturl parameters.

The URL is whatever link you want shared, and the most popular way to use it is probably a tracking service such as bit.ly, but at the same time adding additional Google Analytics tracking variables to the URL that is bookmarked within bit.ly.

This is what the Twitter code looks like

					<script src="http://platform.twitter.com/widgets.js" type="text/javascript"></script>
					<a href="http://twitter.com/share?url=http%3A%2F%2Fwelcome.uqast.com%2Fpage13312&counturl=http%3A%2F%2Fwelcome.uqast.com%2Fintro%2F&text=How To Profit By Leveraging The 7 Hidden Trends Driving The Internet&count=horizontal&via=uQast&related=uqast%3AWebinar" class="twitter-share-button" rel="nofollow">How To Profit By Leveraging The 7 Hidden Trends Driving The Internet</a>

And this is the result (this uses my affiliate link for uQast, but notice that the count shown is an aggregate canonical value because of the defined counturl

How To Profit By Leveraging The 7 Hidden Trends Driving The Internet

With Facebook it is a little different – within the button code you include the tracking link you want to share, and the “counturl” is actually grabbed from the Facebook Open Graph meta data on the destination page.
Facebook also pull in the thumbnail and descriptions from that meta data.

This is what the code looks like

<script src="http://connect.facebook.net/en_US/all.js#xfbml=1"></script><fb:like ref="13312" href="http://welcome.uqast.com/page13312" send="false" width="120" show_faces="false" layout="button_count" ></fb:like>

The result is an aggregate count even though every shared URL from the like button could be completely different.

Google +1 Only Gets Half A Banana

As Google +1 has only just been launched, the uQast landing page hasn’t received 100s or 1000s of bookmarks but it is a good example of the current problem with Google’s implementation of the +1 button.

This is the same URL we were using in the above example, my uQast affiliate link, but it could be any tracking link, or just using Google Analytics tracking parameters.

http://welcome.uqast.com/page13312

This is how that button should be encoded

<script type="text/javascript" src="http://apis.google.com/js/plusone.js"></script>
<g:plusone size="tall" href="http://welcome.uqast.com/page13312"></g:plusone>

With this result… I gave it a plus one to test this earlier as I am considering adding Google +1 to our landing pages, not just for our launch signup, but also throughout uQast and within our embeddable players.

If Google had implemented +1 correctly, then the count for a URL that points directly to the page would be the same.

<script type="text/javascript" src="http://apis.google.com/js/plusone.js"></script>
<g:plusone size="tall" href="http://welcome.uqast.com/intro/"></g:plusone>

At time of writing the affiliate link shows 1 and the “clean” link shows 0 – I am sure that will change over time

This is how a +1 ends up on a Google profile (this is my primary Andy Beard profile)

Lets see how Google fairs with my +1 test

First of all my profile URL is

https://profiles.google.com/list.andy

But from a search result Google themselves redirect to

https://profiles.google.com/list.andy/about

And if I want to ensure the English version of a page is shown then I have to add an additional language parameter.

https://profiles.google.com/list.andy/about?hl=en

These redirects are a fact of life on the web and so are additional parameters for all kinds of things, but Google +1 doesn’t allow for it, even for a Google URL

Google +1 is broken even for Google’s own pages

In addition they really should take advantage of the thumbnail defined in the Facebook Open Graph, as they currently just scrape the first image on the page.

The Solution For Google +1

It is actually fairly simple for Google to fix this.

They are already looking at the destination page, and all they have to do is look for a canonical link tag in the header for which URL to count… anyone who cares about correct canonicalization for their pages who uses various tracking parameters is likely to already be using link canonical… after all it is stronly promoted by Google themselves as a way to get attribution for the correct link.

<link rel="canonical" href="http://welcome.uqast.com/intro/" />

You would have thought issues like this would have been fixed during the initial beta period – when news of Google +1 first came about, there was a call for interested websites to sign up early maybe to give feedback… I signed up at that time but from the looks of things on the Google announcement post that was mainly as a PR exercise to get large publishers using Google +1.

Google +1 is really easy to set up, there are also some quite powerful features I haven’t explored yet for triggering various actions similar to a Facebook like button

However as currently implemented it is almost like a negative advert if you implement it with any kind of dynamic URLs showing effectively the same content.

Canonicalization is just as important in social media as search.

p.s I have given up trying to get a “+” to appear correctly with a retweet button with WordPress.

I went as far as trying to use the unicode version within the title but that ended up as the raw text, and then WordPress didn’t save the unicode anyway.

+1.... out

Watch this video for the details

To check the cached Google page you will have to switch off javascript using something like the web developer toolbar for Firefox.

Google is not being blocked as there is a cached page, but the cached page is not what is given to users, even when not logged in.

The pages are significantly different in structure.

Google have quite extensive webmaster guidelines including a special on on cloaking and sneaky javascript.

Just to qualify the value of this video – some people rightly or wrongly look on me as an SEO expert – I don’t proactively consult partially because I don’t believe my own opinions half the time and in the past I have spent a great deal of time trying to prove myself wrong.

Ultimately even Google engineers directly involved with webspam or working directly on Google’s algorithms don’t know every aspect of how Google’s algorithms might work in every eventuality, and some results or side effects may be unintended.

It could well be that Twitter left old Twitter in place because Google was having problems crawling the new primarily javascript Twitter. From my perspective the mistake was to do a funky javascript redirect and having a significantly different link graph, but there could easily be something I have overlooked.

Here is the evidence

That is frightening… the chance that I was hacked as well.

The Trail Of Evidence

• My son was in the other room
• My wife’s laptop is open with a Youtube video playing Big Green Tractor
• The tweet was sent via one of the new tweet buttons and has @youtube – not a guarantee my account hasn’t been hacked, but a fair chance some legitimate “interaction” has taken place.
• He normally uses Chrome not Firefox – it is possible I have logged in to Twitter on my wife’s laptop and saved the cookie

On the laptop he is not logged in to Youtube as me – he has his own Youtube account with a playlist of songs he likes (other than the ones that have now been moved to Vevo which we can’t access)

Verdict:- A 3 Year Old Tweeted A Miley Cyrus Video To 10,000 Marketing Geeks

I still can’t work out how a 3 year old can:-

• Click a Share button below a video
• Then click a specific Twitter button to actually tweet
• Then somehow “send” the tweet

It is possible he found a link somewhere on Youtube that was formatted to Tweet the video, or even an advert.. he clicks those all the time – I sometimes feel guilty but he is a consumer, a future buyer and “get’em early” – he has good taste in cars.

Maybe hot keys?

Anyway… no I am not back on Twitter and it seems my son likes older women.

Youtube & Twitter both have an interface even a 3 year old can use.

What I am about to reveal in some ways is akin to publishing 0-day security stuff, and some SEOs will look on this as revealing things that they have been happy to exploit for months – thus some of this is fairly widely known, but maybe you don’t.

I will also include some thoughts on “the reset vector” and “evaporating PageRank” in later parts (my views radically differ to those widely publicised – even at recent conferences)

Lots of this article series is not based upon verifiable facts or even poor attempts to construct tests that would be ripped apart anyway. I am just throwing my thoughts onto the interwebs.

I have decided not to do this as a video – some of my loyal readers have accessibility problems with videos and I don’t want to exclude them.

This is a little technical and some aspects would be highly dependent on how Google treat internal linking within Twitter. It is hard to exclude external factors as there are very few ways to create a controlled experiment and there are so many possible ways to filter trust based on influence within the Twitter ecosystem and the whole social graph.

Robots.txt

#Google Search Engine Robot
User-agent: Googlebot
# Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl
Disallow: /*?
Disallow: /*/with_friends

This code prevents Googlebot crawling your historical tweets
Individual tweets if they get some juice from elsewhere can be indexed, but twitter aggregators tend to nofollow links to the content they repurpose.

You would think as Google crawl robots.txt on every visit, they would always have a copy of the robots.txt available… well unless you used x-robots in some way to nocache it.
You would also expect it to be reasonably up to date.

However here is a link to their current robots.txt as visible to humans, and it matches the cached version, and the version I have on this page above.

Interesting Factoid

Twitter doesn’t curently rank in the top100 Google results for Microblogging
http://www.google.com/search?q=microblogging&pws=0&hl=en&gl=US&num=100

So What Does Twitter Want To Rank For?

Quite probably Twitter’s only goal is to rank for your name or your brand.

In fact if you don’t have a Twitter account for your brand that is actively used (it is fairly safe to squat), it is highly likely that you will never have Twitter appear with realtime results for brand searches in the main SERP unless you are a trending topic.

Once you invite social media into your house, it is hard to get rid of.
(it isn’t as bad/good as it was a few months ago)

I noted to some friends when Ikea was having some social media unrest, and had just announced something so lots of press attention, they still didn’t have a scrolling realtime search box appearing in their brand search engine results, whilst it might only take a mention of a popular Twitter user’s name a few times, maybe a few retweets of a blog post for it to normally appear.

I assume Ikea own their trademark account or someone is being very careful squatting it (maybe even Twitter management) – it has PageRank – it has some followers – but it isn’t active.

The Twitter account currently ranks ~181 in the US search results on Google, just behind quite a recent article on Huffington Post – oh and I just linked to it.. oops.

Considering the strength of the brand, that is a pretty strong ranking without any further support such as the link I gave it.

Real Time SERPs

There have been some changes to realtime SERPs recently which Dave covered.

Here you can see that the people attending the Mozinar went to bed fairly early (must believe in the witching hour on the NW coast of the US, or the bar closed)

However unlike a few months ago it is very hard to get realtime results to appear in the main search results pages.

Stay tuned for part 2 – we are going to look at the internal linking structure, and how to leverage it to maximum effect either for your own brand, or someone else’s.

I know some people look on SEOs as spammers – some are – I look on my job as helping people find content they are looking for. I am passionate about it. I wouldn’t class myself as white grey or blackhat, but I also try not to do anything that would pollute the web.

It is one of the main reasons I left Twitter. The SEO strategy in place at Twitter was holding my content hostage from normal indexation by Google, and Google have so far failed at indexing historical content.

Topsy Search

I welcome the news yesterday that Topsy have improved their search & Danny has more in-depth coverage.

Topsy has some great search functions – here are some example searches

Influencers

It is very easy to work out who are your biggest Twitter fans, though there is no way to sort by number of tweets. They have some kind of # tweets x influence calculation.

I really haven’t promoted my posts that much compared to many, considering @andybeard was always my official channel.

This method of sorting hides sock puppet bot accounts run by you or others (I don’t have any pimping this site)

Sphinn Pimps

Often people are complaining about excessive self promotion on social media – Topsy makes it fairly easy to work out who is pimping what on sites like Sphinn.

Historical Tweets

How many tweets does Topsy have in their archive from me?

1075 Tweets (you will see why this is significant later)

Friend Feed

I have a lot of content piped into Friendfeed since it launched but you can’t trust Facebook to keep Friendfeed online, especially the search which for me was one of Friendfeed’s best features.

That was only a temporary glitch – Friendfeed search is working again now, but I didn’t see lots of screaming and blog posts about the demise of Friendfeed search

Finding My Tweets

Can I find the tweets I want to find? No

The alternative is I can back up my own Tweets and then do something with them…. well at least some of them

Twitter only allows your last 3200 tweets to be backed up in any way.

Here are some options

Online Twitter Backup Services

Online services in theory once you join will continue to backup future tweets, and if Twitter ever lift their limits might backup tweets beyond the 3200 limit.

Tweet Backup

http://tweetbackup.com/
So far has managed to retrieve 2080 of my historical tweets which are available in RSS (XML), txt & html format.

It works, but with almost 1000 historical tweets not retrieved, they possibly have some bugs to fix.

Tweet Scan

http://www.tweetscan.com
Claims to have retrieved tweets all the way back to Nov 2007 but there seems to be lots missing.

Backup your Twitter account. Your last 1000 messages are downloaded from Twitter and combined with results from our historical database. Tweet Scan can retain your tweets to improve later backups.

What I ended up with – 1076 Tweets

That is a screenshot of the html output in TwiddlyWiki format – they also provide CSV – the biggest failing? No dates.

My guesstimate is that of what were retrienved maybe 500 were from the front of my queue, and 500 or so from the history. Some were from earlier periods than found by other services.

Just a sec – 1075 tweets from Topsy search – 1076 from Tweetscan?

It wouldn’t surprise me if all Tweet Scan is doing is a backup of the tweets Topsy have available via API, or just scraping them.

Tweetake

http://tweetake.com
I couldn’t get this to work, it kept throwing up errors suggesting I tried again.

Twitter Safe

http://www.twittersafe.com/
Defunct

Backup My Tweets

http://backupmytweets.com/
Requires you to tweet for access for a year, or pay. Fair exchange, but I am not tweeting… yet.
They also highlighted on the home page that they could only manage 3200 tweets thus they don’t backup any more tweets than anyone else, though with a business model may last longer.

Twitter Backup

Twitter Backup
Allowed me to back up close to the full 3200 tweets (2993 to be precise) in XML format flawlessly in about 1 hour – whilst it asks for a password in the interface, it seems you can enter some random junk and it will use Oauth, thus the programmer is on the ball, though maybe needs to update that interface so it doesn’t even ask for a password.

With a desktop app you are not storing any information on a 3rd party server, and as this app is written in Java, you might be able to get it to work on any desktop.

MK Twitter

http://sourceforge.net/projects/mktwitter/
This seems to be fairly early stage – doesn’t use login at all and gets 3000 or so tweets which is all you can really expect with Twitter’s restrictions. Output stored in a .mdb file for further use.

Close to 4000 Tweets

Between various combinations of backup service and possibly scraping Friendfeed or Topsy I probably have close to 4000 of my historical tweets. That is over 80%.
I have heard that Twitter have all of them – hopefully one day they will all be available within a searchable interface whether from Twitter, Topsy or Google (who knows Microsoft or Yahoo might surprise us)

But in the 6 months since I left Twitter, they still haven’t fixed their SEO

Soon… maybe even tomorrow I will tell you how to use and abuse Twitter’s current linking structure… for fun or profit.

Guess what Leo? It happens on both Buzz & Twitter – Facebook as well. They couldn’t scale and cope with millions of followers & friends without it.

Interactions

So Leo is missing the interactions he used to get when Buzz first started.

I think part of the problems is that everyone has added their Twitter, Friendfeed, refeed of Tweets via Friendfeed, Google Shares, refeed of Google shares from Friendfeed etc. You can also add extra laters and add in your Mybloglog feed of your Friendfeed for good measure.

At the same time most of those people have left Google Buzz – they left their litter behind but they are not there interacting, thus even a online media celebrity such as Leo gets drowned out.
And if he didn’t, he would probably drown out all other conversations like happened with many celebrities early in the life of Buzz.

Google has to get Salmon working with true canonicalization & threading of all these conversations fast – I would also suggest removing share totally, and just using a “like” equivalent for now. Encourage one canonical conversation.
You will still get some fragmentation due to Google Reader – but reader should never have had conversations anyway ;p

Memories

I am old… very old… 4 decades old. (that is just a decade short of half a centuary)

For me remembering things is important.

I am naturally reasonably good at remembering certain kinds of information. I remember conversations I have had on Twitter from a year ago, even 2 or 3 years ago.
Sometimes however the details are a little vague and I want to find them again… and I can’t.

They have disappeared into a black hole and I have no way to retreive them.

I described Twitter as having Alzheimers 6 months ago when I left Twitter

Friends keep on asking when I am coming back – the answer is when I can find my conversations, otherwise they are meaningless to me.

I can probably find syndicated copies of my Tweets on other services, but that isn’t the same as you lose context, plus those services are most likely on life support – who really expects Friendfeed to be around for another 12 months?

Whilst the number of interactions I have on buzz are significantly less, part of the reason is that I don’t push for interaction there.
I have seen friends like Chris Lang absolutely killing it on Buzz – he drives traffic to buzz from his email list, he pulls in experts to conversation by referencing them (which at the same times promotes these people to his audience) and he hasn’t seen a slow down in conversations because Google Buzz is his chosen battle ground or stomping ground and he is leveraging all its possibilities.

Everything you publish on Buzz still exists

You can search on it using is:buzz in Gmail and you will always be able to find it.
Finding conversations other people are having about your content on Buzz isn’t easy, and a huge negative with Buzz is that it takes forever to import an original post.
Often it takes 2-3 hours for a fresh blog post to appear on Buzz as an imported feed – that cripples Buzz.

To hell with whether the content you create is syndicated or not, or whether there were responses. Well responses are nice, but ultimately unless there is a long term record, what you wrote didn’t even happen.

Just like “Pictures or it didn’t happen” at least with Google Buzz the words are recorded.

Imagine in a few years you want to find something you said – on Twitter you won’t be able to because their search sucks, and they block Google with robots.txt – the import of backdated Tweets into Google just doesn’t seem to be happening and it is quite possible the tweets no longer exist.

Would failing to store tweets be the social media equivalent of burning books?

Facebook’s Achilles Heel

Without a doubt it is search – finding conversations on Facebook is just a pointless exercise which is why I can’t understand why it is popular for family interaction. I can understand it for students, but I still reference email exchanges with my family from 5 years ago.
I would never entrust those kinds of exchanges to Facebook.

Just Social Media Magic Roundabout?

People just hop from one platform to another like the seasons – hell I just submitted someone else’s content to Sphinn for the first time in 2 years and have no idea why I did it, other than lots of the people I communicate with daily are still there and the story was actually news (plus it linked to me :) ) – if you are a YouTube search optimization freak you might want to take a look.

Feel free to comment on Buzz as well and Chris has an active conversation with back story here.

I wonder when will be the first time a buzz conversation gets listed on Techmeme.

This mainly covers WordPress – if you are on another platform, find a geek that uses it to try to do the equivalent, but I doubt this is possible on Blogger.

I should point out there are already implementations out in the wild that are significantly better in some ways, though might not meet all the goals of the Twitter developers.

As an example, in my comments on the last post one of the goals of the Twitter button was for it to work even with javascript switched off. Some existing sharing solutions use an empty anchor link href=”#” – from a SEO perspective that is better – for usability maybe not.

Here is some default code generated by the official Twitter button generator – the data-url is optional if you are using canonical tags and don’t have lots of buttons on the home page of a blog, or within archives – otherwise it is best to be specific.

<a href="http://twitter.com/share" class="twitter-share-button" data-url="http://andybeard.eu/2902/7-reasons-not-to-use-the-new-tweet-buttons.html" data-text="7 Reasons Not To Use The New Tweet Buttons" data-count="vertical" data-via="AndyBeard" data-related="seodojo:SEO Training">Tweet</a><script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>

We are going to try to solve a number of issues

Twitter decided to use “data” attributes that are in the HTML5 spec. On one project I am using data attributes with spans which are not in the spec.
What some share widget developers are using is an empty anchor link with no destination.

If Twitter wanted the link to function as well as a defined button if Javascript is turned off, they should have opted to use their implmentation which uses URL parameters.

In my last post criticising the tweet buttons, I have added some test implementations of what is discussed here. Test the 3rd one “With Nothing Defined” – that is a text link as implemented by Twitter, it doesn’t pick up the title.

I should also note that Google have used HTML5 data for their buttons by default, so they suck just as much.

This is their current static text link for sharing

http://twitter.com/share

It picks up the referring URL, but it doesn’t seem to pick up the page title as it is meant to when using a text link (it works with Javascript), and you would somehow need to work out defining rel=”me” in the header which is something I have never even heard of before.

So that is the page you are sending “link love” or “Google Juice” to – a crappy form on a social network – this is going back to 2006 when I was first moaning about plugins doing this.

I was then compelled to make my own version of the Socialble plugin “Antisocial” as the original author refused to add nofollow, and eventually Yoast took the Sociable project over and one of the first official things he did was add nofollow to the links.

If they want the link, there needs to be some content of value on the page.

However we can do something about the link formatting to ensure it passes some useful data.

http://twitter.com/share?url=http://andybeard.eu/2902/7-reasons-not-to-use-the-new-tweet-buttons.html&text=7%20Reasons%20Not%20To%20Use%20The%20New%20Tweet%20Buttons&count=vertical&via=AndyBeard&related=seodojo:SEO%20Training

All those parameters have been added to the URL, which has a benefit with W3C validation & content syndication, and when someone clicks through to the form, it has the data filled out.

Whoopie!

So lets take that link code and slap it into the original

<a href="http://twitter.com/share?url=http://andybeard.eu/2902/7-reasons-not-to-use-the-new-tweet-buttons.html&text=7%20Reasons%20Not%20To%20Use%20The%20New%20Tweet%20Buttons&count=vertical&via=AndyBeard&related=seodojo:SEO%20Training" class="twitter-share-button" rel="nofollow">You should Tweet This Post</a><script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>

I also added a nofollow to the link, but even then it still means Google could opt to take juice away from other links on the same page. Sucks doesn’t it.
However Twitter are also gaining nothing of value to them, and even without it they possibly wouldn’t because of how they messed up their Robots.txt

Part of me still wants to believe that Twitter added this Robots.txt just to block indexation of search results, and had no intention of holding my content to ransom by preventing Google crawling it.

http://twitter.com/robots.txt

#Google Search Engine Robot
User-agent: Googlebot
# Crawl-delay: 10 — Googlebot ignores crawl-delay ftl
Disallow: /*?
Disallow: /*/with_friends

#Yahoo! Search Engine Robot
User-Agent: Slurp
Crawl-delay: 1
Disallow: /*?
Disallow: /*/with_friends

#Microsoft Search Engine Robot
User-Agent: msnbot
Crawl-delay: 10
Disallow: /*?
Disallow: /*/with_friends

# Every bot that might possibly read and respect this file.
User-agent: *
Disallow: /*?
Disallow: /*/with_friends
Disallow: /oauth
Disallow: /1/oauth

The 4th line down blocks any URL which has a parameter in it. It is a side effect of messing it up or being greedy (or incompetent) – various SEOs would describe it different ways.

Sometimes Google allows blocked URLs to accumulate PageRank – sometimes they don’t – in a case where that PageRank might end up somewhere that isn’t a benefit to Google depending on how they handle their “reset vector”, they may discount the juice.
I don’t know the order Google process the link graph – it is quite possible there would be a difference between linking to a page blocked with robots.txt, and linking to the same page using nofollow – that difference could be for both sites – Google has no intention of enlightening us on this.

This is especially so as the page has no unique or relevant content – the static text link does not display any of the information that is available from the widget, and neither does the connected page.

In my opinion the page should be merged with the search results for references to the URL – that would then be a worthwhile page to index especially if it also included all the threaded conversation.
There is no logical reason to have 2 actions from the same button – the destination page should have both the tweet form and the search.
If it was on a real permalink, that is something of value to be indexed.

I actually have no idea whether Twitter even care about SEO – the actions they take seem almost random – search other than on their own brand accounts for less than 2% of their web traffic according to Alexa.

The Javascript

The Next Web have posted some async javascript code. It makes sense to use it, but as you should have the javascript for the Twitter widgets loading at the bottom of all your pages, it doesn’t really make a huge amount of difference.
At this time Twitter are loading the javascript from a CDN, so it shouldn’t be affected by fail whale.
If Akamai dies, you might as well stop browsing as half the internet will die with it.

<script type="text/javascript">
//async script, twitter button fashiolista.com style
(function() {
var s = document.createElement('SCRIPT');
var c = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://platform.twitter.com/widgets.js';
c.parentNode.insertBefore(s, c);
})();
</script >

If you have this code in the footer of every page, then all you need to load a button is a static link, or one created using PHP if you are using WordPress.

I am not going to create a whole WordPress plugin here… so how about a quick WordPress shortcode.

There are tons of Tweet Button plugins appearing, but I haven’t seen any that do a decent link implementation which is essential for a shortcode – after all blog posts get syndicated… and javascript generally gets stripped out.

<?php
// Usage tweet rel="scobleizer:Tech Geek" or tweet rel="mashable:Tech Blog"
function mytweet($atts) {
extract(shortcode_atts(array(
		"rel" => 'seodojo:SEO Training'
	), $atts));
$via = "AndyBeard";
$url = urlencode(get_permalink());
// Different Themes & Plugins implement titles in different ways - find a way to code this that works for you
$tweettitle = urlencode(the_title("", "", false));
$related = urlencode($atts['rel']); // this should maybe be defined in some custom meta

$code = '<a href="http://twitter.com/share?url=' . $url . '&text=' . $tweettitle . '&count=vertical&via=' . $via . '&related=' . $related . '" class="twitter-share-button" rel="nofollow">You should Tweet This Post</a>';
return $code;
}
add_shortcode("tweet", "mytweet");
?>

To use this it would be something like

[tweet rel="seodojo:SEO training"]

That comes out like this

Really to be useful it should include a dynamic graphical image so when content gets syndicated there is some social proof with the link… but at least the link will now work when it is syndicated.

I still need to work out what data-counturl that I spotted on techcrunch is for. I am assuming that it can be used to set a canonical URL, while the URL used for each Tweet could be different (based on source, affiliate, membership status etc)

If you click this link, you will be taken to this same page, but residing at a slightly different URL. Tweetmeme makes a mess of URLs like this.
http://andybeard.eu/2907/better-twitter-tweet-button.html?mysource=12345

The new Twitter buttons and Topsy seem to get it right on the landing pages, I need to do more testing on tweeted URLs.

The Twitter retweet code places a link to Twitter on every page you include it, and they didn’t add a nofollow to the link. It is quite possible Google will decide to ignore all these links in the future, especially as it is effectively hidden and not a “vote” for a particular page. The destination of the link doesn’t show similar information to what is in the button.

2. History

The button has no historical count of tweets as is clearly evident if you search for tweets on something “historical” such as an individual domain. If I added a Tweet button to my home page the link through to search for Tweets would be something like this.

http://twitter.com/#search?q=http://andybeard.eu

3. Count Accuracy

If I glance back at Shaun’s post, the tweet button currently says 23 Tweets – clicking the button to see those tweets results in only 17 tweets showing.

Here are just the influential Tweets counted by Topsy, half of which Twitter hasn’t found even though they actually use the button to submit their Tweet.

Here is a link for the Twitter search results to see if they ever catch up with Topsy.

4. Longevity

Lets look at those links for a second

http://topsy.com/www.hobo-web.co.uk/seo-blog/index.php/new-twitter-button-conveniently-without-nofollow-attribute/?infonly=1&sort_method=influence

http://twitter.com/#search?q=http%3A%2F%2Fwww.hobo-web.co.uk%2Fseo-blog%2Findex.php%2Fnew-twitter-button-conveniently-without-nofollow-attribute%2F

Topsy uses a permalink – there will be a permanent record of all of those Tweets, and their history goes back a long way, and even copes with 301 redirects to some extent if you change your permalink structure. I haven’t experimented with this extensively, but that has some useful potential for marketers adding parameters to links.

For Twitter the link is one of their funky search URLs which uses a # named ancor that is ignored by search engines – Twitter search is totally unreliable, and only has a 7 day history – do you think you are going to have a record 1 month, or 1 year down the line?

5. Broken Retweets

The new “expanded” links on the web interface get truncated – if you retweet using copy/paste links will quite often end up being broken.
This is a subtle way of enforcing retweets using their “new” official retweet method which still isn’t as popular as Twitter would like people to believe.
It also defeats the purpose… you can’t see the whole URL, sneaky redirects or affiliate tracking parameters, and for many URLs might even hide what the URL is actually about, or where you might end up.

6. Validation

Shaun is pretty hot on W3C validation of websites though maintaining it on content that is being published in response to news is a challenge.
One of the problems with the original Tweetmeme button was validation so I am hardly being a validation nazi here.
http://validator.w3.org/check?uri=http%3A%2F%2Fwww.hobo-web.co.uk%2Fse o-blog%2Findex.php%2Fnew-twitter-button-conveniently-without-nofollow- attribute%2F&charset=%28detect+automatically%29&doctype=Inline&group=0

Twitter decided to invent their own link attribute – that sucks – they will never manage to get that accepted as valid code, as it is proprietary to them and doesn’t serve any purpose. This was one of the major complaints against rel=”nofollow” which took 4 years to get any kind of official acceptance.

7. Data

He who has most data in online marketing often has a huge advantage. By adding the official Twitter button they suddenly gain a whole load more information about who visits which sites, which can be tied into their member profiles.
I can’t currently see a reciprocal benefit in Twitter having access to that data.

I gave Twitter a 7 day Ultimatium – now 6 months ago
I strongly criticized their search and the ability to find my own content
I ripped apart their SEO efforts

I left Twitter 6 months ago because I felt they were holding my own content hostage and demanding a ransom from other search engines to access the data.

As a concrete example, when Matt Cutts first mentioned evaporating PageRank and the “reset vector”, there was a fair amount of chatter on Twitter, including some tweets from me. You won’t find those conversations now.

So why would I give them more data directly?

They have no respect over your content, my content… anyone’s content

Conclusion

Use Topsy – they have much more accurate tracking of tweets, handle 301 redirects pretty well and even give you some really useful historical Twitter data for a whole domain.
Here is Andy Beard on Topsy – that is a vote.

I am not saying everything is perfect with Topsy – they are not doing silly stuff with Robots.txt but their nofollow of links is inconsistent and might benefit from a threshold based on authority and an understanding of first link priority.

There is only one significant benefit of the new official retweet button, you can now define retweet text, which is something I wanted Tweetmeme to include a year ago.
Combined with custom URLs in theory you can use that for split testing – in practice I don’t think it is going to work as there is no way to define a canonical URL, plus a URL that gets tweeted with a URL decorated with tracking parameters.

Thus the new Tweet button sucks for marketers even 1 year on.

More on Mediagazer… this seems to have rolled off of Techmeme in less than 24 hours.

Update

Just adding a test new button that has been modified manually

You should Tweet This Post

As text link: You should Tweet This Post

Test without defining the title

You should Tweet This Post

As text link: You should Tweet This Post

With Nothing Defined

You should Tweet This Post

As text link: You should Tweet This Post

A Shortcode Test