Robert Scoble asked earlier today if Twitter is Now Reliable?
I have been using Twitter a little more than normal over the last week, maybe it will last. During that time I have been looking at the reliability of consuming Twitter content via various tools (no I haven’t tested everything).
Last night was actually a very good opportunity to do some data analysis, because Robert was very active on Twitter, with lots of updates in a very short period of time.
Could you follow the conversation on every platform?
Twitter Web Interface
The first interface I tested was the Twitter web interface.
I grabbed a screenshot of Robert’s twitter updates, and then went to my friends feed on Twitter to compare how many of Robert’s tweets actually made it to my Twitter Web page, to allow me to follow the conversation.
What I have done is highlighted in yellow the tweets that didn’t make it though to my feed.
4 out of 20 messages made it through to my Twitter home page “with friends”
20% Deliverability = Fail
FriendFeed
This is a slightly different situation to the main Twitter web interface, because of the clumping I previous objected to. They only pull some of the updates.
When you click on that huge lump of tweets you get something like this
I have used numbers to represent the order that the tweets were actually written, and you can see that they appear out of order on FriendFeed.
When grabbing the tweets in the clump, Friendfeed is able to make a specific call to retrieve Robert’s tweets for that time segment using the Twitter API, thus all the tweets appear.
To follow people who don’t tweet very often and to be sure that you see every one of their tweets, Friendfeed might actually be a better interface for twitter than twitter itself, if it wasn’t for the clumping of high activity.
If Twitter continue to only deliver 20% of tweets to other interfaces, it makes more sense to use Friendfeed, but reading them out of order, you might as well click through to Robert’s timeline on Twitter.
Tweets in clumps out of order = fail
Twhirl
I have been testing out Twhirl for a while, and used the same data to see what is picked up.
I haven’t used the same screenshot as for the web interface, and the data was compiled from my Twhirl history probably 3 hours later than the first screenshot.
You will see that Twirl again only has 4 out of 20 tweets
note: I only have Twhirl pulling in data 40 times per hour – as it is only pulling data that is the same as on the web interface, this isn’t an issue.
20% Deliverability = Fail
20% Delivery = Low Traffic
If you were using an email marketing solution, and it offered only a 20% chance of the message being delivered to your prospects’ email box, without allowing for open and CTR, it would be a major business liability.
What is the point of having 10,000+ Twitter followers if your messages are only going to be delivered to 20% of them?
In addition there is the problem that only a small percentage continue to use Twitter – heavy attrition.
Direct messages seem to be a lot more reliable, and I haven’t got the patience to manually determine if this deliverability affects people with less than 100 followers.
Twitter is not reliable, they just throttle the amount of updates you see
Update
A number of people have raised one issue with this post, the use of @
What is being filtered from what you receive are @ messages to people you are not following, but this isn’t a universal rule.
As an example, here is a tweet from Rob, Former Fat Guy to Chris Pirillo
I don’t follow Chris, maybe I should… but then I am only following just over 300 of the people who are following me – there are others I am not following, I am slowly increasing the people I follow, but priority is regular readers and commenters – my core audience.
For me seeing @ messages to people I might not be following is extremely important, as it plays a vital role in discovery of other friends using Twitter.
People use the @ reply mechanism to ensure that people who would be most interested in the tweet see it, not just as a reply in conversation.
They are not aware that by using @ they are also preventing, or reducing the number of other people the message will be sent to.
Just imagine Robert discovered through his Microsoft contacts that a Yahoo deal was finally on the table. Rather than use a normal tweet, he uses @techcrunch to make sure Michael Arrington sees it.
That would then potentially limit who sees that message to those following both Robert and Michael, plus an arbitrary number of additional random people who are just following Robert, or maybe none if there is some kind of threshold based on the number of people following both.
23 Comments
Good to see you more frequently on Twitter. I was wondering myself about tweets coming through, as it seems I miss some once in awhile. I was checking out twittervision, and I guess they only display a group of tweets at a time, and then refresh with a new group. After leaving it up on my screen for a while, it looked like many tweets from people I follow were not showing up.
I am not sure how much time I will spend directly on Twitter, if it isn’t delivering a message to all the people who follow, it seems like a huge time investment into a fragmented unreliable communications service.
I have a feeling all Twitter clients will generally work in the same ways as Twhirl, and the web interface. You don’t receive full content from the people you follow.
Maybe someone will come up with a tool that allows you to direct message all your followers., but that only works because lots of them will subsequently be sent emails – why not just encourage email?
Andy, thanks for pointing out this ordering issue. FYI, we are not making calls to the Twitter API in the way you mention, but the ordering issue is annoying independent of the technical details. We will fix the issue this week.
Thanks for using FriendFeed!
Interesting, so if you are storing all the data this is a search play?
The biggest problem with Twitter is that things that use the API can only pull 20 items (20 replies, 20 DMs, and 20 tweets) at a time, Gtalk and RSS get all of them.
There isn’t an easy way to handle the level of info and still follow a decent number of people. I think I’m going to try Sam Harrelson’s suggestion to sub to the RSS feed and then use filter within FeedDemon to highlight the folks I need/want to highlight.
Now the question will be is if the Twitter proxy idea will take off.
Could Twitter be more reliable? Yes, but not as it is now.
When some one tweets using “@” character.. You will get that message in your feed ONLY when you are following the person to whom the reply is sent. This is not an error .. It is a FEATURE.
Great post Andy – I have been trying to understand why Twitter gets a pass on these clear flaws. So many ‘tech gurus’ who would lambaste another tech product for these problems, seem to see Twitter through rose colored glasses. I like Twitter, I am on it a decent amount, I have met some people and had some good conversations, but the bugs drive me crazy – what drives me the most crazy though are those in the tech community that seem to ignore the bugs and unfairly give Twitter a reliability pass.
nishu there is a preference to let you see @’s from people you don’t follow. This is a great feature.
The setting says @ replies to the people I am following, which I have taken to mean that if in the above example Chris Pirillo sent an @ to Rob, I would see that as well, without having to follow Chris.
In that way you would see both sides of the conversation, a bit like sites which don’t normally appear on Techmeme can be brought into the conversation with links.
I have now set mine to see all @ messages
I have always seen some of the @messages sent from people I follow to people I don’t follow, but not all of them.
However the explanation in the help section is a little clearer
So if I had my settings on the default, how do I see Rob’s reply to Chris?
Twitter, Digg, Stumble, EntreCard, FaceBook. I can’t keep up. Us part time bloggers can get overwelmed by all the activity. I wish there was a way to consolidate all of these into one access point.
I have noticed a disconnect in a lot of the Twitter software as well. Thanks for pointing this out, I thought I was crazy ;)
I use the Gmail G-Talk function to update my status, I think that works pretty well
I’ve always wondered what the best way to use twitter. i didnt know they supplied a lot tools to their users.
Nice article, the screenshots are amazing. I’ve played with Twitter on many “platforms” and the best integration tool for me is the Facebook app. I can make a Facebook status change and it shows up on both.
have noticed a disconnect in a lot of the Twitter software as well. Thanks for pointing this out, I thought I was crazy ;)
How much traffic do this tool bring in your site? In percentage.. if you don’t mind.
I think twitter has proved it is simply just another site that spammers can you use to generate traffic to their bogus sites. Its going to be a long time before we find a tool/site that can give us good content that we want. Thoughts?
Very insightful post here.
Seems like a lot of these social sites get
a lot of press and hype but it’s really all
about deliverability and accountability.
You should ask yourself…IF your messages
get through does that do anything for you?
HITS= How Idiots Track Sales…
Therefore, the quality of the traffic
and the deliverability of the messages
to that traffic is critical…
Great post, keep up the good work
Daniel McGonagle
http;//marketersrelief.com
I’m still yet to see the relevance of using twitter. I guess it would be efficient for a workplace environment, but it just seems to take up much needed time!
Maybe I am missing something- I have tried and tried but I just don’t get the whole Twitter thing… I know several people who are addicted to it. I WANT to like it.. but I can’t get into it.
i had a doubt about twitter and you made it a reality with this post.thanx
I am far from being afraid to public admit my ignorance. I just don’t get the tweeter concept. Anyone care to break this down for the beginner?
Andy – I think it’s important to look into the reliability of Twitter. It’s something we’re spending a lot of time improving and there’s definitely things we need to do better.
In this case, however, I think there’s more confusion over one of our settings than there is a problem with our reliability.
As you note in your update and the comment above, you did not see the @ replies because they were directed to people you do not follow.
By default we do not show you @ replies directed at people you do not follow. Therefore those posts were never going to make it into your friends timeline. Now that you’ve changed your setting, you’ll see everything.
The reason you saw Rob’s reply to Chris is because he made a typo. There is no user @chrisprillo; Chris’s username is @chrispirillo. If a username doesn’t match in an @reply, we send it out to everyone (because people use the @ reply for other meanings).
Now, I don’t want to give the impression that everything works perfectly. For one thing, it’s clear that this setting is confusing. And we’ve had bugs with how @ replys appear. There’s a lot more we need to do to improve both our user experience and our overall reliability.
But in this case I hope this helps explain why your test doesn’t actually show a deliverability problem.
Thanks.
The social part of the web is spread across thousands of different sites. Wouldn’t it be great to rather have a seamless layer on top of the web that everyone can build from and equally have access to?
Twitter has improved a lot, though.
3 Trackbacks
[...] Twitter – Just Because It Is Up Doesn’t Make It Reliable [...]
[...] Beard has an excellent study posted regarding the reliability of Twitter clients. The results are quite bleak and I’m really rethinking my current Twitter application because [...]
[...] from the people I follow. Despite the fact that Twitter is actually pretty awful at updating feeds (Andy Beard discovered that actually only about 20% of tweets ever seem to make it through to the people following you), [...]