Twitter – Just Because It Is Up Doesn’t Make It Reliable

Robert Scoble asked earlier today if Twitter is Now Reliable?

I have been using Twitter a little more than normal over the last week, maybe it will last. During that time I have been looking at the reliability of consuming Twitter content via various tools (no I haven’t tested everything).

Last night was actually a very good opportunity to do some data analysis, because Robert was very active on Twitter, with lots of updates in a very short period of time.

Could you follow the conversation on every platform?

Twitter Web Interface

The first interface I tested was the Twitter web interface.

I grabbed a screenshot of Robert’s twitter updates, and then went to my friends feed on Twitter to compare how many of Robert’s tweets actually made it to my Twitter Web page, to allow me to follow the conversation.

What I have done is highlighted in yellow the tweets that didn’t make it though to my feed.

Which tweets Robert Scoble sent actually made it to my timeline?

4 out of 20 messages made it through to my Twitter home page “with friends”

20% Deliverability = Fail

FriendFeed

This is a slightly different situation to the main Twitter web interface, because of the clumping I previous objected to. They only pull some of the updates.

FriendFeed Clumping

When you click on that huge lump of tweets you get something like this

Friendfeed gets all the tweets, but out of order

I have used numbers to represent the order that the tweets were actually written, and you can see that they appear out of order on FriendFeed.
When grabbing the tweets in the clump, Friendfeed is able to make a specific call to retrieve Robert’s tweets for that time segment using the Twitter API, thus all the tweets appear.

To follow people who don’t tweet very often and to be sure that you see every one of their tweets, Friendfeed might actually be a better interface for twitter than twitter itself, if it wasn’t for the clumping of high activity.

If Twitter continue to only deliver 20% of tweets to other interfaces, it makes more sense to use Friendfeed, but reading them out of order, you might as well click through to Robert’s timeline on Twitter.

Tweets in clumps out of order = fail

Twhirl

I have been testing out Twhirl for a while, and used the same data to see what is picked up.

Twhirl Timeline the same as Web

I haven’t used the same screenshot as for the web interface, and the data was compiled from my Twhirl history probably 3 hours later than the first screenshot.
You will see that Twirl again only has 4 out of 20 tweets

note: I only have Twhirl pulling in data 40 times per hour – as it is only pulling data that is the same as on the web interface, this isn’t an issue.

20% Deliverability = Fail

20% Delivery = Low Traffic

If you were using an email marketing solution, and it offered only a 20% chance of the message being delivered to your prospects’ email box, without allowing for open and CTR, it would be a major business liability.

What is the point of having 10,000+ Twitter followers if your messages are only going to be delivered to 20% of them?

In addition there is the problem that only a small percentage continue to use Twitter – heavy attrition.

Direct messages seem to be a lot more reliable, and I haven’t got the patience to manually determine if this deliverability affects people with less than 100 followers.

Twitter is not reliable, they just throttle the amount of updates you see

Update

A number of people have raised one issue with this post, the use of @

What is being filtered from what you receive are @ messages to people you are not following, but this isn’t a universal rule.

As an example, here is a tweet from Rob, Former Fat Guy to Chris Pirillo

Tweet From Rob, Former Fat Guy

I don’t follow Chris, maybe I should… but then I am only following just over 300 of the people who are following me – there are others I am not following, I am slowly increasing the people I follow, but priority is regular readers and commenters – my core audience.

For me seeing @ messages to people I might not be following is extremely important, as it plays a vital role in discovery of other friends using Twitter.

People use the @ reply mechanism to ensure that people who would be most interested in the tweet see it, not just as a reply in conversation.
They are not aware that by using @ they are also preventing, or reducing the number of other people the message will be sent to.

Just imagine Robert discovered through his Microsoft contacts that a Yahoo deal was finally on the table. Rather than use a normal tweet, he uses @techcrunch to make sure Michael Arrington sees it.
That would then potentially limit who sees that message to those following both Robert and Michael, plus an arbitrary number of additional random people who are just following Robert, or maybe none if there is some kind of threshold based on the number of people following both.

Liked this post? Follow this blog to get more. Follow

Comments

  1. says

    Good to see you more frequently on Twitter. I was wondering myself about tweets coming through, as it seems I miss some once in awhile. I was checking out twittervision, and I guess they only display a group of tweets at a time, and then refresh with a new group. After leaving it up on my screen for a while, it looked like many tweets from people I follow were not showing up.

    • says

      I am not sure how much time I will spend directly on Twitter, if it isn’t delivering a message to all the people who follow, it seems like a huge time investment into a fragmented unreliable communications service.
      I have a feeling all Twitter clients will generally work in the same ways as Twhirl, and the web interface. You don’t receive full content from the people you follow.

      Maybe someone will come up with a tool that allows you to direct message all your followers., but that only works because lots of them will subsequently be sent emails – why not just encourage email?

  2. says

    Andy, thanks for pointing out this ordering issue. FYI, we are not making calls to the Twitter API in the way you mention, but the ordering issue is annoying independent of the technical details. We will fix the issue this week.

    Thanks for using FriendFeed!

  3. says

    The biggest problem with Twitter is that things that use the API can only pull 20 items (20 replies, 20 DMs, and 20 tweets) at a time, Gtalk and RSS get all of them.

    There isn’t an easy way to handle the level of info and still follow a decent number of people. I think I’m going to try Sam Harrelson’s suggestion to sub to the RSS feed and then use filter within FeedDemon to highlight the folks I need/want to highlight.

    Now the question will be is if the Twitter proxy idea will take off.

    Could Twitter be more reliable? Yes, but not as it is now.

  4. says

    When some one tweets using “@” character.. You will get that message in your feed ONLY when you are following the person to whom the reply is sent. This is not an error .. It is a FEATURE.

  5. says

    Great post Andy – I have been trying to understand why Twitter gets a pass on these clear flaws. So many ‘tech gurus’ who would lambaste another tech product for these problems, seem to see Twitter through rose colored glasses. I like Twitter, I am on it a decent amount, I have met some people and had some good conversations, but the bugs drive me crazy – what drives me the most crazy though are those in the tech community that seem to ignore the bugs and unfairly give Twitter a reliability pass.

    • says

      The setting says @ replies to the people I am following, which I have taken to mean that if in the above example Chris Pirillo sent an @ to Rob, I would see that as well, without having to follow Chris.

      In that way you would see both sides of the conversation, a bit like sites which don’t normally appear on Techmeme can be brought into the conversation with links.

      I have now set mine to see all @ messages

      I have always seen some of the @messages sent from people I follow to people I don’t follow, but not all of them.

      However the explanation in the help section is a little clearer

      * Default: @ replies to the people I’m following

      I receive @replies from people I follow under the condition that I also follow the person they are replying to. This setting is ideal for those seeking the happy medium in Twitter interaction, as @replies are still visible, but restricted to mutual followers. (Do you err on the side of caution? I am for you.)

      * Always: all @ replies

      I receive all @replies from people I follow, even if I don’t follow the person to whom the @reply is directed.This setting is ideal for those seeking maximum social Twitter interaction, as viewing replies to people you don’t follow may spur impulsive or compulsive Twitter reading (or both!) as well as more potential @replying on your part. You may develop an unexpected interest in friends of friends, followers of friends, friends of followers. (Do you fly by the seat of your panties? Pick me!)

      * Never: no @ replies

      I never, ever receive @replies, and I never want to. I could care less about twitters directed at other people, whether I know them or not. This setting is ideal for those looking for minimal social-conversational Twitter interaction. (Slightly antisocial? Abhor social networking? It’s me you want.)

      So if I had my settings on the default, how do I see Rob’s reply to Chris?

  6. says

    Twitter, Digg, Stumble, EntreCard, FaceBook. I can’t keep up. Us part time bloggers can get overwelmed by all the activity. I wish there was a way to consolidate all of these into one access point.

  7. says

    I have noticed a disconnect in a lot of the Twitter software as well. Thanks for pointing this out, I thought I was crazy ;)

    I use the Gmail G-Talk function to update my status, I think that works pretty well

  8. says

    Nice article, the screenshots are amazing. I’ve played with Twitter on many “platforms” and the best integration tool for me is the Facebook app. I can make a Facebook status change and it shows up on both.

  9. says

    have noticed a disconnect in a lot of the Twitter software as well. Thanks for pointing this out, I thought I was crazy ;)

  10. says

    I think twitter has proved it is simply just another site that spammers can you use to generate traffic to their bogus sites. Its going to be a long time before we find a tool/site that can give us good content that we want. Thoughts?

  11. Daniel McGonagle @ http://danielmcgonagle.name says

    Very insightful post here.

    Seems like a lot of these social sites get
    a lot of press and hype but it’s really all
    about deliverability and accountability.

    You should ask yourself…IF your messages
    get through does that do anything for you?

    HITS= How Idiots Track Sales…

    Therefore, the quality of the traffic
    and the deliverability of the messages
    to that traffic is critical…

    Great post, keep up the good work

    Daniel McGonagle
    http;//marketersrelief.com

  12. says

    I’m still yet to see the relevance of using twitter. I guess it would be efficient for a workplace environment, but it just seems to take up much needed time!

  13. says

    I am far from being afraid to public admit my ignorance. I just don’t get the tweeter concept. Anyone care to break this down for the beginner?

  14. says

    Andy – I think it’s important to look into the reliability of Twitter. It’s something we’re spending a lot of time improving and there’s definitely things we need to do better.

    In this case, however, I think there’s more confusion over one of our settings than there is a problem with our reliability.

    As you note in your update and the comment above, you did not see the @ replies because they were directed to people you do not follow.

    By default we do not show you @ replies directed at people you do not follow. Therefore those posts were never going to make it into your friends timeline. Now that you’ve changed your setting, you’ll see everything.

    The reason you saw Rob’s reply to Chris is because he made a typo. There is no user @chrisprillo; Chris’s username is @chrispirillo. If a username doesn’t match in an @reply, we send it out to everyone (because people use the @ reply for other meanings).

    Now, I don’t want to give the impression that everything works perfectly. For one thing, it’s clear that this setting is confusing. And we’ve had bugs with how @ replys appear. There’s a lot more we need to do to improve both our user experience and our overall reliability.

    But in this case I hope this helps explain why your test doesn’t actually show a deliverability problem.

    Thanks.

  15. says

    The social part of the web is spread across thousands of different sites. Wouldn’t it be great to rather have a seamless layer on top of the web that everyone can build from and equally have access to?

    Twitter has improved a lot, though.

Trackbacks