<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Internet Business &#38; Marketing Strategy - Andy Beard &#187; http header</title>
	<atom:link href="http://andybeard.eu/tag/http-header/feed" rel="self" type="application/rss+xml" />
	<link>http://andybeard.eu</link>
	<description>Internet Marketing, Lead Acquisition, Online Business Strategy and Social Media with Original Opinion and Loads of Attitude</description>
	<lastBuildDate>Sun, 12 Feb 2012 06:16:17 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	
		<item>
		<title>Dell Better At Social Media Than SEO?</title>
		<link>http://andybeard.eu/2529/dell-better-at-social-media-than-seo.html</link>
		<comments>http://andybeard.eu/2529/dell-better-at-social-media-than-seo.html#comments</comments>
		<pubDate>Tue, 08 Dec 2009 20:36:42 +0000</pubDate>
		<dc:creator>Andy Beard</dc:creator>
				<category><![CDATA[Google]]></category>
		<category><![CDATA[SEO Blog]]></category>
		<category><![CDATA[301 redirect]]></category>
		<category><![CDATA[dell]]></category>
		<category><![CDATA[http header]]></category>
		<category><![CDATA[robots.txt]]></category>

		<guid isPermaLink="false">http://andybeard.eu/?p=2529</guid>
		<description><![CDATA[
<p><a href="http://www.bloomberg.com/apps/news?pid=newsarchive&#038;sid=akXzD_6YNHCk">Lots</a> <a href="http://www.techcrunch.com/2009/12/08/online-holiday-spending-reaches-16-billion-social-media-continues-to-influence-purchases/">of</a> <a href="http://www.marketingpilgrim.com/2009/12/social-media-darling-dell-updates-numbers.html">reports</a> are out today about how effectively Dell is using social media marketing and especially Twitter to generate revenue, $6.5M in sales are the headlines, I wonder what that translates to in margins.</p>
<p><a href="http://andybeard.eu/2529/dell-better-at-social-media-than-seo.html" class="more-link">Read more on Dell Better At Social Media Than SEO?&#8230;</a></p>
<div class="topsy_widget_data topsy_theme_brick-red" style="float: right;margin-left: 0.75em; background: url(data:,%7B%20%22url%22%3A%20%22http%253A%252F%252Fandybeard.eu%252F2529%252Fdell-better-at-social-media-than-seo.html%22%2C%20%22style%22%3A%20%22small%22%2C%20%22title%22%3A%20%22Dell%20Better%20At%20Social%20Media%20Than%20SEO%3F%22%20%7D);"></div>


	Tags: <a href="http://andybeard.eu/tag/301-redirect" title="301 redirect" rel="tag">301 redirect</a>, <a href="http://andybeard.eu/tag/dell" title="dell" rel="tag">dell</a>, <a href="http://andybeard.eu/tag/http-header" title="http header" rel="tag">http header</a>, <a href="http://andybeard.eu/tag/robotstxt" title="robots.txt" rel="tag">robots.txt</a>, <a href="http://andybeard.eu/tag/seo" title="SEO Blog" rel="tag">SEO Blog</a><br />
]]></description>
			<content:encoded><![CDATA[
<p><a href="http://www.bloomberg.com/apps/news?pid=newsarchive&#038;sid=akXzD_6YNHCk">Lots</a> <a href="http://www.techcrunch.com/2009/12/08/online-holiday-spending-reaches-16-billion-social-media-continues-to-influence-purchases/">of</a> <a href="http://www.marketingpilgrim.com/2009/12/social-media-darling-dell-updates-numbers.html">reports</a> are out today about how effectively Dell is using social media marketing and especially Twitter to generate revenue, $6.5M in sales are the headlines, I wonder what that translates to in margins.</p>
<p>However yesterday I read a post <a href="http://peterlaird.blogspot.com/2009/12/ranking-cloud-computing-vendors-2009.html">ranking cloud computing vendors</a> based on mind share using a points system based on Google ranking for the term [cloud computing]</p>
<p>The winner was Rackspace, but <a href="http://andybeard.eu/1879/rackspace-cloud-mosso-link-to-us-payola.html">we have some idea</a> how they achieved that</p>
<p>But I was more interested in the result for Dell</p>
<p><img src="http://cdn5.andybeard.name/wp-content/uploads/dell-no-snippet.png" alt="dell-no-snippet" title="dell-no-snippet" width="486" height="400" class="aligncenter size-full wp-image-2532" /></p>
<p>First off they are doing some funny redirects so the displyed URL in the snippet<br />
www.dell.com/cloudcomputing<br />
Redirects to their cloud computing page on a subdomain which I assume is some kind of CDN</p>
<p>http://content.dell.com/us/en/enterprise/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz</p>
<p>Notice that there is only a URL displayed in the snippet and not an excerpt from the page or meta description. Google is just displaying the page as a reference, but for some reason couldn&#8217;t crawl the page.</p>
<p>Now before I show you what they have done wrong I did try to contact Dell about this yesterday. I sent a tweet to the guy who maintains at least one of their social media profiles, and since this tweet he has been tweeting on his account so I assume he has had a chance to read the tweet I sent.</p>
<p><img src="http://cdn5.andybeard.name/wp-content/uploads/dell-robotstxt.png" alt="dell-robotstxt" title="dell-robotstxt" width="436" height="74" class="aligncenter size-full wp-image-2531" /></p>
<p>I didn&#8217;t receive a response, and as this is a significant problem I am blogging about it, mainly because Google seemed to be doing something contrary to what I previously understood (they weren&#8217;t if you read more) and have seen written by authoritative sources such as Vanessa Fox.</p>
<p>Here is the robots.txt of that Dell subdomain</p>
<div style="font-size:x-small">
User-agent: *<br />
Allow: /at/de/home/<br />
Allow: /au/en/business/<br />
Allow: /ca/en/business/<br />
Allow: /de/de/corp/<br />
Allow: /de/de/home/<br />
Allow: /hk/en/business/<br />
Allow: /ie/en/business/<br />
Allow: /ie/en/home/<br />
Allow: /in/en/business/<br />
Allow: /my/en/business/<br />
Allow: /nz/en/business/<br />
Allow: /sg/en/business/<br />
Allow: /uk/en/business/<br />
Allow: /uk/en/home/<br />
Allow: /us/en/business/<br />
Allow: /us/en/corp/<br />
Allow: /us/en/enterprise/<br />
Allow: /us/en/home/<br />
Allow: http://content.dell.com<br />
Allow: http://content.dell.com/<br />
Allow: http://content.dell.com/au.sitemap.txt<br />
Allow: http://content.dell.com/hk.sitemap.txt<br />
Allow: http://content.dell.com/in.sitemap.txt<br />
Allow: http://content.dell.com/jp.sitemap.txt<br />
Allow: http://content.dell.com/my.sitemap.txt<br />
Allow: http://content.dell.com/nz.sitemap.txt<br />
Allow: http://content.dell.com/sg.sitemap.txt<br />
Disallow: /
</div>
<p>The page concerned us under /us/en/enterprise/ so should be controlled by this directive.<br />
Allow: /us/en/enterprise/</p>
<p>As <a href="http://janeandrobot.com/library/managing-robots-access-to-your-website">Vanessa states</a></p>
<blockquote><p>If conflicts exist in the file, the robot obeys the longest (and therefore generally more specific) line.</p></blockquote>
<p>So somthing is going wrong in between the URL displayed in the search results, and the final landing page.</p>
<h2>Server Header Analysis</h2>
<p>Search result<br />
>> [302] http://www.dell.com/cloudcomputing<br />
>> [302] http://content.dell.com/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz<br />
>> [301] /us/en/enterprise/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz</p>
<div style="font-size:x-small">
<p>http://www.dell.com/cloudcomputing</p>
<p>GET /cloudcomputing HTTP/1.1<br />
Host: www.dell.com<br />
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5<br />
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<br />
Accept-Language: en-gb,en;q=0.5<br />
Accept-Encoding: gzip,deflate<br />
Accept-Charset: UTF-8,*<br />
Keep-Alive: 300<br />
Connection: keep-alive<br />
Referer: http://www.google.com/search?hl=en&#038;pws=0&#038;gl=US&#038;q=cloud+computing&#038;start=20&#038;sa=N<br />
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&#038;pc=pl&#038;bandwidth=NA</p>
<p>HTTP/1.x 302 Found<br />
Content-Type: text/html; charset=utf-8<br />
Location: http://content.dell.com/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz<br />
Server: Microsoft-IIS/7.0<br />
Set-Cookie: stop_mobi=; path=/<br />
X-Powered-By: ASP.NET<br />
x-ua-compatible: IE=7<br />
Date: Tue, 08 Dec 2009 19:42:34 GMT<br />
Content-Length: 196</p>
<p>http://content.dell.com/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz</p>
<p>GET /cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz HTTP/1.1<br />
Host: content.dell.com<br />
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5<br />
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<br />
Accept-Language: en-gb,en;q=0.5<br />
Accept-Encoding: gzip,deflate<br />
Accept-Charset: UTF-8,*<br />
Keep-Alive: 300<br />
Connection: keep-alive<br />
Referer: http://www.google.com/search?hl=en&#038;pws=0&#038;gl=US&#038;q=cloud+computing&#038;start=20&#038;sa=N<br />
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&#038;pc=pl&#038;bandwidth=NA</p>
<p>HTTP/1.x 301 Moved Permanently<br />
Date: Tue, 08 Dec 2009 19:42:35 GMT<br />
Server: Microsoft-IIS/6.0<br />
X-Powered-By: ASP.NET<br />
X-AspNet-Version: 2.0.50727<br />
Location: /us/en/enterprise/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz<br />
Cache-Control: private<br />
Content-Length: 0</p>
<p>http://content.dell.com/us/en/enterprise/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz</p>
<p>GET /us/en/enterprise/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz HTTP/1.1<br />
Host: content.dell.com<br />
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5<br />
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<br />
Accept-Language: en-gb,en;q=0.5<br />
Accept-Encoding: gzip,deflate<br />
Accept-Charset: UTF-8,*<br />
Keep-Alive: 300<br />
Connection: keep-alive<br />
Referer: http://www.google.com/search?hl=en&#038;pws=0&#038;gl=US&#038;q=cloud+computing&#038;start=20&#038;sa=N<br />
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&#038;pc=pl&#038;bandwidth=NA</p>
<p>HTTP/1.x 200 OK<br />
Cache-Control: private,max-age=0<br />
Date: Tue, 08 Dec 2009 19:42:36 GMT<br />
Content-Type: text/html; charset=utf-8<br />
Expires: Wed, 01 Jan 1997 12:00:00 GMT<br />
Server: Microsoft-IIS/6.0<br />
X-Powered-By: ASP.NET<br />
X-AspNet-Version: 2.0.50727<br />
X-Awesomed-By: Thundera RE-TJN<br />
Set-Cookie: lwp=c=us&#038;l=en&#038;s=biz&#038;cs=555; domain=.dell.com; path=/<br />
Set-Cookie: dus=ci=cloud-computing&#038;th=Default; path=/<br />
Content-Encoding: gzip<br />
Vary: Accept-Encoding<br />
Transfer-Encoding: chunked
</p></div>
<h2>Problem &#8211; Robots.txt Disallow Match</h2>
<p>The inbetween URL redirect is blocked by robots.txt</p>
<p>http://content.dell.com/cloud-computing.aspx?c=us&#038;cs=555&#038;l=en&#038;s=biz</p>
<p>It is actually matched by<br />
Allow: http://content.dell.com/<br />
Disallow: /</p>
<p>Google has decided to take the most restrictive version, to disallow the URL.</p>
<p>I haven&#8217;t checked how many links are pointing to that URL both from internal and external sources, but I would guess it is a fair amount and I am assuming at this time that is a page they would want to rank as in their line of business it is probably quite a competitive term.</p>
<p>At a guess without that one URL blocked by robots.txt, they would have been much higher on the comparison charts and ranking higher for cloud computing &#8211; most likely that is a 1st page result thrown away by robots.txt</p>
<p>I haven&#8217;t checked how many other key landing pages are blocked in a similar manner &#8211; I bet it is quite a few</p>
<p>This is just chalking up one more reason why robots.txt should be avoided as it is so easy to mess things up.</p>
<p>p.s. My niece is 18 in 2 months &#038; I am sure <a href="http://andybeard.eu/contact">she would love a Dell</a></p>
<div class="topsy_widget_data topsy_theme_brick-red" style="float: right;margin-left: 0.75em; background: url(data:,%7B%20%22url%22%3A%20%22http%253A%252F%252Fandybeard.eu%252F2529%252Fdell-better-at-social-media-than-seo.html%22%2C%20%22style%22%3A%20%22small%22%2C%20%22title%22%3A%20%22Dell%20Better%20At%20Social%20Media%20Than%20SEO%3F%22%20%7D);"></div>


	Tags: <a href="http://andybeard.eu/tag/301-redirect" title="301 redirect" rel="tag">301 redirect</a>, <a href="http://andybeard.eu/tag/dell" title="dell" rel="tag">dell</a>, <a href="http://andybeard.eu/tag/http-header" title="http header" rel="tag">http header</a>, <a href="http://andybeard.eu/tag/robotstxt" title="robots.txt" rel="tag">robots.txt</a>, <a href="http://andybeard.eu/tag/seo" title="SEO Blog" rel="tag">SEO Blog</a><br />
]]></content:encoded>
			<wfw:commentRss>http://andybeard.eu/2529/dell-better-at-social-media-than-seo.html/feed</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
	</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Page Caching using disk: basic
Database Caching 31/44 queries in 0.008 seconds using disk: basic
Object Caching 708/727 objects using disk: basic
Content Delivery Network via cdn5.andybeard.name

Served from: andybeard.eu @ 2012-02-13 01:36:48 -->
