Comments on: Robots-Nocontent | How To Spot Abuse By Sneaky Webmasters http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html Internet Marketing, Lead Acquisition, Online Business Strategy and Social Media with Original Opinion and Loads of Attitude Mon, 15 Mar 2010 04:39:33 +0000 hourly 1 By: Matt Ellsworth http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-32511 Matt Ellsworth Wed, 06 Jun 2007 20:57:09 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-32511 I can implement this in our site in about 3 files - the header/footer/navigation. I think I will give it a shot because I have definitely seen pages on our site getting ranked based on things that are in the navigation column. I can implement this in our site in about 3 files – the header/footer/navigation. I think I will give it a shot because I have definitely seen pages on our site getting ranked based on things that are in the navigation column.

]]> By: Andy Beard http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21114 Andy Beard Tue, 08 May 2007 10:26:55 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21114 I am using a beta version of DiggIt http://tuggo.org/projects/diggit/ Unfortunately the version on the front page is broken in IE, and the forums where 1.1.4 is available are currently down It is the only plugin I know of that uses the real (new) Digg buttons with perfect detection, and allows the buttons to operate on a front page. It took the guy a little hacking to do it. The translation plugin creates cached pages and Google have specifically stated that translated pages even using their tools are not a problem for supplemental results. They do eat up a lot of server allocation, and bandwidth when the bots come visiting. This blog currently has over 7000 pages cached and only 2 in supplemental. I must admit that number is a bit of a roller coaster, I have seen it as low as 3000 recently. That number will also increase as the bots pick up all my translated pages. Theoretically they should have close to 20K pages cached. I don't need to ask people whether they are useful, because half of my visitors are from North America, any poll would be worthless, and I look at my stats. Just today I know that French, German, Japanese, Russian, Korean, Spanish, Portuguese and both forms of Chinese pages were viewed, and I do see improved results with search traffic that so far are hard to quantify. I live in Europe, I know the search habits of Europeans, even if they speak fluent English. On page loading there are a few things I can change and will change, though the Digg button seems to have as much effect as most of the header other than the subscriptions. The subscription buttons are due for another redesign anyway. I am using a beta version of DiggIt

http://tuggo.org/projects/diggit/

Unfortunately the version on the front page is broken in IE, and the forums where 1.1.4 is available are currently down

It is the only plugin I know of that uses the real (new) Digg buttons with perfect detection, and allows the buttons to operate on a front page. It took the guy a little hacking to do it.

The translation plugin creates cached pages and Google have specifically stated that translated pages even using their tools are not a problem for supplemental results.

They do eat up a lot of server allocation, and bandwidth when the bots come visiting.

This blog currently has over 7000 pages cached and only 2 in supplemental.
I must admit that number is a bit of a roller coaster, I have seen it as low as 3000 recently. That number will also increase as the bots pick up all my translated pages. Theoretically they should have close to 20K pages cached.

I don’t need to ask people whether they are useful, because half of my visitors are from North America, any poll would be worthless, and I look at my stats.

Just today I know that French, German, Japanese, Russian, Korean, Spanish, Portuguese and both forms of Chinese pages were viewed, and I do see improved results with search traffic that so far are hard to quantify.
I live in Europe, I know the search habits of Europeans, even if they speak fluent English.

On page loading there are a few things I can change and will change, though the Digg button seems to have as much effect as most of the header other than the subscriptions.
The subscription buttons are due for another redesign anyway.

]]> By: The SEO Blogger http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21074 The SEO Blogger Tue, 08 May 2007 06:17:24 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21074 Hey, I've just read the post and apparently you have visited the site =P. Anyway, I don't think you're using the 'Digg That' plugin, though--or are you? Hey, I’ve just read the post and apparently you have visited the site =P. Anyway, I don’t think you’re using the ‘Digg That’ plugin, though–or are you?

]]> By: The SEO Blogger http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21067 The SEO Blogger Tue, 08 May 2007 05:51:48 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-21067 WOW...Thanks for the code, but Andy your page loading time is killing------my---patience =P. Maybe trying to lose some java a little bit--just a suggestion =? *(I think the translation is not a lot of use, but you can post a poll to see if that function's use or not. Really, seriously it takes quite long to load the page ;). Also, there's an updgrade/more secure plugin for 'Digg This' called 'Digg That'--if you are interested in security: http://www.harrymaugans.com/digg-that/ WOW…Thanks for the code, but Andy your page loading time is killing——my—patience =P. Maybe trying to lose some java a little bit–just a suggestion =? *(I think the translation is not a lot of use, but you can post a poll to see if that function’s use or not. Really, seriously it takes quite long to load the page ;).

Also, there’s an updgrade/more secure plugin for ‘Digg This’ called ‘Digg That’–if you are interested in security:

http://www.harrymaugans.com/digg-that/

]]> By: Andy Beard http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20514 Andy Beard Sun, 06 May 2007 06:08:58 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20514 Just a floated DIV before the content [code]<?php the_post(); ?> <div id="post-<?php the_ID(); ?>" class="<?php blogtxt_post_class(); ?>"> <h2 class="entry-title"><?php the_title(); ?></h2> <div class="entry-content"> <div class="mybotbox"> <a href="http://www.stumbleupon.com/submit?url=<?php the_permalink(); ? rel="nofollow">&title=<?php the_title(); ?>"><img src="http://www.stumbleupon.com/images/stumble7.gif" width="100"></a> <div style="float:left; margin:0; padding-left:22px;"><?php if(function_exists('digg_this')) { digg_this('', '', '', 'tech_news'); } ?></div> <p style="margin:0;padding-left:10px;"><a rel="nofollow" href="http://technorati.com/faves?sub=addfavbtn&add=http://andybeard.eu" rel="nofollow"><img src="http://static.technorati.com/pix/fave/btn-fave2.png" alt="Add to Technorati Favorites" /></a></p> <p style="margin:0;padding-left:14px;"> </p> <p style="margin:0;padding-left:10px;"> <?php if (function_exists('bump_this_widget')) bump_this_widget(); ?> </p> </div> <?php the_content('<span class="more-link">'.__('Continue Reading »', 'blogtxt').'</span>'); ?> <div style="clear:right"> </div> <?php link_pages('<div class="page-link">'.__('Pages: ', 'blogtxt'), "</div>\n", 'number'); ?> </div> <!-- <?php trackback_rdf(); ?> --> </div>[/code] The CSS is fairly simple too [code] .mybotbox{ float: right; margin: 10px 5px 10px 10px; width: 100px; height: 250px; min-height: 250px; background:#ffffff; padding: -8px 0em 0em 0.5em; } [/code] I still have a few bugs in the CSS on this theme, but it will get fixed gradually. I make no claims to being a designer. Just a floated DIV before the content

[code]< ?php the_post(); ?>

< ?php the_title(); ?>

< ?php if(function_exists('digg_this')) { digg_this('', '', '', 'tech_news'); } ?>

Add to Technorati Favorites

< ?php if (function_exists('bump_this_widget')) bump_this_widget(); ?>

< ?php the_content(''.__('Continue Reading »', 'blogtxt').''); ?>

< ?php link_pages('

'.__('Pages: ', 'blogtxt'), "

\n", 'number'); ?>

[/code]

The CSS is fairly simple too

[code]
.mybotbox{
float: right;
margin: 10px 5px 10px 10px;
width: 100px;
height: 250px;
min-height: 250px;
background:#ffffff;
padding: -8px 0em 0em 0.5em;
}
[/code]

I still have a few bugs in the CSS on this theme, but it will get fixed gradually. I make no claims to being a designer.

]]> By: SEO Tips, Techniques & Tutorials http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20428 SEO Tips, Techniques & Tutorials Sat, 05 May 2007 23:37:52 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20428 BTW-Andy, I'd really like to know how you manage to incorporate "Stumble Upon, Digg-It,Techn Fav & Bump-It" in your 'post'. Do you have to add the codes (with the plugins activated) every time when posting or via the template files? BTW-Andy, I’d really like to know how you manage to incorporate “Stumble Upon, Digg-It,Techn Fav & Bump-It” in your ‘post’. Do you have to add the codes (with the plugins activated) every time when posting or via the template files?

]]> By: SEO Tips, Techniques & Tutorials http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20427 SEO Tips, Techniques & Tutorials Sat, 05 May 2007 23:31:59 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20427 Hmn...I think this feature/tag will be abused by all the spammer who knows how to abuse it (i don't want to give any idea here). But not so much for 'most' people. It would be nice for Yahoo to implement a 'spam alert' system where using too many of those tags triggers the alarm. Anyway, I think it's much better to have a tag similar to Google Section Targeting tag for Adsense rather than this one: less place to place the tags. Hmn…I think this feature/tag will be abused by all the spammer who knows how to abuse it (i don’t want to give any idea here). But not so much for ‘most’ people. It would be nice for Yahoo to implement a ’spam alert’ system where using too many of those tags triggers the alarm.

Anyway, I think it’s much better to have a tag similar to Google Section Targeting tag for Adsense rather than this one: less place to place the tags.

]]> By: The Venture Skills Blog Friday round up - Digg Eaten | Yahoooooo | money is power « http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20067 The Venture Skills Blog Friday round up - Digg Eaten | Yahoooooo | money is power « Fri, 04 May 2007 23:00:23 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-20067 [...] only to Yahoo and causing a large amount of controversy around the blogs. Another great write up by Andy Beard covers most of the perceived problems and offers a nice solution to spotting hidden [...] [...] only to Yahoo and causing a large amount of controversy around the blogs. Another great write up by Andy Beard covers most of the perceived problems and offers a nice solution to spotting hidden [...]

]]> By: Sebastian http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-19842 Sebastian Fri, 04 May 2007 13:08:25 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-19842 I don't listen to whining designers arguing "SEO classnames conflict with CSS" because that's not true. What is true is that such crawler/indexer directives are meta data which should not alter or influence the markup on element level. Referencing existing classes and DOM-IDs in robots.txt to assign crawler/indexer directives would be elegant and way more flexible. And it would save bandwidth and maintenance costs. I don’t listen to whining designers arguing “SEO classnames conflict with CSS” because that’s not true. What is true is that such crawler/indexer directives are meta data which should not alter or influence the markup on element level. Referencing existing classes and DOM-IDs in robots.txt to assign crawler/indexer directives would be elegant and way more flexible. And it would save bandwidth and maintenance costs.

]]> By: Andy Beard http://andybeard.eu/712/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-19590 Andy Beard Fri, 04 May 2007 00:52:54 +0000 http://andybeard.eu/2007/05/robots-nocontent-how-to-spot-abuse-by-sneaky-webmasters.html#comment-19590 That partially depends on "how" legacy they are. If sites used some PHP or server side includes they might well not have too many problems. If they didn't, they might still be able to use some search and replace with a regex based tool. There are some fairly smart search and replace tools available, and if anyone is maintaining a large site, they probably have to use these already. I am not anti-nofollow, I find it easier to use than messing with javascript - I am not anti this tag either in the sense that I like having control, and it is better than having to make graphic elements for stuff like shipping information which SEOs already do, but are not ideal for accessibility. There is also the question of footprints - this could effectively at least smudge them a little. Whether it will be abused at a later date by the search engines reinventing the purpose is hard to tell at this point. Maybe if the SEs backed down a little on nofollow for paid links, or gave clearer signals, people would be happier about this implementation. I don't buy the "designers" crying about a break in standards and purity of CSS. Allowing a configurable class as currently discussed on threadwatch might be useful, or the ability to define page elements. That partially depends on “how” legacy they are.

If sites used some PHP or server side includes they might well not have too many problems.

If they didn’t, they might still be able to use some search and replace with a regex based tool. There are some fairly smart search and replace tools available, and if anyone is maintaining a large site, they probably have to use these already.

I am not anti-nofollow, I find it easier to use than messing with javascript – I am not anti this tag either in the sense that I like having control, and it is better than having to make graphic elements for stuff like shipping information which SEOs already do, but are not ideal for accessibility.

There is also the question of footprints – this could effectively at least smudge them a little.

Whether it will be abused at a later date by the search engines reinventing the purpose is hard to tell at this point. Maybe if the SEs backed down a little on nofollow for paid links, or gave clearer signals, people would be happier about this implementation.

I don’t buy the “designers” crying about a break in standards and purity of CSS.
Allowing a configurable class as currently discussed on threadwatch might be useful, or the ability to define page elements.

]]>