Something interesting has just come across one of my networks (hat tip to datadial), just a few days after Digg have announced that they are building a replacement for the much loved Google Reader, they have (coincidentally?) disappeared from the primary google index.
Is it an SEO penalty for links? That seems to be the number one reason that brands are getting booted from google’s index these days… Some conspiracy theorists will no doubt be proclaiming its something to do with their announcement to build a replica of the now defunct Google reader, but personally I really cant see that having any effect. Could there?
Doing a site: search for Digg certainly demonstrates that they are no longer in the index:
Its likely (only if it is link based however) that it would be down to what individuals who submit content do after the fact – ie. sending spammy links at their posts to try and build the pagerank, and create “authority” which they then pass back to their own sites. Digg has long been listed in every “linkwheel” sellers handbook, and if that is the reason then what does it mean for every community site on the internet?
Will we have to manually aprove all new links soon at this rate? Come on Google – WTF – let the internet know what you’re doing please.
UPDATE: Its looking like Digg may have suffered a technical SEO issue with their robots.txt file being unavailable to google’s crawlers. In a nutshell what this means is that google could not see a vital part of their site, which controls access to the spiders. In these circumstances Google may remove the site – however this would probably be the biggest example seen of such a problem.
UPDATE2:
I have had a tweet from Matt Cutts, head of Google’s Webspam team where he references a post he made on Ycombinator, explaining the reason for Digg’s disappearance:
“We’re sorry about the inconvenience this morning to people trying to search for Digg. In the process of removing a spammy submitted link on Digg.com, we inadvertently applied the webspam action to the whole site. We’re correcting this, and the fix should be deployed shortly.
From talking to the relevant engineer, I think digg.com should be fully back in our results within 15 minutes or so. After that, we’ll be looking into what protections or process improvements would make this less likely to happen in the future.”
Some personal notes: Its fantastic that Google have taken the time to let the online community know what has happened, and we cant really hold any ill-will towards them for a human error, its a testament to their professionalism that this hasnt to my knowledge happened in the past, and we applaud their transparency in this matter. I guess the only concern for the average webmaster might be what would happen if this happened to their site, and didnt receive the kind of attention that this did.
Still, thank you Matt, and Google as an organisation for the clarification.
Please, voice your opinion in the comments below!
