Google Moves Towards Continual Crawl Based Link Devaluation

Since November 22nd we have been noticing a new trend with how Google applies algorithms to detect bad links. Earlier in 2012 the focus was very much around major penalisations where a site would literally disappear overnight for lots of keywords but the trend across November, December and January is very much about Google reducing the rankings of certain sites gradually week after week.

We all know that Google has algorithms to detect bad links and the evidence of this was very visible earlier in 2012 but with this latest change Google seems to have developed the capability to continually detect and devalue bad links as part of the standard crawl process, something they were never very good at before. The net result of this is that as Google completes a crawl of a website and the sites linking to it they can devalue those links in real time rather than waiting to roll out a major update such as Penguin or a manual link penalty.

A lot of sites that are suffering are ones that may have been penalised before due to bad links and have not yet removed them all but there are some that are losing ground from a strong position which indicates the latest change is not confined to sites that were hit in the first half of 2012. Google has made it clear that the disavow tool works on a crawl basis whereby all sites submitted will be crawled before the link weight can be cancelled out so there is every reason for them to have merged this capability with the "bad site detection" algorithms they already have in place.

Realistically Google has to move towards a continual process because manual penalisations are not scalable in the long term and they much prefer algorithms over people.

Below are some examples of this trend as shown on Searchmetrics for some of the several hundred sites we monitor recently as part of our competitor analysis. There are lots more that we have seen similar patterns for.

I have only included the past few weeks data rather than the full year because I didn't want to identify any sites in particular. We would be interested to hear from you in the comments if you have noticed any sites on a gradual slide recently and whether they had previously been penalised or not?

BY Patrick Altoft AT 2:59pm ON Friday, 18 January 2013

Patrick Altoft is Director of Search at Branded3 and has worked in the SEO industry for over 10 years. With experience across some of the worlds largest brands as well as startup businesses Patrick is well known in the industry and speaks regularly at the major SEO conferences and events. Follow Patrick on Twitter or Google+

Comments

• I Have Noticed a distinct change in previously penalised sites that I am monitoring. It is looking more and more like a backlink to page based slap, with traffic and rankings increasing all around the 'bad' pages.

  I have also seen the sort of patterns you describe with what I can only describe as gradual link value rot where links are not dropping off but there is a gradual slide.

  Putting the two together it could be assumed that Google is moving towards a more laser guided slap, rather than a 'throw the whole site out' slap. Which of course makes sense, as they get to keep the quality content, without having the spammy pages show up.

  I really hope that google do get their act together with regard backlinks as there are a lot of people suffering out there, and nothing they can really do about it due to the fact they don't know what links are hurting them as WMT doesn't show them all.

• It could also be usage metrics – I have soem data to suggest this may be the bigger culprit – if interested just tweet / email me

• sometimes i wonder if they won't be happy until almost all links are harmful to your site

• Hi Patrick,
  Great Information, Do you recommend any Link building strategy or any other SEO strategy to get back penalyzed sites ?

• Very insightful post, Patrick. Using the Searchmetrics seems to be very useful when doing the competitors analysis. Thanks a lot for making a great effort to compile and share this invaluable information backed up with the graphs.

• I experienced a major drop in ranking for some of my targeted keywords previous week. And as far as I remember, there are no spam links or link exchange. And even the site serves good content to the users and search engines. Perplexed what to do next ??

• Another Algorithm rolling out right ?

• Hmm, they have always been after this, but I just don't think they are at the necessary level yet for real-time link valuing/devaluing.