Is The Guardian having problems with its domain name migration?

The Guardian, seeking to appeal to a more international audience, recently changed its domain name from guardian.co.uk to theguardian.com. 

It seems that the newspaper's traffic has dipped as a result of the migration, with some tools showing a drop in key metrics which affect its rankings. 

We have first hand experience of this issue at Econsultancy, as we migrated our domain back in 2009, with a drop in referral traffic from Google being the consequence. 

So, has The Guardian handled this migration correctly, or is there more that Google should have done to help such a massive site with the change? 

The Guardian migration: drop in traffic? 

The Telegraph recently reported that the domain name change has led to difficulties, with traffic dropping as a result. It also mentioned that 'senior executives were seeking urgent help from Google'. 

The Guardian itself acknowledges this problem in a recent report on the latest digital ABC figures: 

Guardian News & Media's website network reported a 5.83% month-on-month dip in daily browsers to 4,519,849, as a transition to a new global domain, theguardian.com, affected traffic. Monthly browser numbers fell 1.56% to 83,519,898.

Our research analyst Andrew Warren-Payne has been looking at the number in Majestic. This first chart shows the results for the old .co.uk domain: 

You can see the difference when we look at guardian.com, with a big drop in the number of backlinks and referring domains: 

There is a caveat here, as pointed out below, as the data reported by Majestic may not yet be fully up to date. 

Joost de Valk, who has been helping with the migration, isn't too concerned:

We're not concerned about traffic at all, numbers are looking good and in fact, ahead of expectations. Migrations like these always take some time to settle in and may sometimes be a rough ride.

So far though, I can honestly say it's looking good and better than I've ever seen a migration of this size - if there ever was such a thing - turn out. Of course there are still things to fix, as there always will be on a site this size, but we're well aware of our issues and fixing those.

I've been asking search experts for their views on the migration, and whether Google should be making this easier for sites...

Are domain migrations such as this supposed to be easier now? Is it not possible to 'tell' Google what you are up to?

Kevin Gibbons, UK MD at BlueGlass Interactive:

There are more ways than ever of implementing site migrations, and the more signals you can send Google that your website has moved, the better. In theory at least!

However, in practice domain migrations are still always a risk and you have to accept that there will be short-term loses and unexpected problems. Even if you've done this hundreds of times before, Google is an unpredictable beast at the best of times!

You really have to make the decision having carefully balanced out your options and considered the short term loss vs the long term gain, which I'm sure is what happened in this case.

Dan Barker, E-Business Consultant: 

Funnily enough, you can tell Google what you're up to. There is a 'change of address' tool within Webmaster Tools. Domain migrations don't always go entirely smoothly, but they are definitely far more reliable than they were 10 years ago.

Julia Logan, SEO consultant (better known as Irish Wonder): 

In case of a large site it is totally up to Google how fast it would fully reindex the new site instead of the old one. Domain migrations of large sites have always been problematic due to sheer volume of the site.

Has the Guardian done something wrong? If so, what?

Kevin Gibbons:

Not necessarily. It's very difficult to judge from an external perspective especially at an early stage where progress is unclear, as there's always different methods, reasons and goals when implementing changes like this.

Yes, it does appear that the Guardian has lost significant UK organic visibility and market share since the domain migration: 

However, it has started to make traction in the US which is important to remember as a key goal in the move: 

I suspect that providing the UK improves back to where it was in the medium-term (rather than long) and the US organic traffic continues to rise, the short-term loss will be outweighed by the overall long-term gains.

Dan Barker: 

I think it's actually done pretty well. The paper had four options really:

1. Stick with the old .co.uk domain and try and make a go of it internationally.
2. Launch global content on a different domain to the .co.uk, keeping the UK-centric content where it was.
3. Relaunch on a more internationalised domain name.
4. Launch sites for individual regions one at a time.

The Guardian had tried the fourth of those to an extent previously, and obviously decided now was the time to go big bang.

The timing couldn't have been better, as it had the enormous series of NSA/Prism splashes which drove both a lot of traffic (nullifying any loss in ad revenue that domain migration might cause), but also drove lots of social shares, links, etc. to help support organic search results in the longer term.

Quietly, it's also made the site adapt to the device you're viewing it from. Try loading the same URL on a phone and on your desktop to take a look.

There are a few odd bits and bobs in there, for example you can see there's piles of ancient stuff lying around still on guardian.co.uk with links that could have been redirected somewhere.

For example: http://education.guardian.co.uk/universityguide2005, http://blogs.guardian.co.uk/quran/, and so on. 

Lots of this is terribly old, legacy content, but it has all picked up valuable links over the years, and would be worth either moving and redirecting as-is, or killing off and redirecting the URLs to somewhere as appropriate as possible.

There are some really odd 302 redirects too. For example m.guardian.co.uk, which itself has a pile of links, 'temporarily' redirects to www.theguardian.com. That doesn't really make sense to me as they've standardised on 'www.' as far as I'm aware, but perhaps there is a logic to it I'm unaware of.

Another really odd one is http://witness.guardian.co.uk which is a fairly new project that's now being 'temporarily' redirected to http://witness.theguardian.com. Again, perhaps that project is being shelved or there's some other logic I'm unaware of. 

Most of these temporary redirects are 302, but there are a couple of 303s among them. That usually either means someone's being really clever, or that a maverick developer has gone crazy.

Julia Logan: 

I am not sure what it was trying to achieve but from what I see, the old domain still has more than 6million URLs in Google's index, mainly content on subdomains of the old site (e.g. blogs.guardian.co.uk, travel.guardian.co.uk, etc.).

Moreover, there is no reason for Google to deindex them as at least some of them are still alive and not redirecting to the new domain. From what I can see, only the main domain has been redirected but not all the subdomains.

Whether or not this has been the Guardian's goal, I have no idea but looks like it's not making things any easier.

All the links on the old subdomains that currently are alive still point to the old domain version, even for pages that have been moved to the new domain. That surely sends Google a dubious signal.

What, if anything, can the Guardian do to fix it?

Kevin Gibbons:

It would require knowing the goals from the domain migration. Is the Guardian happy with where things are at this stage?

I think it's clear that short-term there were certainly better options, one being to stick with guardian.co.uk for the UK site and then implementing an option such as 'hreflang' for additional territories or even canonical tags to run the UK content side by side.

This seems like a logical solution which places less risk on the UK and allows them to build domain authority on theguardian.com in the meantime, leaving the option to make the full switch in the future.

It has also applied a geo-detection redirect based on where you are accessing the site from, which can often be tricky to get right with SEO, but again this is very easy to say without an inside view, as there's always technological challenges and internal thought processes behind these decisions.

That may have worked before, however a roll-back at this stage seems very extreme and decisions like this aren't made on short-term gains, quite the opposite. Plus it's unlikely that this purely for SEO reasons, it's an online branding move in many ways too and allows them to enter new global territories, specifically the US.

So while the above option makes more SEO sense in my opinion, it may not have been practical in this scenario.

Obviously everyone always wants the best case scenario, but you also have to be prepared for the worst. If it has analysed the situation and is confident that the best decision has been made in the mid to long term, the answer is likely to be patience.

Dan Barker: 

I'd be surprised if the Guardian isn't already on it, but it would be worth going through all those legacy subdomain.guardian.co.uk assets figuring out whether to ditch them, move them, or simply redirect the URLs to somewhere more appropriate.

It has a fairly small but excellent team over there, and the paper brought in @yoast to help out with the migration on top of the already excellent in-house/contract people.

To be honest, I'm not sure the Majestic data does tell us there's been an enormously big issue. It's tough to tell without actual webmaster tools or sitecatalyst access. If I remember right Majestic's fresh index covers a 90 day period, and The Guardian domain migration only happened a couple of months ago.

If you look at the link flow and citation flow metrics too, you'll notice those are actually higher for theguardian.com. Majestic SEO is a wonderful tool, one I pay for every month and would recommend to anyone, but I wouldn't take the two reports here as an indicator that the wheels have come off.

Some axle alignment would be useful, but, to overstretch an already tedious analogy, the car is still a mostly-valeted Dodge Ram with neon chassis lighting, a fresh christmas tree air freshener, and a tank 3/4 full of diesel.

As a side-note, if you take a look at the Doubleclick Display Ad Planner stats for theguardian.com you'll see it's listed at a fairly healthy 25-30million impressions a week vs. the rather tragic 500k-1million impressions thesun.co.uk shows following the erection of its paywall. 

Julia Logan: 

If the purpose has been to just move everything completely to the new domain, there is definitely a lot of work still remaining to be done.

Check everything that hasn't been redirected, redirect it, double check again. If, on the other hand, it did intend to leave some of the old subdomains alive, the paper needs to fix all the links from those pages to the content that has moved to the new domain.

Is domain migration a common problem? Could Google do more to help here?

Kevin Gibbons:

You have to appreciate that Google's in a very difficult position here. The easy answer is it can pass signals across from redirects quicker so that the new domains rank quicker.

The difficulty is that if it does that, SEOs will start to exploit it, meaning that people will overuse the tactic of redirecting purchased authority sites into new domains for SEO gains.

So actually what it is doing in making this difficult, whilst frustrating for legitimate redirects and brands such as The Guardian, does make sense.

You have to build a certain element of trust and reputation in your own right, as a new domain, in order to gain Google's full trust. Fortunately as a huge publisher, the Guardian is in about as strong a position as you can get in order to be able to recover from this. 

It will certainly be interesting to see how this develops over the coming months, but personally despite what appears to be frustrating results, so far I wouldn't see a need for a kneejerk reaction.

Dan Barker: 

It's less of an issue than it was years ago, but any big technical changes are always a risk. Bigger problems now are usually either site relaunches where URLs change with no direct one-to-one mapping, or internationalisation where suddenly a single-country site is split into different regions and languages.

Google has tools and notes on most of these areas, so much of the battle is simply knowing there's likely to be a problem, and knowing where to look to begin mitigating the risks. 

Outside of that though, and perhaps a problem with knock-ons in other areas, is that social networks don't make it easy to retain social share history on domains.

For example, take a look at this article which has 2,700+ comments, but very low tweets and Facebook shares.

It's worth noting that the Google+ number has updated, indicating that it's taking a bit more interest in this area than Facebook, Twitter, and the other big social networks.

Julia Logan: 

Google could certainly do better in terms of dropping pages no longer alive and switching to the new domain instead, if told so explicitly by the verified domain owner.

However, with large sites, there is a tremendous amount of crawling involved, so surely that takes time. 

What do you think? Is The Guardian suffering unduly as a result of a standard domain name migration? Do you think Google should do more to assist? Let us know below...