Why You Should Never Blanket-Redirect All Your Invalid URLs

Here’s a conversation I’ve found myself having too many times with clients over the years not to feel like it would be worthwhile to make my sentiments public. So here goes: it’s well-intentioned but short-sighted to configure your server to redirect all invalid URL requests to the homepage.

At first blush, the idea seems like it would offer a clear enough user experience benefit to be worth considering. Instead of the cold-water shock of a 404 page — which, however gentle, is still at its core a message that the user has done something wrong — you’re offering the front door of the site, from which the user would theoretically have the easiest time finding the page he or she was originally looking for (or any other). But the signal of the 404 error page is sent in two ways: in the form of the error message and available options presented to the user, and in the form of the 404 response code presented to the server making the request. There’s a reason why that code exists.

Implementing the blanket-redirect means invalid URLs are going to return a code of 301 (permanent redirect) rather than 404 (not found), which means that servers requesting an invalid URL aren’t going to get the message that it’s invalid. It’s just going to look like a redirect. This means that search engine bots won’t know when their crawl lands them on a bad URL, which in turn means they might not purge the URL from their index. It also means that if you’re the webmaster, you will have no way of knowing what URLs need fixing. Broken link management is a core task of webmastery and you can’t manage what you can’t see.

Now don’t get me wrong: configuring page-specific 301 redirects for pages that you’ve removed is not only a good idea but by every measure a better choice than letting the pages 404. If you take a page down and replace it with a new page that makes for a fair content match, then by all means, set up a 301 redirect from the old page to the new and make it effortless for users, bots, and link equity to get where they’re trying to go. That’s a matter of assigning an expiring page a forwarding address, and it’s a basic best practice. [Example: you run a donut shop, and one day you decide to lose the plain cake donut that almost nobody buys, but keep the glazed cake one. You 301 redirect donutstopbelievin.com/menu/plain-cake/ to donutstopbelievin.com/menu/glazed-cake/. Fully legit.]

What I’m cautioning against is a catch-all that would respond to any request of a URL not found – that is, any URL that didn’t retrieve a page or a specific 301 redirect as described above — with a 301 redirect to the homepage. [Example: dignifying a request for donutstopbelievin.com/4hfd74T4rgGxZnm/ with a homepage redirect.] Here are some potential site problems that no amount of scanning your server logs or your Google/Bing webmaster consoles would ever inform you of, were you to set that up:

• If another site had linked to yours, but mistyped the URL they were trying to link to;
• If a previous webmaster had taken down a once-popular page without setting up a page-specific redirect, and people were still trying to find it via bookmarks or external links;
• If a page were retained in search engine indices after being taken down and the poor folks clicking through from search results were ending up on the homepage;
• If a page were removed from indices because it returned a 301 for long enough, but if search engines took the redirect to mean that the content of the homepage should be considered a substitute for the content of the removed page.

But what about that supposed user experience benefit that comes with keeping your users’ delicate eyes away from the bad, bad 404 page message? Isn’t it better to put them on the bright, welcoming homepage than in front of a great big Dead End sign? Wouldn’t that outweigh the cost of the above risks? Frankly, no.

First of all, any 404 page worth its salt is going to have good UX built in. If the site is following usability best practices, the error message will explain — in a lighthearted tone — that the page requested couldn’t be found, and the page will retain the navigation menu and search bar, and will probably also provide conspicuous links to the homepage and other top pages. Take the starbucks.com 404 page as a particularly strong example:

Secondly, if the page that your user was requesting genuinely couldn’t be found, wouldn’t it be better to just shoot straight and say so instead of relocating the user to the homepage for no stated reason? Remember in the early days of social media on mobile when you’d click a shared link to a publisher’s site, and the universal redirect from the desktop version to the mobile version of the publisher’s site would fire, destroying your URI string and spitting you out on the homepage? The feeling invariably was “What? I was trying to read this specific article and I ended up on the homepage? Why?” A homepage redirect under 404 circumstances furnishes the same experience. Better to disappoint your users with the news that the page they wanted isn’t there than to confuse them.

And, again, there’s the possibility that the blanket-redirect approach could result in invalid URLs being retained in search engine indices. Imagine someone clicking through on a search result for a page that you’ve taken down and ending up on the homepage, again for no apparent reason. Is that a good user experience scenario? Not so much. Also, if your site were large enough, this approach would mean your server would be processing potentially hundreds or thousands more redirects per day than it otherwise would have to, creating undue labor which would be passed to all your users in the form of slower load times. Also not good.

So in closing: the blanket-redirect approach forces SEO sacrifices, robs you the webmaster of information vital to proper site maintenance, and creates potential server strain, all in the name of a supposed user experience gain that itself vanishes upon closer examination. When anyone — user or bot — requests a URL that neither currently exists nor resembles one you purposefully removed, show them the 404 code. You’ll all be better off that way.