Google is home to more than 130 trillion web pages.
However, not all of them have existential value, and many even fail to open.
If you are an owner of any such web page, what should you do with these URLs? That’s what we are going to discuss in this blog post.
Why remove a URL from Google Search?
Removing a URL from Google search might sound tricky at first because you may think what’s the harm if a page exists.
But Google’s algorithm keeps track of the individual links as well as the complete website.
You should consider removing URLs because:
- If you have the same content on multiple pages on your website, the SEO link juice will be divided amongst URLs, thereby affecting page rank. Consider it as a leaking bucket where the SEO value is literally leaking to non relevant pages.
- When a URL on your website no longer provides any useful information, you should remove it to improve user experience and page SEO.
- If your web page doesn’t match the search intent, or if your URL is misleading Google, you should consider removing it and replacing it with a better one.
Ways to remove a URL from Google Search?
You have the privilege to follow the approach of “there’s no one fit for all” when you remove unnecessary links from Google search.
Google itself provides a potent tool for link removal, but other methods can also be put to use. They are mentioned below:
Method 1: URL removal tool – using Google Search Console
Follow these steps to remove the URL:
1. Log in to the Google Search Console account.
2. Within the “Index” menu, look out for “Removals”.
3. Then click on “New Request” as shown below.
4. Next, fill in the URL that you want to be removed from Google’s index for 6 months and leave the rest to Google. You will get a confirmation after the links are removed. The same would also reflect in the list of pending URLs for removal.
5. Keep in mind that this tool will temporarily remove the URLs (for around 6 months). These pages will reappear after this time period.
Things to note before removing URLs via Google Search Console
When not to use this tool
You should not use this tool when you want to modify specific aspects of any URL like hiding all pages with “www”.
This tool will remove all URLs, and you’ll lose your precious web pages.
When you’ve deleted a page that opens to a search query, Google removes such links by itself.
Therefore, you don’t need to use this tool for this case too.
Misuse of the removal tool
This tool offers the fastest way to remove (temporarily) a URL from Google’s index.
However, it’s purpose is restricted to hiding the pages from Google’s search results and not from your console account.
Manually removing these URLs from your Console account can affect your website too.
Also, if you want to differentiate different versions of similar web pages, you shouldn’t use the removal tool.
This is because you might end up removing all the URLs, including the ones you never wanted to remove.
How to cancel a removal request
If left untouched, the page won’t show up for a minimum period of 90 days.
However, if you want to provide visibility to that URL before this period, you need to open the “Remove URLs” section.
Choose the URL for which you wish to cancel the removal request and then click on “Reinclude”. The page will be back within 4-5 business days.
Method 2: Noindex
Noindex is an HTML meta tag that informs Google and other search engines not to index a particular web page.
It’s a single line code, but it duly fulfils your aim of making sure that a specific URL shouldn’t make it to the SERPs.
Once you are done with edits and your page is ready, you can remove this tag from the code and the page would once again become available for indexing.
Indexing is necessary because Google’s ranking algorithm only ranks indexed pages.
Here’s a sample for the Noindex meta tag:
Don’t worry if you don’t have access to the HTML code. If you are administering the website on a CMS (content management system), you can find the option under the SEO plugin that you’re using.
Here’s how to implement noindex within the Yoast plugin in WordPress
As you can see, you have to toggle in the page settings and click on “No” to remove the page from Google’s indexing.
If you want the URL back on track, change it to “Yes”. Within a few days, Google will consider the web page for indexing and consequently assign it a rank.
Method 3: Restricting access
Restricting access to any URL entitles it to be visible to some users while it stays un-indexed by the web crawlers.
Consider this example where Jason wants to gain feedback for his new website.
He already has a subscriber base on his previous website, and he provides them exclusive access to visit the new one. At the same time, access is restricted for a random visitor.
As soon as he modifies the website after working on the feedback, he can list it for indexing. It’ll also become accessible to all the users.
You can either implement IP Whitelisting which enables particular IP addresses to open the restricted URL.
Another method to do so is by coming up with a login portal, and you get to decide who should get access to it.
Method 4: Delete the content
Google doesn’t want to feature results that ultimately redirect to 404s (not found) or 410s (gone).
For the same reason, if you remove the web page and place one of these redirects, Google’s algorithm will not feature these pages in search results.
However, this doesn’t take place immediately, i.e., the page is not removed then and there.
It is removed after a recrawl takes place. In between this phase, the URL may still show up in the search results. One may also encounter a cached version of it.
Method 5: Canonicalisation
Canonicalisation isn’t the most reliable method for removing URLs, but it does the job nonetheless.
The process involves canonical tags and redirects that inform crawlers about traffic retention and your intent for that page.
When you have similar links or in some cases, identical web pages, the crawler can get confused.
With the help of a canonical tag, you can specify which page or version you want to be treated as the primary one.
Another way is to use redirects that divert traffic permanently and temporarily – 301 and 302 respectively.
The 301 redirect informs the bot that you want the new version of the URL to be indexed and ranked.
This prevents the deterioration of link juice, and you don’t lose out on your traffic even after the original link dies out.
With redirect 302, the web crawler gets the information that the redirection is temporary. Therefore, your original link is treated as the primary URL.
A pictorial summary of removing URLs from Google Search
Removing links from Google search is dependent on what you envision for that particular web page.
The flowchart from Ahrefs below describes the various circumstances and the appropriate method you should follow for URL removal:
What to do when you don’t own the Content?
Copyright issues frequently happen in the digital industry, and you need to file a complaint about taking that page down.
Many feel that they cannot take action against such wrongdoings, but there’s DMCA for your help.
You have a plethora of security options at DMCA apart from raising copyright concerns:
The procedure takes time, and you need to put forth proof of being the owner of that content.
After successful trials, the accused website needs to take down the content within the specified time. That’s some way of removing a URL, isn’t it?
Common removal mistakes to avoid
While removing URLs from Google search is beneficial for SEO and eventually scoring better SERP rankings, following mistakes should be avoided:
1. Using robots.txt to block URLs
When you use robots.txt to block URLs, the traditional meaning of “block” doesn’t stay true.
This is because even though you have prevented the URL from being crawled, the internal pages are still indexed by Google.
Since Google has not crawled the specific URL, it won’t know of the content or attributes of that link.
But this doesn’t restrict Google from not only indexing the co-related pages and also show them in search results.
No-follow is often confused with noindex. However, there’s a clear fundamental difference between the two.
While noindex is a meta tag “command” that prevents the URL from being indexed, no-follow is just a “hint.”
Google has made it official that no-follow pages are open to crawling and would depend totally on the crawler.
This implies that there can be a scenario where the page with no-follow is crawled every time a crawl takes place.
No-follow should, therefore, be avoided as a measure to remove a URL. That’s because it simply fails to provide you with the assurance of a URL removal.
3. Noindex and canonical to another URL
This is a combination that may or may not raise a contradiction. As mentioned above, noindex is used to ensure that a particular URL should not be indexed.
On the other hand, a canonical tag is used to divert the attention of the web bot to a new version of the web page.
The idea behind using both of these methods simultaneously is to make sure that the existing page is not indexed.
Complex algorithms are working together, and they may very well consider the signals of the noindex tag.
If that happens, crawling won’t take place, but internal pages would still be indexed. The result? The page would still show in the search results.
There you have it — different methods for removing URLs that serve differential purposes.
You now know how to remove URLs temporarily from Google search and permanently as well.
The knowledge of canonical tags and redirects helps you to retain link juice while you work on new web pages.
With precautionary measures, you now also know about common mistakes that you need to avoid for safely removing URLs.
It’s now time for you to evaluate the links that you own and remove the ones that don’t add any value.