We use cookies to improve your experience. Please read our cookies policy here.


How do I ensure my website is being indexed?

3 minute read


In today’s video I look at how you can ensure your website is being indexed by Google. This includes what you can do to make sure Google has discovered your website, how to find out how many pages have been indexed by Google already and what problems you may be facing if you find that some/all of your pages have failed to be indexed.

Covering Google search console accounts, internal linking and creating and submitting an XML sitemap, have a watch of my video to learn more.

Video Transcript

[music] ♪ Hello and welcome to the UK Domain. Today’s question is from Charlotte Woodruff. She asks, “How do I ensure my website is being indexed?”

While the search engines are generally pretty good at discovering and crawling new content, there’s lots of things you can do to help them along the way. So I’ve covered some of those over here.

So the first thing I recommend is that you make sure you’ve got a Google Search Console account. Now this is a free tool offered by Google. If you go to google.com/webmasters/tools, you can sign up there, and it will allow you… It got lots of very handy tools, but for the purpose of this exercise, I was going to talk about the indexing. So to begin with, if you haven’t already, you can submit your URL or your domain to Google. That’s the very first step.

The next thing you should do is create an XML site map. So this is basically a directory of all of your pages on your website. It allows you for each of your pages specify the priority you’re giving to that page, so the home page would probably be a priority one. You might have category pages, which could be an 0.9, and then your product pages, which would be an 0.8. That basically allows Google to see how…the importance that you’re giving those pages.

You also got the last mod date, so that shows the last date the page was actually modified. And then you could submit this to Google via Google Search Console, and then Google will just get straight to work and start crawling all of the pages if suggested, and hopefully start indexing them.

Another point that’s very important and always important for SEO, that’s internal linking. So for Google to be able to discover a page, it has to be able to follow a link to get there. So, for an example here you’ve got your home page, the spiders will come in, and they’ll see all of your category pages, which have been nicely linked to, and then from those category pages, they’ll follow the links through to the product pages and they’ll be able to discover everything. And if you want to actually find out how many pages you’ve got indexed in Google already, you can also do that in Google Search Console. So if you go straight into Google Search Console, so click on Google Index, and then below that you’ll have Index Status. If you click on that, you get a nice graph which shows you over time how many pages that you’ve had indexed. You might see some peaks and troughs and you’ll be able to investigate what caused those.

Another nice way of doing it is just using Google Search. So there’s the what’s known as search operators. If you just go to google.co.uk and you put your domain, so if you put site:yourdomain.co.uk, whatever your domain is, without any spaces, and click Enter, it will bring back all of the results. Now if you look at the top it will say how many results it’s brought back and that will give you an indication of how many pages Google has indexed. If you’re having problems with this and you’re finding that none of your pages are being indexed, then you can quickly go over.

So it’s a good idea to check for the no index tag, so that’s a special tag that you can have to the code your website to prevent Google from indexing your page. So you want to look out for that and make sure it’s not just been put site wise on all of your pages. Another thing to check is your robots.txt file. So you can normally find that at your site.co.uk/robots.text. And if you see something like this, so disallow:/ that’s basically disallowing the crawlers from indexing any of your website, so that’s really bad. So that could be two problems. But also you can use those to prevent Google from crawling pages that you don’t want them to index. So I hope that answers your question. Thank you very much. ♪ [music] ♪


Sam Gooch is Web Optimisation Lead at the UK Domain. Sam leads the online search strategy, combining both organic and paid advertising to drive awareness around our brands and content, helping small businesses to succeed online.

Sign up to the UK Domain newsletter

Get all our monthly news and updates direct to your inbox