If a page is blocked from Google by a noindex tag or robots.txt, it will never appear in search results. This is a common issue, especially after site migrations, redesigns, or CMS updates. In this guide, you will learn how to check if a page is blocked, the difference between noindex and robots.txt, and how to fix blocking issues.
Noindex vs robots.txt: what is the difference
Both noindex and robots.txt can block a page from search, but they work differently.
Noindex is a meta tag or HTTP header that tells search engines not to index the page. The page can still be crawled, but it will not appear in search results.
Robots.txt is a file that tells search engines not to crawl certain pages or sections. The page is not crawled, but it can still appear in search results if other sites link to it.
Key difference: noindex removes a page from the index. Robots.txt prevents crawling but not indexing.
How to check if a page has noindex
Three ways to check.
Method 1. Noindex checker tool
The fastest way is to use an online noindex checker. The SerpMax noindex checker tests any URL and shows:
— whether the noindex tag is present;
— the exact meta tag or HTTP header;
— whether the page is blocked by robots.txt;
— recommendations.
Enter the URL, click Check, and review the result. The tool works with any website, not just your own.
Method 2. View page source
Open the page in a browser. Right-click and select "View page source". Search for noindex in the HTML.
Look for:
<meta name="robots" content="noindex">
or
<meta name="googlebot" content="noindex">
If you see either, the page is blocked from indexing.
Method 3. Google Search Console URL Inspection
Open Search Console → URL Inspection. Paste the URL. Check the status. If the page is blocked by noindex, Search Console will show "Excluded by noindex tag".
This is the most reliable method because it shows what Google actually sees.
How to check if a page is blocked by robots.txt
Robots.txt blocks crawling, not indexing. To check if a page is blocked:
1. Open your robots.txt file (usually at yourdomain.com/robots.txt).
2. Look for Disallow rules that match the page URL.
3. Or use the robots.txt checker to test a specific URL.
If the page is blocked by robots.txt, Google will not crawl it. But if the page is already indexed and you add it to robots.txt, it may stay in the index.
Why noindex is often left by mistake
Noindex is commonly left on pages by accident. Common scenarios:
1. Staging site. Developers add noindex to the staging site to prevent indexing. When the site goes live, noindex is not removed.
2. CMS template. A noindex tag is added to a template (e.g., for tag pages or pagination) and accidentally applies to all pages.
3. SEO plugin settings. An SEO plugin is configured to noindex certain page types, but the setting is too broad.
4. Migration. During migration, the noindex tag from the old site is copied to the new site.
5. Testing. A developer adds noindex for testing and forgets to remove it.
How to remove noindex
If a page has noindex and you want it indexed:
Step 1. Find where the noindex tag is added — in the page HTML, in the CMS template, or in the SEO plugin settings.
Step 2. Remove the noindex tag or change it to index, follow.
Step 3. If the page was previously indexed and then blocked, request reindexing via Google Search Console URL Inspection → Request Indexing.
Step 4. Wait 3-14 days for Google to recrawl the page and add it to the index.
Noindex for images: noimageindex
There is a separate directive for images: noimageindex. It tells search engines not to index images on the page.
<meta name="robots" content="noimageindex">
Use it if you want the page indexed but not its images. Check via the noindex checker — it shows both noindex and noimageindex.
How to check noindex across your site
The noindex checker works on one page at a time. For a full site check, use the SEO Audit by SerpMax. The tool crawls all pages and shows:
— pages with noindex;
— pages blocked by robots.txt;
— pages with canonical issues;
— recommendations.
This is faster than checking each page manually.
When to use noindex
Noindex is not always a problem. Use it intentionally for:
— thank you pages;
— internal search results;
— pagination pages (if you do not want them indexed);
— tag pages with thin content;
— staging sites;
— private pages.
The problem is not noindex itself — it is noindex on pages that should be indexed.
Frequently Asked Questions
How do I check if a page is noindexed?
Use the noindex checker, view page source and search for noindex, or check Google Search Console URL Inspection.
What is the difference between noindex and robots.txt?
Noindex removes a page from the index. Robots.txt prevents crawling but not indexing. If a page is already indexed and you add it to robots.txt, it may stay in the index.
Can I remove a page from Google with noindex?
Yes. Add noindex, and after Google recrawls the page, it will be removed from the index. This usually takes 1-4 weeks.
What is noimageindex?
Noimageindex is a directive that tells search engines not to index images on the page. The page itself can still be indexed.
Why is my page noindexed by mistake?
Common causes: leftover noindex from staging, noindex in CMS template, SEO plugin settings, migration, testing. Check the page HTML, template, and plugin settings.
How do I remove noindex?
Find where the noindex tag is added — in the page HTML, CMS template, or SEO plugin. Remove it or change to index, follow. Then request reindexing in Search Console.
Does noindex affect rankings?
Noindex removes the page from search results entirely. It cannot rank because it is not in the index. Use noindex only for pages that should not appear in search.
