How to Find the Pages AI Cannot See on Your Website
Short answer: open Google Search Console, go to the Pages report, and look at the number under Not indexed. Most business owners have never opened that screen. The pages listed there load perfectly well in a browser, which is exactly why nobody notices they are effectively invisible.
A page that is not indexed cannot be found, cited or recommended. Here is how to find yours and what to do about the reasons you will see.
What Search Console and sitemaps actually are
Google Search Console is a free tool that shows you what Google sees when it looks at your website. Not what you see. What it sees.
A sitemap is a file listing every page you want found. It usually lives at yourdomain.co.nz/sitemap.xml.
Together they answer a question you cannot answer any other way: is my content actually in the index, or does it only exist for me?
Why this matters for AI visibility
Almost every AI tool that answers questions about real businesses is either reading search results directly or working from an index built the same way.
If your service page is not indexed, it is not in the pool of things that can be retrieved. All the entity work and schema markup in the world will not help a page that was never picked up.
This is also your only honest feedback loop. Everything else in AI visibility is you telling machines about your business. This is the machine telling you what it made of it.
How to check
Go to search.google.com/search-console.
If you have never set it up, that is your answer. Skip to the setup steps below.
If you have, open Indexing, then Pages.
Look at the number under Not indexed, then read the reasons listed beneath it.
Most owners look at this for the first time and find a number far larger than they expected. That is normal. A good chunk of it will be harmless.
Setting it up if you have not
Verify your site. The easiest routes are a DNS record through your domain provider, or the built in integration most website platforms offer. Shopify, Squarespace, Wix and most WordPress SEO plugins all have a verification field you paste a code into.
Find your sitemap. Shopify generates one automatically. WordPress generates one if you run Yoast or Rank Math. Try yourdomain.co.nz/sitemap.xml and yourdomain.co.nz/sitemap_index.xml.
Submit it under Sitemaps in the left menu. This takes about thirty seconds.
Wait 48 hours, then read the Pages report properly.
The reasons you will see, in plain English
Crawled, currently not indexed. Google looked at the page and decided not to keep it. This usually means the page is thin, duplicated in substance, or has nothing on it that another page does not already say. Fix by adding genuinely useful content, not more words.
Discovered, currently not indexed. Google knows the page exists but has not bothered to look yet. Often a crawl budget or site quality signal. Improve internal linking to the page and be patient.
Duplicate without user-selected canonical. Two or more addresses show the same content and you have not told Google which one is the real one. Common with product variants, tag pages and http versus https leftovers. Fix with a canonical tag pointing at the version you want.
Duplicate, Google chose different canonical than user. You picked one, Google disagreed. Usually means the page you nominated is weaker than the alternative. Worth investigating rather than forcing.
Alternate page with proper canonical tag. Normal. Working as intended. Ignore it.
Excluded by noindex tag. You or a plugin told Google not to index it. Sometimes deliberate, like a thank you page. Sometimes a staging setting that never got switched off after launch, which is a genuine emergency worth checking first.
Blocked by robots.txt. Something is stopping the crawl entirely. Worth fixing immediately, and worth checking whether AI crawlers are blocked by the same rules.
Not found (404). The page is gone. Either restore it or redirect it if anything links to it.
Page with redirect. Normal. This is the report telling you a redirect worked.
Soft 404. The page loads but looks empty or error like to Google. Common on empty category pages and out of stock products.
A prompt to turn the report into a to do list
In Search Console, open Pages, hit Export at the top right and choose copy to clipboard. Then:
Below is the Pages report from my Google Search Console showing why pages are not indexed. For every reason listed, tell me what it means in one plain sentence, whether it is a real problem or completely normal, and the single first thing I should do about it. Then rank the whole list in the order I should work through it, putting anything costing me visible pages first. Be blunt about which ones I can safely ignore forever. My business is [what you do and where]. Here is the report: [paste]
The blunt instruction matters. Without it you get a comprehensive list you will never finish instead of a short one you will.
What to actually fix, and what to leave
Fix, in this order:
Anything with noindex on it that should be public. This is the one that is genuinely costing you.
Blocked by robots.txt on pages you want found.
Duplicate without user-selected canonical, if it affects service or product pages.
Crawled currently not indexed, on pages that matter commercially. Thin service pages are the usual culprit.
Leave alone: alternate pages with proper canonicals, redirect pages, and 404s for things that genuinely no longer exist and nothing links to.
Common mistakes
Trying to get everything indexed. You should not. Tag archives, filtered views and thin pages are correctly excluded. Chasing zero is a waste of a weekend.
Requesting indexing repeatedly. The Request Indexing button does not work harder if you press it more. Fix the underlying reason instead.
Submitting a sitemap and considering the job done. Submission is the start. The report is the point.
Never going back. Check quarterly. Put it in your calendar. That is the entire ongoing commitment.
Frequently asked questions
How long does indexing take after I submit a sitemap? Anywhere from a day to several weeks depending on your site's size and how often Google crawls it. New small sites are usually slowest.
Do I need Bing Webmaster Tools as well? It helps, and it takes ten minutes since it can import directly from Search Console. Bing feeds Copilot and parts of ChatGPT search, so it is more relevant now than it has been in years.
My site has 12 pages but Search Console shows 400. Why? Almost always parameters, filters, tags or pagination generating extra addresses. Usually harmless. Worth a look if the numbers are extreme.
Does Search Console show me anything about AI tools? Not directly. There is no AI visibility report. What it shows you is whether your content exists in the index those tools draw on, which is the closest available proxy.
Is a sitemap still necessary if my site is small? It is not strictly required, but it costs nothing and removes any ambiguity about which pages you want found. Submit it.
This is check four of seven
Search Console tells you whether the work is landing. It sits in the middle of the list for a reason: fix your crawler access and your facts first, then confirm the machines are picking them up.
Get Named By AI is the full 7-Point AI Visibility Check. Seven checks in the order they have to be done, a scorecard so you know where you stand, and nine copy and paste prompts that do the work for you. 21 pages, $17, and about an afternoon of your time.
