How to Fix Crawl Errors in Google Search Console (2025 Guide)

If Google cannot crawl a page, it cannot index it. If it cannot index it, you will never rank for it no matter how good the content is.

Crawl errors are one of the most common technical SEO problems, and Google Search Console shows you exactly which pages are failing. The problem is that the interface changed significantly in 2022, and most guides still show the old version.

This guide walks you through the current Search Console interface, explains every error type, and gives you the exact steps to fix each one.

What are crawl errors?

A crawl error happens when Googlebot tries to visit a URL on your site and something goes wrong. The page does not exist, the server does not respond, or the page is blocked from being crawled.

There are two main categories:

• Site errors: problems affecting your whole domain, such as DNS failures or server outages
• URL errors: problems with specific pages, such as 404s, redirect loops, or pages blocked by robots.txt

Google Search Console reports both, but URL-level errors are where most sites have the most work to do.

Where to find crawl errors in the current Search Console

The old standalone “Crawl Errors” report no longer exists. In the current interface, crawl data is spread across three places.

1. Pages report (start here)

Go to Search Console, then Indexing, then Pages.

This is the most important report. It shows every URL Google has discovered and explains why pages are or are not indexed. Each reason listed in the “Why pages aren’t indexed” section is effectively a crawl or indexing issue.

Click any reason to see the full list of affected URLs.

2. URL Inspection tool

Go to Search Console, then URL Inspection, and paste any specific URL.

This shows you exactly what happened the last time Google tried to crawl that page: the crawl date, the HTTP status code returned, and any specific issues found.

Use this to investigate individual problem URLs you find in the Pages report.

3. Settings, then Crawl stats

Go to Search Console, then Settings, then Crawl stats.

This shows the volume and frequency of Googlebot’s crawls over the past 90 days. Useful for spotting sudden drops that indicate a server or robots.txt problem.

The most common crawl errors and how to fix them

404: Page not found

What it means: Google found a link pointing to a URL that returns a 404 status. The page either never existed or was deleted.

Why it matters: 404s on pages that used to have backlinks or organic traffic waste your link equity. Google will eventually drop these from its index, but until it does, they show up as wasted crawl budget.

How to fix it:

1. In the Pages report, click “Not found (404)” to see the affected list
2. For each URL, ask: does this page exist somewhere under a new URL?
   • If yes: set up a 301 redirect from the old URL to the new one
   • If no and the page never had traffic or links: leave it as a 404
   • If no but the page had meaningful traffic or backlinks: either restore the content or 301 redirect to the closest relevant page
3. If the 404 is caused by a broken link in your own navigation or internal links: fix the link in your CMS

Not all 404s need fixing. If a page was intentionally removed and had no value, a clean 404 is the correct response. Only prioritise 404s that previously had organic traffic or inbound links.

Redirect error

What it means: Google followed a redirect chain and either hit a loop, too many redirects (5 or more), or an empty redirect destination.

How to fix it:

1. Use a tool like Screaming Frog or Ahrefs Site Audit to crawl your redirect chains
2. Find any chains longer than 2 hops and collapse them. Redirect A directly to C, skipping B
3. Check for redirect loops such as A to B to A and break them by pointing to the correct final URL
4. Check that all redirect destinations return a 200 status code

Server error (5xx)

What it means: Your server received Google’s request but could not complete it. Common causes include the server being overloaded, the page taking too long to respond, or a database connection failing.

How to fix it:

1. Check your hosting provider’s error logs for the time period Search Console flagged
2. If errors were temporary due to a traffic spike or a deployment, they should resolve. Request reindexing of affected URLs via URL Inspection
3. If errors are recurring, investigate server capacity, slow database queries, or PHP or Node timeout settings
4. If needed, reduce Googlebot’s crawl rate in Search Console under Settings, then Crawl rate

Real context: In 2022, John Mueller from Google confirmed on Twitter that persistent 5xx errors on important pages can cause Google to reduce its crawl frequency for your entire site, not just the affected pages. This is worth fixing quickly.

Blocked by robots.txt

What it means: Your robots.txt file contains a Disallow rule that prevents Googlebot from accessing the URL.

How to fix it:

1. Go to Search Console, then Settings, then robots.txt to view and test your current file
2. Use the robots.txt tester to check if a specific URL is blocked
3. If the blocked URL should be indexed: remove or update the Disallow rule
4. After editing, submit your robots.txt URL for re-fetch in URL Inspection

Be careful when editing robots.txt. A single misplaced Disallow: / can block your entire site from Google. For more on the difference between blocking crawling and blocking indexing, read robots.txt vs noindex: what is the difference.

Blocked by noindex

What it means: The page is accessible to Googlebot, but it has a noindex tag in the head or HTTP header telling Google not to include it in search results.

How to fix it:

1. Open the flagged URL and check the page source using Ctrl+U for <meta name="robots" content="noindex">
2. Check HTTP response headers using URL Inspection, then View crawled page
3. If the noindex is intentional such as on thin pages, thank-you pages, or admin areas: leave it
4. If it was added by mistake: remove the tag and request reindexing

Soft 404

What it means: The page returns a 200 HTTP status telling Google it exists, but the actual content signals it is empty or missing. Common examples include a search results page returning zero results or an out-of-stock product page with no content.

How to fix it:

1. Open each flagged URL and check what a real user would see
2. Empty or placeholder pages: return a proper 404 or 410 status
3. Out-of-stock product pages: add meaningful content such as similar products, a notify-me form, or a back-in-stock date
4. Search result pages returning no results: block these from indexing using noindex or robots.txt

Discovered but currently not indexed

What it means: Google knows the URL exists but has not yet crawled it. This is not an error. It is a crawl queue issue, usually caused by low crawl budget or poor internal linking to the page.

How to fix it:

1. Request indexing via URL Inspection for your highest-priority pages
2. Submit an updated XML sitemap including these URLs
3. Add internal links to these pages from higher-authority pages on your site. More internal links signal importance to Googlebot. For more on how crawl budget works, see what is crawl budget in SEO
4. Check crawl stats. If Googlebot visits your site rarely, improve page speed and reduce crawl load

How to prioritise which errors to fix first

Not all crawl errors need immediate attention. Work through them in this order:

1. Server errors (5xx). Fix first. They can affect your whole site’s crawl frequency.
2. Redirect loops. Break them before they waste crawl budget.
3. 404s on pages with backlinks or historical traffic. Add 301 redirects.
4. Incorrectly blocked pages. Noindex or robots.txt errors on pages that should rank.
5. Soft 404s. Especially on ecommerce product pages.
6. Discovered but not indexed. Improve internal linking and submit sitemaps.

How to request reindexing after fixing errors

Once you have fixed an error:

1. Go to URL Inspection and enter the fixed URL
2. Click “Request indexing”
3. For large batches, resubmit your XML sitemap via Indexing, then Sitemaps

Google does not re-crawl immediately. Allow one to two weeks for changes to be reflected in the Pages report.

Frequently asked questions