Search results
Search Console. Glossary. Crawling. Crawling. Crawling is the process of finding new or updated pages to add to Google (Google crawled my website). One of the Google crawling engines crawls (requests) the page. The terms "crawl" and "index" are often used interchangeably, although they are different (but closely related) actions.
Crawl. Crawling is the process of finding new or updated pages to add to Google (Google crawled my website). One of the Google crawling engines crawls (requests) the page. The terms "crawl" and "index" are often used interchangeably, although they are different (but closely related) actions. Learn more.
Crawl Stats report. The Crawl Stats report shows you statistics about Google's crawling history on your website. For instance, how many requests were made and when, what your server response was, and any availability issues encountered. You can use this report to detect whether Google encounters serving problems when crawling your site.
Type the fully-qualified URL to inspect in the inspection search bar at the top of any Search Console screen. The URL must be in the currently opened property. Click an Inspect link next to a page URL in most reports. Sometimes you need to hover over the URL to see this option. Open the URL Inspection Tool.
To control Googlebot's crawling of your content, use a robots.txt file. If you're worried about rogue bots using the Googlebot user-agent, you can verify whether a crawler is actually Googlebot. If you would like to change how frequently Googlebot crawls your site, you can request a change in Googlebot's crawl rate. Hosting providers can verify ...
Crawling. Beim Crawling werden neue oder aktualisierte Seiten für Google gefunden (Google hat meine Website gecrawlt). Ein Google-Crawler crawlt die Seite – oder mit anderen Worten: Er fordert sie an. Die Begriffe "crawlen" und "indexieren" werden oft gleichbedeutend verwendet, obwohl es sich um unterschiedliche (wenn auch eng miteinander ...
Open the URL Inspection tool. Enter the URL of the page or image to test. To see whether Google could access the page the last time it was crawled, expand the "Coverage" section and examine the results as described below. To see whether Google can access the resource now, click Test live URL, expand the "Coverage" section, and examine the ...
Check that robots.txt allows crawling. The ads.txt/app-ads.txt for a domain may be ignored by crawlers if the robots.txt file on a domain disallows one of the following: The crawling of the URL path on which an ads.txt/app-ads.txt file is posted. The User Agent of the crawler. Example: Crawling disallowed on ads.txt file path
This help content & information General Help Center experience. Search. Clear search
Read the basics of crawling and indexing. Crawling issues. Google needs to be able to find your pages in order to index them. Here are the main ways to help Google find your pages: Submit a sitemap. This tells Google exactly which pages you want crawled. Many website hosting services create and submit a sitemap for you, so you don't need to ...
