10 Ways to Increase Your Site Crawl Rate
10 Ways to Increase Your Site Crawl Rate
- Update your content often and regularly
- Make sure your server works correctly: mind the uptime and Google Webmaster tools reports of the unreached pages. Two tools I can recommend here are Pingdom and Mon.itor.us.
- Mind your page load time: note that the crawl works on a budget – if it spends too much time crawling your huge images or PDFs, there will be no time left to visit your other pages.
- Check the site internal link structure: make sure there is no duplicate content returned via different URLs:
again, the more time the crawler spends figuring your duplicate
content, the fewer useful and unique pages it will manage to visit.
- Get more back links from regularly crawled sites.
- Adjust the crawl speed via Google Webmaster tools.
- Add a sitemap (though it’s up for a debate whether the
sitemap can help with crawling and indexing issues, many webmasters
report they have seen increased crawl rate after adding it).
- Make sure your server returns the correct header response.
- Make sure you have unique title and meta tags for each of your pages.
- Monitor Google crawl rate for your site and see what works and what not
No comments:
Post a Comment