Ensuring the information a search engine has on your site is of critical importance. We need to ensure that new content is added into the search engines as quickly as possible, web crawlers visit your site at various intervals, some search engines will allow you to set the rate at which they visit your site, google and ask.com are the two that come to mind, but this still doesnt ensure your new page is added into the index as soon as possible.
The best way I have found is to alert the search engine to an update, you can do this by sending a ping request to the search engine via an api. This api alerts the index that new content is available and to instruct the crawler to visit the site, below are the uri’s of the top search engines sitemap ping api’s
- Google – http://www.google.com/webmasters/tools/ping?sitemap=
- Yahoo – http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=
- Live.com – http://webmaster.live.com/webmaster/ping.aspx?siteMap=
- Ask.com – http://submissions.ask.com/ping?sitemap=
Just append the url with your sitemap location and off you go. I have modified my sites cms to send a request via cURL to the api’s when ever a new page is added or deleted from my site. This ensures search engines are alerted to any changes in my sitemap.