[hunchentoot-devel] Google search result

[Edi Weitz]

On Fri, 14 Dec 2007 17:18:38 -0500, Gary King <gwking at metabang.com> wrote:

> IIRC, there is some way to ping Google and get them to spider your
> page more quickly...

The text is /not/ a result of spidering - it never appeared on my
webpage in this form.  Google spiders weitz.de pretty often anyway.
For example, Googlebot looks at the Hunchentoot page about once per
day:

  edi at nanook:/var/log/apache$ nslookup 66.249.67.206
  Server:         81.169.163.104
  Address:        81.169.163.104#53

  Non-authoritative answer:
  206.67.249.66.in-addr.arpa      name = crawl-66-249-67-206.googlebot.com.

  Authoritative answers can be found from:

  edi at nanook:/var/log/apache$ cat weitz_access.log.1 weitz_access.log | grep -i 66.249.67.206 | grep "GET /hunchentoot/ "
  66.249.67.206 - - [02/Dec/2007:12:10:00 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [03/Dec/2007:12:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [03/Dec/2007:16:40:12 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [04/Dec/2007:18:51:13 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [05/Dec/2007:16:16:03 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [05/Dec/2007:22:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [05/Dec/2007:22:30:16 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [07/Dec/2007:05:49:10 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [07/Dec/2007:08:09:28 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [08/Dec/2007:08:41:29 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [09/Dec/2007:17:13:07 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [11/Dec/2007:14:32:29 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [12/Dec/2007:15:36:36 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [13/Dec/2007:16:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [14/Dec/2007:15:32:27 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"
  66.249.67.206 - - [14/Dec/2007:19:47:44 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"