[Tom]

You may have noticed we had quite a few "guests" viewing the forum today. As a matter of fact, we set a new record for most users ever online. In case anyone was wondering, we added some code the other day to allow the two major search engines to crawl the forum, and it just so happens Mr. Google was paying us a visit.

[MCaver]

Very cool. More the merrier.

[Tom]

Just to clarify, google bots were crawling the forums, not necessarily an actual guest.

[catwoman]

What do you mean by that? What are google bots? And what do you mean by they were crawling the forum?

[Tom]

http://www.usatoday.com/tech/2002/05/13/search-engines.htm

[catwoman]

I still don't get it. (I'm webmastering, too, and I'm just trying to understand all this stuff.)

[Odonata]

Try googling "how a web crawler works"

[MCaver]

Here's the deal, Catwoman. When you notify a full text search engine like Google that you want your site indexed, it starts up "robots" -- automated web users -- that browse through the website and record the information it needs to index the pages (for places like Google it's the full page). But the robots don't hit just one page. They also follow the links on that page, and the links on those pages, eventually indexing the entire site that way. Larger search engines like Google usually have several robots working on a site at once, and since each one gets counted as a Guest user here, it caused our numbers for the day to spike. They weren't real users, but simulated users recording data for the search engines. What this means for NWHikers is that each topic will now come up in Google searches. If someone is doing a search on Pete Lake, for example, the recent thread extolling the virtues of the Pete Lake Trail will possibly come up in the search results, which has the potential to bring more people to the site. There are ways to block robots, assuming they obey the standard protocols, and I assime what Tom did is remove the block prohibiting search engines from crawling the site. If you want to know more, just send me a message. I'd be happy to tell you what i know. I don't know how many others here really care about this.

[cory]

can sumone tell me how a web crawler works

[Tom]

You can always STFW, no pun intended.

[clatz]

Well, I search google for "picture attachment phpbb" and ended up here because this site get's the lucky 3rd and 4th link on that search. I'm setting up a phpbb forum for something completely unrelated to hiking and wanted to see an example of the phpBB picture mod in action. Just for giggles I tried "northwest hiking forum" and you're first, 6th for "oregon hiking forum," and 9th for "washington hiking forum." I ended up browsing around here for like 10 minutes and I suddenly have the desire to go camping or backpacking. It also reminded me that this time last year I had just been to Vancouver, WA on a business trip. Have fun out there... Craig Latzke Fort Collins, CO

[soggy]

Business before pleasure, and hiking before business.