Will 2.2 be webspider-friendly?

Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here.
Forum rules
Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here. Feature requests are closed.
njtom
Registered User
Posts: 1
Joined: Sun Jun 29, 2003 3:43 pm

Will 2.2 be webspider-friendly?

Post by njtom »

Will phpBB 2.2 be more compatible w/ google, etc? I know that the current version needs a small mod to allow them..?
stiffler
Registered User
Posts: 6
Joined: Fri Jul 04, 2003 7:52 pm

Re: Will 2.2 be webspider-friendly?

Post by stiffler »

I also support a more webspider-friendly version of phpbb.
Cheers,
Erick

[:: vTeen.com :: ]
Wert
Registered User
Posts: 400
Joined: Tue Jul 03, 2001 8:33 pm

Re: Will 2.2 be webspider-friendly?

Post by Wert »

I'm hoping it won't be. These bots can really hit your forums hard and use up a lot of systems reasources. (not good if you're on shared hosting)

But if it is, no big deal as I'll use "Robots.txt" to tell search bots not to crawl my forums.
Need good web hosting? I recommend Hostrocket.
User avatar
Techie-Micheal
Registered User
Posts: 566
Joined: Sun Oct 14, 2001 12:11 am

Re: Will 2.2 be webspider-friendly?

Post by Techie-Micheal »

Unless I'm missing something, I haven't had any trouble with bots spidering my board and without any modifications to my board ...
User avatar
Lugie
Registered User
Posts: 163
Joined: Sat Nov 16, 2002 9:56 pm
Location: Wisconsin
Contact:

Re: Will 2.2 be webspider-friendly?

Post by Lugie »

njtom wants spiders to crawl his board. So you two have different problems.

Right now spiders don't crawl phpBB for a reason I am uneducated on. :)
I can't see the meaning of this life I'm leading,
I try forget you as you forgot me...
This time there is nothing left for you to take -
this is goodbye...
User avatar
Acyd Burn
Posts: 1838
Joined: Tue Oct 08, 2002 5:18 pm
Location: Behind You
Contact:

Re: Will 2.2 be webspider-friendly?

Post by Acyd Burn »

My personal board was indexed quite havily by google, that havily i had to block the googlebot IP. ;)

Submitting a bug report to them helped a little bit but not ironed out the main problems. The googlebot produced very high load on the server.
User avatar
Techie-Micheal
Registered User
Posts: 566
Joined: Sun Oct 14, 2001 12:11 am

Re: Will 2.2 be webspider-friendly?

Post by Techie-Micheal »

Lugie wrote:njtom wants spiders to crawl his board. So you two have different problems.

Right now spiders don't crawl phpBB for a reason I am uneducated on. :)
I know. And I am saying my board is indexed without any MODs.
User avatar
psoTFX
Registered User
Posts: 1984
Joined: Tue Jul 03, 2001 8:50 pm
Contact:

Re: Will 2.2 be webspider-friendly?

Post by psoTFX »

Yes, you're quite correct ... it will be. However, in doing so it will generate dozens of new sessions with the same (or similar) IP. The request, and something that is on the "ideas" list for M-3 is to introduce better support for spidering. In this case a method (yet to be determined) would be used to allow Google et al to spider a site without regenerating dozens of IPs, and to (potentially) limit what it can and cannot view (i.e. some kind of dedicated user/group).
Wert
Registered User
Posts: 400
Joined: Tue Jul 03, 2001 8:33 pm

Re: Will 2.2 be webspider-friendly?

Post by Wert »

Acyd Burn wrote:My personal board was indexed quite havily by google, that havily i had to block the googlebot IP. ;)

Submitting a bug report to them helped a little bit but not ironed out the main problems. The googlebot produced very high load on the server.
Same here. Before I implemented 'Robots.txt' to limit google, it wasn't uncommon for it to open 20-30 processes when spidering. Not optimal on a shared server.
avis
Registered User
Posts: 5
Joined: Wed Jul 16, 2003 11:24 am

Re: Will 2.2 be webspider-friendly?

Post by avis »

Hi Acyd!

Whats the address of your website which was heavily indexed by google ? Coz i want to check the meta tags etc. so i can get my forums in googleand other search engines too!

Thanks!
Post Reply