phpBB2 -> phpBB3 Upgrade Script Discussion

Discussion of general topics related to the new version and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Forum rules
Discussion of general topics related to the new release and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Post Reply
User avatar
Nicholas the Italian
Registered User
Posts: 659
Joined: Mon Nov 20, 2006 11:19 pm
Location: 46°8' N, 12°13' E
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by Nicholas the Italian »

kjtocool wrote: I'm worried about my board as it is approaching 900,000 posts. Hopefully converting will go smoothly, I plan to try when RC 1 is released.

Try it off-line, if you can.
User avatar
SHS`
Registered User
Posts: 1628
Joined: Wed Jul 04, 2001 9:13 am
Location: The Boonies, Hong Kong
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by SHS` »

Nicholas the Italian wrote:
kjtocool wrote: I'm worried about my board as it is approaching 900,000 posts. Hopefully converting will go smoothly, I plan to try when RC 1 is released.

Try it off-line, if you can.


No advice is better than bad advice. The convertor, as with any bit of software which by its nature is demanding on DB performance, will run best of performant DB hardware, which in the case of large forums... will be on the existing hardware they are hosted on as opposed to "on some desktop machine", which by default will be abysmally non-performant as a DB server.
Jonathan “SHS`” Stanley • 史德信
phpBB™ 3.1.x, Bug/Security trackers
phpBB™ Bertie Bear 3.0 — prosilver Edition!Asking Questions The Smart Way
User avatar
Handyman
Registered User
Posts: 522
Joined: Thu Feb 03, 2005 5:09 am
Location: Where no man has gone before!
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by Handyman »

SHS` wrote:
Nicholas the Italian wrote:
kjtocool wrote: I'm worried about my board as it is approaching 900,000 posts. Hopefully converting will go smoothly, I plan to try when RC 1 is released.

Try it off-line, if you can.


No advice is better than bad advice. The convertor, as with any bit of software which by its nature is demanding on DB performance, will run best of performant DB hardware, which in the case of large forums... will be on the existing hardware they are hosted on as opposed to "on some desktop machine", which by default will be abysmally non-performant as a DB server.

For a test run with a forum of that magnitude, don't test it on the server you are running your forum on.
For me, when I did that with my large forum, you couldn't access the forum while the conversion was going on… not a problem with the converter, it just used up a lot of resources… and only for a test run.
I'm on a more powerful server now, but why use up your server resources until you are ready and you have worked out all the bugs on your localhost first… if there are bugs converting on your localhost… they will be there when you convert it on your server as well.
I don't recommend doing it on the same server until you are ready to convert and you have made sure it works on your localhost first.

Also, it doesn't make much sense to test it with the converter packaged in Beta5 as there have been many upgrades to it since then.
So if you are just wanting to give it a test run, you would probably want to download the latest CVS.
My phpBB3 Mods || My Mod Queue
Search Engine Friendly (SEO) URLs || Profile link on Avatar and/or Username || AJAX Chat
Display Posts Anywhere || CashMod || AJAX Quick Edit || AJAX Quick Reply

Image
sscape
Registered User
Posts: 3
Joined: Sat Jun 04, 2005 8:58 pm

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by sscape »

I'm currently testing offline with my forum of 1.5 million posts. (this was from a CVS version from a couple of days ago)

It's taking quite a while and whilst the rows per second started off at around 200 - it's now on about 1.3 million posts and the rows per second have been slowly dropping - so now they're at 9 rows per second.

e.g.: skip_rows = 1346195: 9 rows/s (2000 rows) |

Is this expected behaviour for the convertor?
CPU memory/usage have remained fairly low/constant for both mysql and apache whilst this has been converting.

Are there any queries that get longer (maybe unnecessarily?) as the number of posts increases in the phpbb3 database?
User avatar
Nicholas the Italian
Registered User
Posts: 659
Joined: Mon Nov 20, 2006 11:19 pm
Location: 46°8' N, 12°13' E
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by Nicholas the Italian »

Handyman wrote:
SHS` wrote:
Nicholas the Italian wrote: Try it off-line, if you can.

No advice is better than bad advice.

I'm on a more powerful server now, but why use up your server resources until you are ready and you have worked out all the bugs on your localhost first… if there are bugs converting on your localhost… they will be there when you convert it on your server as well.
I don't recommend doing it on the same server until you are ready to convert and you have made sure it works on your localhost first.

Yep, that's what I meant ;) It's more a debug/check-everything-works issue than a performance issue (also because you have to download, convert, and upload, while online you only have to convert). Also consider that the board should be disabled while doing all this (even if just testing), as otherwise you may get only-god-knows-what.
I wouldn't start a 6hr script that I can't control and have no idea what it is doing becuse it runs who-knows-where. Also, in such a long time on such a far machine anything may happen, while my PC will be 100% dedicated to that task, even if it takes 2 days.
We're beta-testing after all, so I'd first try off-line, and, when no errors occour, make another test on-line.

Of course each one will do what he thinks is best for him. :)
Are there any queries that get longer (maybe unnecessarily?) as the number of posts increases in the phpbb3 database?

Maybe something to do with having to do integrity checks in some 1.5 mln records, and/or having to rebuild indexes each time?
(BTW, are they single queries, or transactions?)
User avatar
Highway of Life
Registered User
Posts: 1399
Joined: Tue Feb 08, 2005 10:18 pm
Location: I'd love to change the World, but they won't give me the Source Code
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by Highway of Life »

SHS` wrote:
Nicholas the Italian wrote:Try it off-line, if you can.
No advice is better than bad advice.
On the contrary, mein freund.
It’s much better to "test" the script on a localhost rather than tie up a server with the load process of running the converter.
An exception might be if you have a "test server", or a server that is empty, or does not get much traffic.
Testing it on your localhost gives you much more flexibility when dealing with the converter, especially if you have 900,000+ posts you need to convert.
I always test on localhost before I try it live, it eliminates several steps with uploading and downloading, this is the best idea, and the best advise. For much the same reason you "don't run a Beta board live" would you not run an intensive conversion script on your remote server unless or until you know it will work for sure.
Image
User avatar
D@ve
Registered User
Posts: 22
Joined: Thu Sep 02, 2004 1:38 am

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by D@ve »

I still got a 500 Server Error (Premature end of script headers) during convert procedutre. Am I the only one with that kind of problem?

Regards, Dave
http://www.2sounde.de" target="_blank - das kostenlose online-magazin für musiker
PaulW
Registered User
Posts: 21
Joined: Sun Aug 31, 2003 1:38 am

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by PaulW »

Not using the updated CVS to convert, just used Beta 5 jobby...

Script kept timing out, even though I increased the timeout value on the apache configs to like 3 years... just kept getting a blank screen during the conversion process after 4-5 mins of action, so just waited, then went back and chose continue... took a few attempts but worked fine!

Would it not be possible though for the script to convert x amount of posts per refresh of the update page?? instead of waiting for the entire script to process each step before continuing?

Just on large boards, it might help stop sysadmins and hosts moaning about server load, as you could do a meta-refresh with defineable update period to help reduce server load (ok so it WILL take longer, but its a sacrifice I would be willing to take...)
matt1206
Registered User
Posts: 4
Joined: Thu Mar 09, 2006 2:56 am
Location: Sheffield
Contact:

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by matt1206 »

I seem to be having some problems testing the update script on the latest CVS....

Code: Select all

phpBB3-CVS_2007-04-14.tar.gz
So, I've got the vanilla phpBB3 working fine, so when I try converting the old install, I keep getting the following error:

Could not find path to your former forum. Please check your settings and try again.
» Specified source path was ../forum

Now, I've got the both the new phpBB3 folder and the original forums folder in the same directory:

Code: Select all

[root@www httpdocs]# ls -al
drwxr-x---  15 matt1206a psaserv    4096 Apr 15 10:44 .
drwxr-xr-x  14 root      root       4096 Apr 15 09:21 ..
drwxr-xr-x  17 root      root       4096 Apr 15 10:12 forums
drwxr-xr-x  14 root      root       4096 Apr 15 10:44 phpBB3
I've tried loads of variations for the path of the old forum, but nothing works. Could someone shed any light on this?

Cheers,

Matt
code reader
Registered User
Posts: 653
Joined: Wed Sep 21, 2005 3:01 pm

Re: phpBB2 -> phpBB3 Upgrade Script Discussion

Post by code reader »

one problem might be the permissions on your httpdocs directory itself.
you may want to try and run (as root) from the httpdocs

Code: Select all

chmod a+r .
also, i would recommend to use your "root" user as little as possible :lol:
Post Reply