Faster Converter to phpbb3

Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here.
Forum rules
Discuss features as they are added to the new version. Give us your feedback. Don't post bug reports, feature requests, support questions or suggestions here. Feature requests are closed.
guischarf
Registered User
Posts: 3
Joined: Wed Apr 23, 2008 9:16 pm

Re: Faster Converter to phpbb3

Post by guischarf »

Hello,
sdawson wrote:We have a board with 400,000 posts and 80,000 users and it took around 2 hours to convert. We run 2 dedicated servers, 1 for database, 1 for httpd.
I would really like to know your trick. I have a similar setting, both servers have double dual core processors and 6G RAM. I have a 1.500.000 posts /65.000 users DB. The conversion has been running all night long and I had to stop it in the morning. This has been performed several times with the same results: aborted conversion. Besides, every time there is an error, such as a lost connection, all work is lost.

Also tried the offline trick. The conversion has been running in my local server for over 5 hours and is has converted only 500.000 rows (1/3 of the total). It still has a long way to go.

Tried tweaking the batch limit. Have tried several values, 1000, 2000, 3000, 6000, with no noticeable improvement. Locally, 6000 lines seems to improve speed a little, on the remote server, it crashes it. I had to set it down to 1000. The other setting, num_wait_rows, don't really understand what is for. Extended Insert ? :shock:
guischarf
Registered User
Posts: 3
Joined: Wed Apr 23, 2008 9:16 pm

Re: Faster Converter to phpbb3

Post by guischarf »

I think that for large boards it would be a better option to have a command line conversion tool.
Glauco
Registered User
Posts: 2
Joined: Sun Apr 20, 2008 12:03 am

Re: Faster Converter to phpbb3

Post by Glauco »

guischarf wrote:Hello,
sdawson wrote:We have a board with 400,000 posts and 80,000 users and it took around 2 hours to convert. We run 2 dedicated servers, 1 for database, 1 for httpd.
I would really like to know your trick. I have a similar setting, both servers have double dual core processors and 6G RAM. I have a 1.500.000 posts /65.000 users DB. The conversion has been running all night long and I had to stop it in the morning. This has been performed several times with the same results: aborted conversion. Besides, every time there is an error, such as a lost connection, all work is lost.

Also tried the offline trick. The conversion has been running in my local server for over 5 hours and is has converted only 500.000 rows (1/3 of the total). It still has a long way to go.

Tried tweaking the batch limit. Have tried several values, 1000, 2000, 3000, 6000, with no noticeable improvement. Locally, 6000 lines seems to improve speed a little, on the remote server, it crashes it. I had to set it down to 1000. The other setting, num_wait_rows, don't really understand what is for. Extended Insert ? :shock:
Are you building an index for fulltext native search method? Fulltext mysql is way faster to build and WORKS. Fulltext native never worked for me in phpbb2.
The conversion, including search index building, took less than 6 hours in a board with 1 million posts and 7000 users.
alex.nenov
Registered User
Posts: 1
Joined: Tue May 13, 2008 1:39 pm

Re: Faster Converter to phpbb3

Post by alex.nenov »

I read all about conversion from phpbb2 to phpbb3 but i can't understand why we have to do many trick to convert our boards? It's fact that this converter is perfect for small sites but site with more that 2 000 000 posting it just not working. I can't stop my forum for the same reasons like many of this theme - advertisers and users. Bu i HAVE to convert the board to new version. So I try to convert it 3 times on my localhost ( Win XP + AppServ ) and every time somewhere about 200 000 topics MySQL just crash. I change the option of php, apache and MySQL so now the proces is faster but when i calculate the time with number of posting it seems to convert the board about 44 hours ?!?! Last 3 times for 200 000 posting converted i wait about 7 hours.

The best thing will be to start conversion from console but is that possible?
guischarf
Registered User
Posts: 3
Joined: Wed Apr 23, 2008 9:16 pm

Re: Faster Converter to phpbb3

Post by guischarf »

Are you building an index for fulltext native search method? Fulltext mysql is way faster to build and WORKS. Fulltext native never worked for me in phpbb2.
The conversion, including search index building, took less than 6 hours in a board with 1 million posts and 7000 users.
Building the index is not a problem since it is performed after the migration has been completed. The problem is the migration itself. The only reasonable (if it could be called 'reasonable') way I found to do it was to download the database to my local PC and perform the migration locally. A shell scritp would be a more reasonable solution for a large forum since it could be set to run as a background process with no interaction with whomever is performing the migration.

We have close to one and a half milion posts and 70.000 users.
User avatar
Acyd Burn
Posts: 1838
Joined: Tue Oct 08, 2002 5:18 pm
Location: Behind You
Contact:

Re: Faster Converter to phpbb3

Post by Acyd Burn »

Sometimes it helps to remove the indexes on the posts table before migration and then adding it back after. This elivates the need for mysql to re-calculate the index on every INSERT. Also, tweaking the variables for the batches for large boards directly within the convertor php file may help too.

Image
Post Reply