Faster Converter to phpbb3

[guischarf]

Hello,

We have a board with 400,000 posts and 80,000 users and it took around 2 hours to convert. We run 2 dedicated servers, 1 for database, 1 for httpd.

I would really like to know your trick. I have a similar setting, both servers have double dual core processors and 6G RAM. I have a 1.500.000 posts /65.000 users DB. The conversion has been running all night long and I had to stop it in the morning. This has been performed several times with the same results: aborted conversion. Besides, every time there is an error, such as a lost connection, all work is lost.

Also tried the offline trick. The conversion has been running in my local server for over 5 hours and is has converted only 500.000 rows (1/3 of the total). It still has a long way to go.

Tried tweaking the batch limit. Have tried several values, 1000, 2000, 3000, 6000, with no noticeable improvement. Locally, 6000 lines seems to improve speed a little, on the remote server, it crashes it. I had to set it down to 1000. The other setting, num_wait_rows, don't really understand what is for. Extended Insert ?

[guischarf]

I think that for large boards it would be a better option to have a command line conversion tool.

[Glauco]

Hello,

We have a board with 400,000 posts and 80,000 users and it took around 2 hours to convert. We run 2 dedicated servers, 1 for database, 1 for httpd.

I would really like to know your trick. I have a similar setting, both servers have double dual core processors and 6G RAM. I have a 1.500.000 posts /65.000 users DB. The conversion has been running all night long and I had to stop it in the morning. This has been performed several times with the same results: aborted conversion. Besides, every time there is an error, such as a lost connection, all work is lost.

Also tried the offline trick. The conversion has been running in my local server for over 5 hours and is has converted only 500.000 rows (1/3 of the total). It still has a long way to go.

Tried tweaking the batch limit. Have tried several values, 1000, 2000, 3000, 6000, with no noticeable improvement. Locally, 6000 lines seems to improve speed a little, on the remote server, it crashes it. I had to set it down to 1000. The other setting, num_wait_rows, don't really understand what is for. Extended Insert ?

Are you building an index for fulltext native search method? Fulltext mysql is way faster to build and WORKS. Fulltext native never worked for me in phpbb2.
The conversion, including search index building, took less than 6 hours in a board with 1 million posts and 7000 users.

[alex.nenov]

I read all about conversion from phpbb2 to phpbb3 but i can't understand why we have to do many trick to convert our boards? It's fact that this converter is perfect for small sites but site with more that 2 000 000 posting it just not working. I can't stop my forum for the same reasons like many of this theme - advertisers and users. Bu i HAVE to convert the board to new version. So I try to convert it 3 times on my localhost ( Win XP + AppServ ) and every time somewhere about 200 000 topics MySQL just crash. I change the option of php, apache and MySQL so now the proces is faster but when i calculate the time with number of posting it seems to convert the board about 44 hours ?!?! Last 3 times for 200 000 posting converted i wait about 7 hours.

The best thing will be to start conversion from console but is that possible?

[guischarf]

Are you building an index for fulltext native search method? Fulltext mysql is way faster to build and WORKS. Fulltext native never worked for me in phpbb2.
The conversion, including search index building, took less than 6 hours in a board with 1 million posts and 7000 users.

Building the index is not a problem since it is performed after the migration has been completed. The problem is the migration itself. The only reasonable (if it could be called 'reasonable') way I found to do it was to download the database to my local PC and perform the migration locally. A shell scritp would be a more reasonable solution for a large forum since it could be set to run as a background process with no interaction with whomever is performing the migration.

We have close to one and a half milion posts and 70.000 users.

[Acyd Burn]

Sometimes it helps to remove the indexes on the posts table before migration and then adding it back after. This elivates the need for mysql to re-calculate the index on every INSERT. Also, tweaking the variables for the batches for large boards directly within the convertor php file may help too.