Splitting & merging fora from different DBs & phpBB Versions

[drcypher]

There is the following problem: I have two Boards, A (the older one, two years old) and B (kind of recently installed).

Board A has about 30.000 posts, runs phpBB 2.0.21. Board B has about 1000 posts and runs phpBB 3 beta2.

There are about 6.000 posts in board A in specific fora (they are not spread all over the Board) that I want to "merge" with the new Board B.

I know there is not conversion tool available at this time from any version of phpBB to phpBB 3. I thought the procedure would be something like the following:
1. Make a fresh (empty) installation in a board C
2. Import the whole Board A (all posts)
3. Delete unnecessary posts (24.000 that will remain in A)
4. Import (merge) the whole Board B
5. Replace Board B with Board C
6. Delete imported posts from Board A

Apparently post and topic IDs *will* change (resulting in broken links within those topics in the "new Board B"), but this is the least that matters. What I want to make sure is
1. The problem does have a solution
2. Imported users in both boards (A and the new, extended B) will have proper values for the "user_posts" fields, that is the user_posts in A will be reduced (since 6.000 posts will be removed) and user_posts in B will be increased (since those 6.000 posts will be merged/added to it).
3. There isn't going to be any "problem" with the dates, given that it's almost certain that some posts with greater IDs will have an older post dated

Any suggestions might help. Time is not an issue. I can wait until an RC comes out, so that I have a conversion tool, but the actual problem is: Will this allow *merging* another Board as well, or just full imports from scratch?

[NNO-Stephen]

lol, please tell me you're kidding...

[profpete]

Here's a little helpful link

http://www.phpmyadmin.net" target="_blank

You're going to need it

[Graham]

I'll say straight off that the code which ships with RC1 will be convert only - it will not support merges. Whether we add a subsequent plugin which can handle merging two forums remains to be seen, it largely will depend on the work involved and whether there is a real requirement for it.

I know of at least one team member who is planning a merge of several 2.0.x boards before they upgrade them, but the scope of that is completely different.

The number of posts you've got does rather imply you've been running the Beta forum live, and given that there will not be a supported update path from the version you've got there to the released version anyway, you're pretty much on your own even getting it to the relevant level before you can think about doing this

[sedrikk]

Well, there are scripts currently available to merge two phpbb2 forums into 1. I am assuming someone will create a script to merge two phpbb3 forums, eventually. How soon is eventually? No clue, and I am pretty sure anyone that makes something like this will have it requiring 2 identical versions of phpbb3. So you will probably have to upgrade your phpbb2 to phpbb3.

When someone finally makes a script your procedure will probably be something like this:
1) Upgrade Forum A from 2.0.21 to 3.0 (official upgrade script)
2) Upgrade Forum B from beta3 to 3.0 (no official support here)
3) Run script to merge B into A (no official support)
4) Delete B tell people to go to the new A

As for pruning, you would probably have to do that prior to merge.

All speculation on my part. My guess it will be a while before you see a merge script.

[code reader]

There is the following problem: I have two Boards, A (the older one, two years old) and B (kind of recently installed).

Board A has about 30.000 posts, runs phpBB 2.0.21. Board B has about 1000 posts and runs phpBB 3 beta2.

There are about 6.000 posts in board A in specific fora (they are not spread all over the Board) that I want to "merge" with the new Board B.

I know there is not conversion tool available at this time from any version of phpBB to phpBB 3. I thought the procedure would be something like the following:
1. Make a fresh (empty) installation in a board C
2. Import the whole Board A (all posts)
3. Delete unnecessary posts (24.000 that will remain in A)
4. Import (merge) the whole Board B
.....

of course, the problematic stage is stage 4.
this is a general problem, not related specifically to your situation, and can be summed up as: "how do i merge 2 phpbb boards?".
it makes sense that a first step would be to bring the 2 boards to one, known board version.
since conversion tools from one board type to another (phpbb <==> smf, phpbb <==> punbb, etc.) exists, and it is safe also to assume a phpbb2 ==> phpbb3 conversion tool, this step should be considered trivial.
however, i am not aware of any well-established "board merge" or "board import" tool.

i will try to outline what i think such a tool will have to accomplish.

let me outline what this involves:
1) loop through the users of board "b". for each user, look to see if it already exists (presumably by comparing email addresses). if it exists, add "old user id => new user id" entry in the "users translation table". if it doesnt exist, create a new entry in the users table, and then add the "old user id => new user id", and "old user name => new user name" (this table may be trivial, if there are no shared users, or if all shared users managed to select the same name. if two different users (=2 different email addresses) managed to register on the 2 boards using the same user name, you'll have to find the right solution.
2) loop through the forums of board "b". add each one. create an "old forum id => new forum id" conversion table. create all new forums as "top level". at this stage, ignore forum hyrarchy. this can be easily re-arranged after the conversion is done.
3) loop through the topics or board "b". copy each one to the new board, while translating the forum id, users id and users name (for first and last poster id, and poster name) based on the tables created in step 1 and 2. create an "old topic id => new topic id" conversion table
4) loop through the posts in board "b". for each post, translate the forum id, topic id and author id created in steps 1, 2, 3, and add to existing board. create an "old post id => new post id" table.
5) loop through the topics again (only the ones added in step 3), and convert the first/last post id based on the table created in step 4.
6) loop though all the posts added in step 4, and fix the links based on the tables created in step 4 and 5.
7) loop through emoticons of board "b". if the smily "code" does not exist, import it.
8) apply similar logic to the tables i forgot.

if you want "perfection" you'll need to create similar steps for private messages, groups, etc.
this script, however is relatively complex, difficult to debug, and would be used exactly once by each board operator.
it is possible that someone already created such a script for phpbb 2.

i read some people floating the idea that this can be done manually, using a tool such as phpmyadmin. i dont think this is a practical option.

[drcypher]

of course, the problematic stage is stage 4.
[..]
this step should be considered trivial.
however, i am not aware of any well-established "board merge" or "board import" tool.

i will try to outline what i think such a tool will have to accomplish.
[..]
i read some people floating the idea that this can be done manually, using a tool such as phpmyadmin. i dont think this is a practical option.

First of all I'd like to thank all the users that commented about the possibly unsupported features. It's pretty certain that I'll have to write some little script for the merge/conversion, and the outline code reader gave me looks fine. I'm relieved by the fact that the users in B is a pure subset of the users in A, so there will be no need for conversion of the users table.

I'll probably wait for an RC to try it out. I'll post results here. I only hope there will be no memory issue for all those "conversion tables" (for old/new fora, topics, posts, etc)

Thanks again for your time!

ps: Yeap, it's true, I've been using a beta "live" (for the small board B), though I keep it up to date with the CVS. Which means that by the announcement of the RC it will be trivial to "convert" B to phpBB 3 RC1...

[code reader]

I'm relieved by the fact that the users in B is a pure subset of the users in A, so there will be no need for conversion of the users table.

well, naybe you won't have to copy user records from one db to the other, but i think you will have to convert the user id: even though the users in one db are a subset of the users in the other db, you shouldn't expect the user-id associated with each user to be the same in the two databases, so you'll have to create the old user id => new user id conversion tablle, and modify the user-id that appear in the topic and posts table accordingly.