[RFC] Migrations

[EXreaction]

Some input and a decision on this would be nice so myself or someone else can get this issue handled so we can start getting ready for 3.1.

[Oleg]

Well, you want to ignore the work already put into migrations PR and do something entirely different. What kind of input/decision are you looking for?

Again, the use case that must work is as follows:

1. As a user, I download an extension.
2. The extension must make database changes in order to function.
3. Extension must ship these changes in a manner not requiring me to edit any files.
4. When I install the extension, such as via the ACP UI that I believe is pull requested/merged, required database changes must happen.
5. If I uninstall the extension, another set of database changes must happen to undo the changes made in 4.
6. All phpbb-supported databases must be supported.
7. It should be possible to execute arbitrary php code; specifically, issue raw sql, ponder results and build/execute more sql based on returned results.

Additional benefits that migrations give:

1. We can transition phpbb itself to use migrations, thus having a single framework for making and undoing db changes between core and extensions.
2. Migrations have a notion of dependency. This means if extension A depends on extension B, and I install extension A, and migrations for both A and B are available, then before migrations for A are executed migrations for B would be executed.
3. Migrations are a proven concept in other frameworks and when designed/implemented correctly have a very usable interface.

Now, how does umil stack up to these requirements and additional benefits?

[EXreaction]

For your use case, all of that is possible and could be implemented with UMIL within a couple hours of work max including testing.

As I said above, UMIL wasn't designed for a lot of simultanious distributed development, so no, it would not be as good as proper migrations.

My argument is that I can get everything we _need_ for 3.1 in an afternoon if I use UMIL, as far as migrations go, who knows how long that will take? Yes we want migrations, but I think it can safely be pushed to 3.2.

[Oleg]

you must declare updates to be specific versions that go in order (so if you'd merge Y in the above example, you'd have to modify the install file for X to have yours be a slightly higher version and then also hackishly install the updates required).

This is very beneficial for phpBB development, but I'd argue that it's not quite so important for mods/extensions because they typically have less distributed development.

My feeling is if migrations are provided, extension authors will take advantage of better api/flexibility migrations offer.

UMIL could actually probably be modified to do this fairly easily as well, it just would be required that UMIL is a part of the core because it would require an additional table in the database (which is not something that would hold it back now anyways if we wanted to do this).

Not really seeing how this would be better than finishing migrations?

I do not really think writing something from scratch that does everything UMIL does, but a bit more, is a good use of time. UMIL took a good bit of effort to provide a friendly install/update/uninstall environment so you could go from what is our database_update.php, to a simple array, like this: https://github.com/EXreaction/User-Blog ... tabase.php

Even in that example you have the code spread all over the place for updating to/from the same version.

We have this issue already in the core where you need to remember to add cases for both schema and data changes. I can tell you that a lot of prs don't do both.

Lastly, umil and migrations are really different layers. Migrations are more about sequencing/dependencies and maybe execution than the actual sql construction. Umil is mostly the opposite. You can have umil (or only borrowed umil guts) under migrations doing the actual queries and such.

I would prefer migrations finished with db_tools/umil under them as necessary, because migrations offer in my opinion a better interface.

[Oleg]

For your use case, all of that is possible and could be implemented with UMIL within a couple hours of work max including testing.

My argument is that I can get everything we _need_ for 3.1 in an afternoon if I use UMIL

I am not in principle opposed to using umil. However, I personally have nearly zero experience with it as I have found it easier to simply write raw sql.

So far you have not provided an API/sample extension client code or anything to give me an idea of what it is exactly you plan to do, other than it's "umil". Again I simply cannot imagine what you have in mind.

Have you discussed this idea of using umil with current/potential 3.1 extension authors to get their opinion on the matter? I would suggest either doing that and if they agree with you providing a proof of concept/basic implementation as a PR, or writing up a longer RFC including at least hints of what the public interface will look like. Or get someone else to approve/disapprove this umil idea.

[EXreaction]

I should have time to work on stuff more this weekend, so I can make a quick PR for UMIL support and how it could be used for extensions. Once that's done I'll let you know and show you how it would work.

[EXreaction]

So I created a branch to have the extension manager able to automatically run a UMIL instruction set and do some basic enable/purge from that set automatically.

https://github.com/phpbb/phpbb3/pull/1014
https://github.com/EXreaction/phpbb3/bl ... n/base.php (these changes are most relevant)

Then I basically wrote up the ext.php script for what I'd need to do for one of my existing mods (advertisement management) to port it to an extension:
http://pastebin.com/aUB1aFYB

With that, when that extension is enabled, it installs/updates the db as appropriate, and sets my enabled config variable to enabled (which I used in that mod).
When disabled it just sets my config var to disabled.
When purged it uninstalls the db changes from the mod.

Another advantage to this for extension authors is that it would allow porting 3.0 mods to extensions much more easily if they already use UMIL (they wont need to rewrite the db install script).

[Pony99CA]

Another advantage to this for extension authors is that it would allow porting 3.0 mods to extensions much more easily if they already use UMIL (they wont need to rewrite the db install script).

That would be a plus. Then, in 3.2, they could migrate the DB updates to Migrations. (By the way, that's another sketchy name -- when I first saw this RFC, I thought that it referred to migrations from phpBB 3.0 to 3.1 or from other BB systems to 3.1. Oh well.)

Or, maybe somebody could write a UMIL-to-Migrations converter, but I have no idea how hard that is as I've never looked at UMIL.

Steve

[DavidIQ]

I agree with both EXreaction and Pony here. Using something as familiar as UMIL in 3.1 would help MOD authors transition easier to the extensions model. Would also reduce the size of extension packages based on many MODs that currently exist in that they wouldn't need to include UMIL anymore (yay).

[MichaelC]

Migrations is not just about extensions, its also about being able to release further olympus releases after an ascreaus release (security issues etc.).