[RFC] UMIL
[RFC] UMIL design proposal: Data providers
Goal of this RFC
This RFC aims to satisfy these proposals through an overhaul of the phpBB database updater with the use of so-called migrations. A migration generally defines a set of schema changes as well as a set of instructions that modify data in the database to accomodate the schema changes or to simply add or modify existing information. The current db_tools class provides all functionality required to make schema changes and it is already used by the database updater. Further functionality for setting configuration values, modifying, adding or deleting permissions, altering control panel modules, etc. will be largely taken from UMIL.
Rather than creating dedicated installation scripts, migrations describe the changes within methods of a class. A generic installer & updater can then run all the necessary changes for phpBB itself as well as all installed MODs. The database updater in the install folder will no longer contain significant code that will be missing for MODs later. This simplifies MOD installation scripts and allows us to create a unified interface that will be used by all MODs. At the same time MODs shipping with their own version of UMIL will continue to work, but will be discouraged.
Other Implementations of Migrations
Traditional implementations of migrations typically apply migrations linearly. Migrations are executed in order of their (often numeric) names or simply in chronological order. Since we work with feature branches in git, it becomes unfeasible to keep a database up to date with these migrations: Consider two branches A and B. Branch A was created before branch B. Both branches contain a migration. Branch B is now merged into develop. A developer updates their board, the migration from branch B is applied. Now branch A is merged. Since branch A's migration chronologically precedes the migration in branch B, it will not be applied. The same problem can be found simply with bugfix releases of multiple concurrently developped feature releases. Someone could either update from 3.0.9 to 3.0.10 to 3.1.1 or from 3.0.9 to 3.1.0 to 3.1.1. Either one or both of these paths would not apply some of the migrations if the are executed sequentially. We could maybe get this to work by merging and modifying old entries in the database updater, but that would be risky. An example:
Code: Select all
3.0.9: database_update.php:
from 3.0.8: set_config('foo', 1);
3.1.0 database_update.php:
from 3.0.8: set_config('foo', 1);
from 3.0.9: set_config('foo', $config['foo'] + 2);
3.0.10 database_update.php:
from 3.0.8: set_config('foo', 1);
from 3.0.9: set_config('foo', $config['foo'] + 4);
3.1.1 database_update.php
from 3.0.8: set_config('foo', 1);
from 3.0.9: set_config('foo', $config['foo'] + 2);
set_config('foo', $config['foo'] + 4); (merged from 3.0.10)
from 3.1.0: set_config('foo', $config['foo'] + 8);
value of foo:
3.0.9 -> 3.1.0 -> 3.1.1
1 3 11
3.0.9 -> 3.0.10 -> 3.1.0 -> 3.1.1
1 5 9 17
This is an example of two simple linear migrations, the first creates a config table with integer ids and the second adds a boolean dynamic flag to it. Similar to the config table in phpBB.
- A simple linear migration
- migrations-1.png (12.08 KiB) Viewed 36310 times
Instead I propose a graph of migrations, in which every migration defines its immediate dependencies. All migrations that can be found will be applied, as long as their dependencies are satisfied. Recursively this will apply all migrations in an acceptable order, regardless of the current database state.
This example shows an additional migration that changes the config table ids from integers to strings. This migration is independent of the migration adding a dynamic column. Each migration is added in a separate branch. They can be applied in either order.
- Migrations in two concurrent branches
- migrations-2.png (20.68 KiB) Viewed 7441 times
phpBB versions are represented by migrations that have neither schema nor data changes. They depend on all the leaf-migrations in the current code, allowing later migrations and MODs to simply refer to the version migration instead of particular other migrations.
- Release migrations without schema and data changes
- migrations-3.png (24.74 KiB) Viewed 7441 times
As a byproduct we can easily use migrations for MODs. If updated MOD migrations are available at the time of a phpBB update they will be updated together. Migrations in MODs encode which phpBB versions they run on. So if you update your MOD to a new version which requires a newer phpBB version, it will not apply the migration until phpBB has been updated.
The following picture shows a MOD called Foo which creates its own configuration table for 3_0_RC3 because it uses string identifiers. After version 3.0.0 is released the MOD first copies all of its own configuration data to the global phpBB configuration table, which now uses string identifiers, and then drops its own config table.
- MOD migrations
- migrations-4.png (43.37 KiB) Viewed 7441 times
In addition to regular migrations I propose that we continue to provide a direct installer since some parts of the update process can be rather lengthy. This is a special migration that lists all the migrations it contains. For phpBB itself this will continue to be the regular installer. MODs will have the option of providing a special installer or simply relying on the application of migrations. Migrations can optionally provide a rollback mechanism. This is useful in development and for the uninstallation of MODs. A MOD can either provide a special purge migration or rely on its migrations to be reversible.
Ticket: http://tracker.phpbb.com/browse/PHPBB3-9737
Sub-task Ticket: http://tracker.phpbb.com/browse/PHPBB3-9738
Patch (in progress): https://github.com/phpbb/phpbb3/pull/1067
