[RFC|Replaced] UMIL

These RFCs were either rejected or have been replaced by an alternative proposal. They will not be included in phpBB.
Post Reply
User avatar
igorw
Registered User
Posts: 500
Joined: Thu Jan 04, 2007 11:47 pm

[RFC|Replaced] UMIL

Post by igorw » Tue Apr 13, 2010 7:23 pm

This RFC was replaced by [RFC] Migrations

I already posted an RFC for some design aspects of an UMIL-like system. I'm going to take this a step further and suggest including such a system in phpBB 3.1. This proposal topic is based on one I posted in January 2010 in one of the private forums. naderman and A_Jelly_Doughnut have stated their support.

What is UMIL

UMIL is a library/framework for MOD installation and updates. It wraps around phpBB's API to provide a unified interface for installation and update scripts. This includes db_tools, set_config, auth, modules and many other parts. It works a lot like phpBB 3.0's database updater. There is UMIL auto which takes a large array of version numbers and associated actions + arguments. Additionally it's possible to include the files and make the according calls manually.

One of the reasons why UMIL was created was to remove the need of SQL in MOD install files. And (as described later) to allow automation and different user interfaces for running the actions.

The abstraction provided by UMIL makes MOD installation and updates very easy. It makes it easier for the mod writer and (in theory) the person installing the MOD.

Status Quo

Currently UMIL is maintained by the MOD team which allows them to release new versions any time. The problem however is distribution. Because they don't want end users to have to maintain their own UMIL installs the MOD authors must include the current UMIL with the MOD. This is bad because a) bugs aren't fixed, b) MOD sizes rise, c) overwriting existing UMIL when extracting new MODs.

The amount of MODs using UMIL is still relatively low. Many MODs still use custom install scripts.

Proposal

The phpBB 3.x core should include UMIL or something comparable. This would resolve packaging issues and provide a standard API for installation scripts to all MOD authors. Ideally phpBB would re-use this system for it's own installation. It could be used to easily create in-memory sqlite databases for unit tests.

There would be some changes required to make this fully possible though. The user interface would need some better abstraction to allow automated calls of the install file. As a possible alternative to plain PHP install files, the instructions could be specified in alternative format, such as XML. Another potential addition would be the possibility to revert certain actions on uninstallation (eg using the Command pattern).

The system should be pluggable, allowing MOD authors to create their own installation "actions". If certain custom actions are highly demanded and were implemented for a MOD, they be contributed back upstream and included in the core.

Some ideas for implementation details can be found in: [RFC] UMIL design proposal: Data providers.

Ticket: http://tracker.phpbb.com/browse/PHPBB3-9737
Last edited by igorw on Mon Jul 19, 2010 4:31 pm, edited 3 times in total.

User avatar
imkingdavid
Registered User
Posts: 1050
Joined: Thu Jul 30, 2009 12:06 pm

Re: [RFC] UMIL

Post by imkingdavid » Tue Apr 13, 2010 9:00 pm

I agree with this idea fully, especially for the point you made where older version of UMIL overwrite newer versions in MODs that are validated before the new release of UMIL. I think it should be a standard feature of 3.1.
I do custom MODs. PM for a quote!
View My: MODs | Portfolio
Please do NOT contact for support via PM or email.
Remember, the enemy's gate is down.

User avatar
VSE
Extension Customisations
Extension Customisations
Posts: 670
Joined: Mon Mar 08, 2010 9:18 am

Re: [RFC] UMIL

Post by VSE » Wed Apr 14, 2010 5:16 am

I agree with this idea too. I have a MOD which uses UMIL to perform three simple SQL queries... It just seems like a lot of overhead to need to provide the entire UMIL package with my MOD, keep up to date with it, and ask end users to install it - all for the sake of a very tiny SQL query.

I like the way UMIL makes it simpler for end users to "install, update, uninstall" a MOD which is why I use it. If it were an innate part of phpBB that would be just awesome, speaking as a MOD author and MOD user 8-)
Has an irascible disposition.

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC] UMIL

Post by Oleg » Wed Apr 14, 2010 8:48 am

A strong opinion I have is that the db altering part of umil and phpbb installer should be included into permanent phpbb install.

Current situation is that during initial install or phpbb update the installer runs lots of queries. These queries are specified either in db-agnostic form or with multiple versions for supported databases. The packaging of code however is such that it cannot be reused elsewhere. After install or update is done, the code is deleted (it's in install directory).

Later prospective mod author wants to write a mod. The mod requires some db changes. phpbb installer could do them, but installer code is not available during mod installation and is not in a usable structure. There is umil, but it does not support everything phpbb installer supports (I think, and maybe vice versa too).

I ended up writing my own db migration code, since neither umil nor phpbb installer were usable as is.

The database part of umil requires no management; it's just a bunch of code that needs to be merged with phpbb installer db code and properly factored to be usable by mods, phpbb installer, and custom scripts (think integration with third-party software). Perhaps it would be worthwhile to split db part from the rest of umil for the purposes of this discussion.

User avatar
naderman
Product Manager
Product Manager
Posts: 1727
Joined: Sun Jan 11, 2004 2:11 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC] UMIL

Post by naderman » Wed Apr 14, 2010 2:19 pm

The basis for all the db operations is already included in db_tools.php in the regular phpBB distribution. I agree that rather than bringing a copy of that with it in database_update.php, phpBB should ideally use that file. It's noteworthy that the updater is built in this way so that bugs in the files used by the updater cannot prevent the updater from updating. However that should only ever be an issue with the files and not the schame if done right, and even that should not usually occur. If something as bad as this ever happened one could deliver a special updater for the problematic version. So I agree that at the very least the database altering and module installation parts of current UMIL and database_update need to come with phpBB and be used by MODs and phpBB itself alike. I'd be happy to see a more detailed proposal / an actual implementation in the form of a patch that we can start with.

User avatar
ToonArmy
Registered User
Posts: 335
Joined: Fri Mar 26, 2004 7:31 pm
Location: Bristol, UK
Contact:

Re: [RFC] UMIL

Post by ToonArmy » Fri Apr 16, 2010 5:50 pm

It's very simple, if new/includes/db_tools.php exists in the updater package, use that.
Chris SmithBlogXMOOhlohArea51WikiNo support via PM/IM
Image

User avatar
nickvergessen
Former Team Member
Posts: 733
Joined: Sun Oct 07, 2007 11:54 am
Location: Stuttgart, Germany
Contact:

Re: [RFC] UMIL

Post by nickvergessen » Sat Apr 17, 2010 10:47 am

ToonArmy wrote:It's very simple, if new/includes/db_tools.php exists in the updater package, use that.
We also do this many other times, f.e. the language/en/install, the includes/diff/ adm/style/install_* etc.
Member of the Development-TeamNo Support via PM

User avatar
igorw
Registered User
Posts: 500
Joined: Thu Jan 04, 2007 11:47 pm

Re: [RFC] UMIL

Post by igorw » Sun May 30, 2010 9:53 am

We could call this system "migrations", since what it does is migrate the database to a new state/version.

Jhong
Registered User
Posts: 50
Joined: Tue Dec 26, 2006 3:28 pm

Re: [RFC] UMIL

Post by Jhong » Tue Jun 22, 2010 10:42 am

I agree that UMIL should be in the core.

I don't think it is very natural for users installing MODs to have to potentially overwrite the UMIL directory with each MOD -- it is not consistent with any other file copy action they will have to perform. Just another layer of complexity/opacity for the end user. UMIL in the core would fix problem.

User avatar
DavidIQ
Customisations Team Leader
Customisations Team Leader
Posts: 1731
Joined: Thu Mar 02, 2006 4:29 pm
Location: Earth
Contact:

Re: [RFC] UMIL

Post by DavidIQ » Tue Jun 22, 2010 12:45 pm

Jhong wrote:UMIL in the core would fix problem.
And also reduce its size. Right now pretty much any existing UMIL language is included from which you'd potentially use only 1 of those.
Image

Post Reply