eviL3 wrote:Perhaps you could state your target audience and use cases a bit more clearly. Unit testing vs manual testing. Is this something for the QA team? How exactly would the person using it use it? I'm just trying to get a better idea here.
Restrictively defined, the goal of phpbb proving ground (working name) is to allow developers to rapidly provision phpbb environments for testing code under development.
Conceivably testers (including QA team) can also provision phpbb environments for testing code after it is "finished", but barrier to entry may be too high given constraints described in the first post.
By "phpbb environment" I mean a combination of the following:
1. Code. Any branch of any fork of any version of phpbb (phpbb3/phpbb4) specifically is to be supported in initial release. Subversion checkouts are possible under the architecture but are not a priority until requested.
Said differently, any developer should be able to deploy any code derived from phpbb source tree for testing.
2. Database. All databases supported by phpbb and requested by developers are to be available at all times.
3. Data. Ppg allows defining sets of data that can be rapidly loaded into the database. The goal is to make it easy to obtain states that are normally not easily obtainable. Datasets are independent from code, databases (as much as possible) and other factors, and are expected to be shared among developers. Using a dataset built by someone else is to be trivial.
4. Web server and php execution model. As with data the goal is to make it easy to obtain unusual combinations, such as apache+fastcgi.ppg and manual testing
Until there is much more unit test coverage, ppg is expected to be used primarily for manual testing. Here "manual testing" refers only to the actual testing part; preparations such as setup and data loading are to be automated and are the scope of ppg.ppg vs unit testing
Unit testing requires having a database configured, writable directories arranged, and perhaps some initial data loaded. Ppg takes over these processes. Once code is deployed to ppg, unit tests can be executed using whichever unit test framework/runner are in use.
Ppg allows in this case:
1. Unit testing on databases developer does not have installed (vs testing locally)
2. Unit testing code that is not merged into main development branch(es) (vs testing in continuous integration tool)
Ppg makes it possible to test things that are hard to write unit tests for:
1. Performance testing, including actually using the board in the state being tested
2. Testing things other than php code, for example mail submission and/or deliveryppg vs continuous integration
Continuous integration usually works on a small set of branches, using code that is "finished". It is not possible to test an arbitrary source tree, and code which is not finished (under active development) is not submitted to continuous integration tools.
Unlike bamboo, ppg is free software.Use cases
For manual testing, I don't think it makes much sense to deploy into several different environments. This is something that should be covered by unit tests.
Let's suppose you are developing a feature like this one: http://tracker.phpbb.com/browse/PHPBB3-9173
You can write a unit test, but do you have firebird and postgres from 7.4 through 8.4 installed on your system? Even if all these databases are supported by CI tool eventually CI only tests code committed to main branch and does not help you during development.
With ppg you only need to commit the changes to your branch and then create 5 different configuration files (~5 lines each) and a dataset for required conditions (specifically the current attachment size sum should be large), and ppg will give you 5 boards that you either 1) run unit tests on or 2) login to, post attachment and check new file size (what would have happened presently).
A short time after the administrators try to send a mass PM to all the users on the board, some pages don't load due to a maximum execution error on the cache/queue.php.
The forum obviously tries to send an email to all the users to notify about the new PM, and for some reason the queue system fails.http://www.phpbb.com/community/viewtopi ... &t=2042715
Unit testing is not helpful here because to start, the problem is not known. To even begin you need a board with a large number of users to spam, and you obviously don't want to spam real users so you need to specify special email addresses that you own for everyone. Ppg can address this use case by providing an MTA that delivers mail locally, so it's a real MTA as far as phpbb can tell but perfectly harmless, and providing a dataset consisting of 2000 users with appropriately configured email addresses.
This is a large chunk of code with nontrivial changes. Testing it requires resetting the board to a particular state where none of the cron tasks have run recently, there are forums that need to be pruned and not pruned, both forums should have topics in certain time ranges. Running the cron tasks destroys the state, and getting it back gets old very quickly. Ppg addresses the issue by storing the state in a format that allows it to be loaded with a single command.
How exactly would the person using it use it?
I take documentation for this project seriously, essentially developing it documentation-first. There isn't much documentation yet because there isn't much code written, but here is a start that should give some idea: http://svn.bsdpower.com/webtools/phpbb- ... kflows.txt