Improved template system for phpBB 3.1
This is a draft and subject to change
A more efficient implementation of phpBBs template engine ported from ascreus-experiment with some minor additions especially for the new hook feature.
Compatibility
- The template class API remains unchanged.
- Newlines in templates will be preserved. The current template engine preserves or removes newlines seemingly at random.
Technical details
The template compiler is implemented as a PHP stream filter. This means that templates are no longer read into memory as a whole, but compiled in chunks. This significantly reduces memory requirements. Due to changes in the compiler logic compilation is (insignificantly) faster.
The generated codes is a bit more optimized, for example it uses references to frequently used variables like $user->lang.
Tracker
Related Ticket: http://tracker.phpbb.com/browse/PHPBB3-9726
Current code in progress: https://github.com/p/phpbb3/compare/dev ... ate-engine
Pull request: https://github.com/phpbb/phpbb3/pull/171
[RFC|Merged] Improved template engine
[RFC|Merged] Improved template engine
Don't give me my freedom out of pity!
Re: [RFC|Accepted] Improved template engine
I like the templating engine approach adopted in phpBB, but dislike some of the unnecessary inefficiencies in it's implementation. By way of an experiment, I decided to implement a templating engine for my own blog drawing on some my experience of using phpBB's and another engine. I discuss this in the article "My blog’s templating engine". The engine works well and is very efficient in compilation and run-time terms.
OK, I use more of a Smarty syntax and it's not worth the regression work changing the clunky phpBB syntax, but some of the issues that I have addressed are well worth considering including in the phpBB engine as these give good performance benefits:
(Sorry if this post is in the wrong topic, but if so perhaps one of the mods can move it and let me know).
OK, I use more of a Smarty syntax and it's not worth the regression work changing the clunky phpBB syntax, but some of the issues that I have addressed are well worth considering including in the phpBB engine as these give good performance benefits:
- Having a per language compilation process. This isn't an issue for most forums which only operate one or two languages. However, this allows the hoisting of all language-specific processing into the template compilation process and language strings such as {L_FORUM_RULES} simply become embedded HTML text in the compiled template.
- The list of variables referenced in the template is collected and a simple 6 line preamble establishes reference synonyms for these to the template variable with a (blank) default if not set. This means that a reference {U_POST_NEW_TOPIC} now gets compiled to echo $varPOST_NEW_TOPIC;.
- A global switch (in phpBB a board configuration setting) cases included templates to be compiled inline during the compilation process. OK, this doesn't make much sense if the installation is using an opcode accelerator such as APC, but it does have performance advantages for smaller boards installed on a service which doesn't have such caching available.
- My language lookup is though a customisable callback and the convention that I have adopted is use the English text as the lookup key and I keep the language table in the database. (This isn't a runtime overhead since error paths are about the only language lookup that I do outside the compilation process). The major advantage is that the templates are a lot more readable.
(Sorry if this post is in the wrong topic, but if so perhaps one of the mods can move it and let me know).
Forum Admin OpenOffice.org User Groups
Re: [RFC|Accepted] Improved template engine
This is a good idea for most boards, but unfortunately for the few boards that do have many styles and many languages it may be a very negative change. Which means either the implementation has to be conditional or some other solution is needed.TerryE wrote: Having a per language compilation process. This isn't an issue for most forums which only operate one or two languages. However, this allows the hoisting of all language-specific processing into the template compilation process and language strings such as {L_FORUM_RULES} simply become embedded HTML text in the compiled template.
I suppose we can do this, are there any downsides? Do you have a patch we can look at?TerryE wrote: [*]The list of variables referenced in the template is collected and a simple 6 line preamble establishes reference synonyms for these to the template variable with a (blank) default if not set. This means that a reference {U_POST_NEW_TOPIC} now gets compiled to echo $varPOST_NEW_TOPIC;.
Inlining code is not automatically a good thing. What are the gains here precisely?TerryE wrote: A global switch (in phpBB a board configuration setting) cases included templates to be compiled inline during the compilation process. OK, this doesn't make much sense if the installation is using an opcode accelerator such as APC, but it does have performance advantages for smaller boards installed on a service which doesn't have such caching available.
The problem with this is it makes it impossible to edit the English language strings in minor releases as it will break all translations and will affect modifications as well.TerryE wrote: My language lookup is though a customisable callback and the convention that I have adopted is use the English text as the lookup key and I keep the language table in the database. (This isn't a runtime overhead since error paths are about the only language lookup that I do outside the compilation process). The major advantage is that the templates are a lot more readable.
Re: [RFC|Accepted] Improved template engine
Per language compilation process
Using reference synonyms
I would be reluctant to propose a patch for something that is really beyond patching. It really needs a proper refactoring / rewrite.
Optional inlining of template includes during the compilation process
[Your point about the string keys decoupling translations from base English is valid, plus regression costs across all templates so forget this one ]
I am happy to look at a rewrite /refactoring of the engine over the next few weeks (initially 3.0.8 compatible but with PHP 5.2 coding) as long as I am not cutting across someone else who is also actively doing this. It's a bit of work, so I'd only do it if it stands a good chance of being used. I would assume that the invariants are the template language / syntax and the data access API.
It would only be a negative change in the scenario where the board has a lot of languages and is using a PHP accelerator such as APC, because of the code cache increase. However, it would be trivial to add an extra flag to the phpbb_config table, so the sysadmin could elect to do this or not.nn- wrote:This is a good idea for most boards, but unfortunately for the few boards that do have many styles and many languages it may be a very negative change. Which means either the implementation has to be conditional or some other solution is needed.
Using reference synonyms
The major benefit is in reduction of the compiled template simplicity and size, both source and compiled opcode (for APC etc.) The runtime difference is in microseconds. See my article My blog’s templating engine where I explain this. It does work well.nn- wrote:I suppose we can do this, are there any downsides? Do you have a patch we can look at?
I would be reluctant to propose a patch for something that is really beyond patching. It really needs a proper refactoring / rewrite.
Optional inlining of template includes during the compilation process
This inlining is transparent at the template level and would only be apparent at the compiled .html.php level. This is really for the smaller installs running on a shared service. Less files = less I/O. The file space overhead of including the overall_header in all templates where it is referenced is minimal.nn- wrote:Inlining code is not automatically a good thing. What are the gains here precisely?
[Your point about the string keys decoupling translations from base English is valid, plus regression costs across all templates so forget this one ]
I am happy to look at a rewrite /refactoring of the engine over the next few weeks (initially 3.0.8 compatible but with PHP 5.2 coding) as long as I am not cutting across someone else who is also actively doing this. It's a bit of work, so I'd only do it if it stands a good chance of being used. I would assume that the invariants are the template language / syntax and the data access API.
Forum Admin OpenOffice.org User Groups
Re: [RFC|Accepted] Improved template engine
Marek (APTX) has been working on a changes to the template engine, so some coordination would be good here. Personally I've been looking into maybe replacing the template engine entirely with twig to give us some more flexibility/extensibility to introduce new constructs for MOD hooking in templates, while keeping things 100% backward compatible. More on that once I actually get to it and can see if this is a viable option
Re: [RFC|Accepted] Improved template engine
I see two main problems with templating engines like twig and smarty: (i) whilst they are extremely flexible and powerful, you do pay a performance penalty; (ii) their syntax and semantics are materially from the existing template language specification (TLS), and because of the package complexity, you would need to do a complex and costly-to-maintain fork to maintain compatibility with the existing TLS. If you want to move to another TLS then you've got a major change and regression test cost for existing templates out there.
I guess that you also thing that the current templating language sucks, but a proper refactoring of the existing engine based on a PHP 5-compliant object model would be far simpler and safer IMHO.
I guess that you also thing that the current templating language sucks, but a proper refactoring of the existing engine based on a PHP 5-compliant object model would be far simpler and safer IMHO.
Forum Admin OpenOffice.org User Groups
Re: [RFC|Accepted] Improved template engine
Please look at https://github.com/APTX/phpbb3/tree/fea ... ngine-fate for the 3.1 template engine.
I'll look into implementing at least the "per language compilation".
I'll look into implementing at least the "per language compilation".
Don't give me my freedom out of pity!
Re: [RFC|Accepted] Improved template engine
APTX, I'll clone your APTX / phpbb3 branch and go through this, and give any specific feedback on this 3.1 snapshot, but at a first look you seem to have done most of the refactoring that I was thinking of.
Forum Admin OpenOffice.org User Groups
Re: [RFC|Accepted] Improved template engine
twig is actually significantly different from smarty in this regard. It's more of a framework to define your own templating language with a sensible default. Using OOP you can define the tokens and AST nodes for parser and lexer. We don't need to fork their code at all to make it support our current templating language. I haven't made detailed performance benchmarks yet, but based on how twig works it should operate at a similar speed as our current template engine. The reason for looking at it at all is to avoid having to spend a lot of time on building a new OOP template engine ourselves. phpBB in general suffers from a NIH mentality problem, and it's really not necessary to reinvent template engines yet again.TerryE wrote:I see two main problems with templating engines like twig and smarty: (i) whilst they are extremely flexible and powerful, you do pay a performance penalty; (ii) their syntax and semantics are materially from the existing template language specification (TLS), and because of the package complexity, you would need to do a complex and costly-to-maintain fork to maintain compatibility with the existing TLS. If you want to move to another TLS then you've got a major change and regression test cost for existing templates out there.
I guess that you also thing that the current templating language sucks, but a proper refactoring of the existing engine based on a PHP 5-compliant object model would be far simpler and safer IMHO.
Apart from that I'm not saying this is going to happen, I'm purely looking at it to see if it can really be done that easily in which case I will propose this as an alternative. So we can get into a more detailed discussion on pros/cons of twig then
Re: [RFC|Accepted] Improved template engine
Work in progress: https://github.com/p/phpbb3/compare/dev ... ate-engine
There are some failing tests: https://gist.github.com/939261
Edit: tests are fixed.
I submitted the pull request: https://github.com/phpbb/phpbb3/pull/171
Let's get this tested and merged.
There are some failing tests: https://gist.github.com/939261
Edit: tests are fixed.
I submitted the pull request: https://github.com/phpbb/phpbb3/pull/171
Let's get this tested and merged.