[RFC|Merged] No db storage of stylesheets and templates

[naderman]

Glossary

• Template: .html file, gets compiled to php code.
• Original stylesheet: a .css file in styles directory. It may contain variables in braces, commonly used for localized images.
• Theme: original stylesheet + images referenced in stylesheet
• Final stylesheet: a valid .css file served to browsers, generated from original stylesheet with all variables substituted

3.0 Behavior

Under some circumstances templates and stylesheets may be stored in the database. These circumstances are:

1. Admin edits a template but board has no write access to templates.
2. Stylesheet references images which have text, and therefore can be localized.
3. Admin edits a stylesheet but board has no write access to stylesheets.

Note: original stylesheets may be edited in acp. If stylesheets are writable, they are written out as individual original stylesheets. If stylesheets are not writable they are combined and stored in the database as a single blob of css.

Storing templates and stylesheets in the database creates the following problems:

1. When admin "refreshes" a template or theme (actually stylesheet is what gets refreshed) any changes the admin previously made that were stored in the database are lost.
2. During version updates the updater does not handle stylesheets or templates stored in the database. This either creates a partially updated board or changes get lost per #1.
3. Dynamic generation of final stylesheets makes proper caching of them difficult/impossible.

Proposal

1. In the ACP template editor, changes will be persisted to disk only. This requires either writable template directories or ftp/sftp credentials to be defined such that phpbb can connect to itself over network and write the changes as if an admin uploaded them to the right files.
2. Final stylesheets should be generated for all possible variable replacements for all original stylesheets and stored on the disk rather than original stylesheets being stored in the database and substituted dynamically.

Some discussion of this subject has already taken place at viewtopic.php?f=81&t=32541

Ticket
http://tracker.phpbb.com/browse/PHPBB3-9741
Pull Request
https://github.com/phpbb/phpbb3/pull/377

[sajaki]

This would have the advantage that debugging a css file in firebug will actually lead to an existing file instead of style.php where you don't really know where the originating css came from. So does this mean that the current model of prosilver will no longer be followed, i.e. compiling stylesheets into style.php ? How do you intend to store the dynamic stylesheets ? in the cache directory ?

[naderman]

The cache directory probably makes most sense, since a compiled stylesheet is somewhat like a compiled template.

[Jhong]

This is excellent news. In addition, might I sugest that the source stylesheets be combined into a single file? The multiple CSS files do not aid in administration or editing at all -- even with Firebug or other tools, I am shooting blind when it comes to actually seeing which source file various CSS declarations are in. For me, three separate files: main.css, ie-specific-hacks.css, and user.css would be sufficient.

[Oleg]

It would be nice if the code that generates these stylesheets was callable from command line. For example, if the entry point was a function in a function file (or similar), such that after bootstrapping phpbb I can include the file and call the function to create the stylesheets without any output being printed. Then I will be able to get rid of the Python script I use to create stylesheets during deployment.

[Oleg]

An issue with storing compiled stylesheets in cache directory is that, so far, cache has had no files serveable by the web server and access to it has been prohibited. Putting web content in it would either require us to also allow php script execution from cache, which is very dangerous (as it is required to be writeable by the web server), or would make it easy for users to misconfigure their servers such that cache allows php script execution.

Another reason for separating cache and generated web content is that the latter may be placed in a directory where more sophisticated admins would physically disable script execution (e.g., via a <Directory> override in apache).

Lastly, even more sophisticated admins could pre-generate all needed files during deployment (see preceding post) and could therefore make the generated web content directory read-only as far as web server/php user is concerned.

-----

The variables involved in style.php are user's style and language. I still don't have an authoritative answer as to what should happen if:

1. I install a style, S, which has English images.
2. I install a language L, which has localized images for prosilver.
3. A user chooses style S and language L.

Which images should be displayed in this case?

----

Generating stylesheets for all possible combinations of styles-languages could get expensive. If I have, say, 10 styles or 10 languages installed (which is probably uncommon but not at all unreasonable) an unlucky request might need to generate 10 stylesheets. That will probably take a long time. If we operate on the assumption that the generated web content directory may be cleared at any time, like cache, then a more reasonable approach is to only check and generate the particular combination that a user requested.

We could provide a hidden option (*) which, when enabled, will disable the checks of generated web content by php code. This option will be enabled by boards that pregenerate their generated web content.

(*) The "hidden" options would be similar to about:config in firefox. We would add a page to acp listing all config keys/values and allow people to change them. This would be for 'advanced users" and would allow us to add options for infrequently used functionality without adding UI for them.

-----

Note that technically replacement of style.php with static files does not require not storing stylesheets in database. Those are actually rather orthogonal issues.

[naderman]

Note that technically replacement of style.php with static files does not require not storing stylesheets in database. Those are actually rather orthogonal issues.

Yes they are. Nonetheless, the plan was to no longer store themes and templates (no clue why I messed up the topic title) in the database, and to no longer dynamically generate stylsheets. The issues with storing them in the database are with updates and refreshing them, where users often unintentionally lose all their changes without backup.

[Oleg]

Relevant code: https://github.com/cs278/phpbb3/compare ... -db-styles

[naderman]

Chris posted the final patch at https://github.com/phpbb/phpbb3/pull/377 I've reviewed and merged it.

[Derky]

Before the removal of style.php commit, phpBB read the @import lines from stylesheet.css, included and read the imported CSS files and stored them in the database. The file style.php just throwed the compiled and merged CSS files.

You are now linking directly to stylesheet.css which of course still uses the CSS imports. However, according to Google's Page Speed optimization the previous method, where @import was not parsed via the browser (because it was being parsed by session.php) is faster because browsers cannot download CSS files parallel with @import.

Source: Minimize round-trip times - Avoid CSS @import