[RFC] Error Diagnostic System (EDS)

[Go2GamerGuys]

Many people have big forums, and sometimes will get errors such as SQL errors when they install mods or these could just happen spontaneously. The job of Admins is to keep the forum working, so they do not have time to check every file or uninstall everything to get their forums working again if they have an error.
Instead, they can use the Advanced Error Diagnostics System (EDS)

How it would work:
The Advanced Error Diagnostics System will make a clean cache copy of your forum every time something as simple as a guest views your forum. It will have bots or crawlers that will constantly scan the forum and do every possible task to check for errors. If there is ever a detected error, there will be a red message displayed at the top of the ACP, and you will receive an automated PM. The Advanced Error Diagnostics System is similar to a back ups system, but much more intelligant. If you find an error that the EDS did not detect with any of its scans, you can ask it to deploy a full scale scan, where your forum is temporarily closed while multiple bots complete every possible task to try to re-create the error, plus a copy of your forums database is made to compare to the original clean cache copy. The EDS will scan both copies, detect the differences, and you will either have the option to use the built in editor tool to correct it, or you can have EDS auto correct it.
If the error is mod related, this is how it would work.
Every modification would be required to also have a standard clean copy of the mod which will go into the EDS library. If you have an error related with mods, EDS will first do the above scanning task, and then will specifically scan all of your mods and compare them to the clean copies of the mods you have installed in your EDS library. If EDS finds a difference between the clean copy of the mod and the one you are using on your forum, it will again either allow you to correct it with your editor or can auto correct it.

EDS is also an advanced Back Up system, which makes routine copies of your MySQL database and your Root database and stores them in its back up library and also offers downloads when routine back ups are done. You can set scheduled back ups through your ACP, and can download copies of back ups through your ACP.

EDS also makes copies of your ACP and MCP settings, and can automatically revert all settings in the MCP and ACP to the phpBB3 default, or back to your saved EDS settings.

How to access EDS:
EDS has standard accessed through the EDS Tab in your ACP, which comes with full control and access over EDS.

EDS also makes a special Bot/Admin account, which you can log into. This has advanced EDS settings including mass functions such as posting hundreds of posts and deleting them instantaneously, scanning the forum, making back-ups and configuring the personal settings of the EDS account.

EDS also has a library in your Root Database, so you can access EDS by going to http://www.example.com/phpBB3/EDS Then you log into any Admin account or use your EDS PIN to log in (Can use bot account to log in, but may not edit bot settings if you logged in that way). In the EDS library, you can access things such as emergency back-ups, emergency board restore, restore admin access to an account, configure any errors, change the running script of EDS, and uninstall EDS. This way of accessing EDS is the super way with every control including the EDS Bot Account, an imitation of the EDS ACP Tab, EDS Script, EDS Back-Ups, EDS Emergency Data, EDS Cloud Back-Ups, EDS Force Configuration, and EDS full logs and information lists. Though you will be able to uninstall EDS through the ACP as well, you will still need to end up coming to the EDS library to run the uninstall script, or things like the Bot and special cookies will end up remaining.

For emergency purposes, EDS comes with 2 emergency scripts, which should only be used if you cannot use the other methods. EDS comes with a SQL Query Script, which will force install EDS and give you the EDS Bot account, which are for temporarily fixing errors and problems until you can install the full version of EDS and fix it through either the EDS ACP tab or the EDS library.

In case you do not have access to phpMyAdmin, EDS also has 2 last resort Python Scripts. It is the users job to regularly run the configuration python script so that you have a clean copy of your forum to restore to. Then if you run a restore python script, EDS will force restore your forum to the clean copy. This is the last resort. To use it, you would simply run the restore script, enter any admin username and password, and an EDS crawler will overwrite all data and mods with the clean EDS version.

[Go2GamerGuys]

Since EDS seems so similar to STK, a mod add-on of EDS will allow the combination of STK for full forum management. This will hopefully eliminate STK all together and combine them into 1 super tool. However EDS will still be a standard while the EDS/STK mod is optional as EDS comes with the core tools of STK, however better and more intelligent versions of those tools.

---

In a possible update, EDS will be given even more integration with mods. It will scan each mod, then go to phpBB.com and search for newer versions of mods, and automatically install them without the user even having to press a button. Plus EDS will have a configuration script to allow users to install mods meant for earlier versions of phpBB3x, by editing the mods script for the current version of phpBB3x.

[Oleg]

The scope here is simply way too big. I see the following pieces that can be meaningfully broken out and implemented:

1. Logging of all sql errors to the error log. There have been some changes to the related logic recently. Since we have an error handler for sql errors it should be fairly easy to log all of them to the error log if this is not already done. Perhaps a setting should be added to toggle this logging to prevent the log table from becoming huge on boards with errors (although most boards should not have any errors on them at all). This would be a good RFC to separate and work on. I seem to recall there possibly being a modification that might be helpful for borrowing code from.

2. Non-sql errors, i.e. php errors and php non-error spam. I don't believe it is currently logged at all, but again we have an error handler and therefore we can log this if we want to. There is more reason to have a setting for logging here as php spam can fairly easily reach gargantuan proportions without any involvement by human beings.

3. Unmodified files. If a board is modified then generally speaking you cannot put unmodified files in place and expect it to work correctly. Therefore at most you can compare against pristine files for something like intrusion detection. However, this is really not very difficult to do manually if you suspect your board may have been compromised: download all files and use a free file comparison program to compare the files on your server with the files you saved on your computer (you did save a copy before uploading them, right?). Building an intrusion detection system into phpbb is perhaps too ambitious of an undertaking.

4. Backups. You cannot rely on php code to perform complete backups because of execution time limits. Also, php can silently corrupt data if gzip is used in a certain way (https://bugs.php.net/bug.php?id=12884, https://bugs.php.net/bug.php?id=24083). Therefore any backup system written in php has dubious value at best.

5. acp/mcp settings. These would be included in the backups that you make using non-php tools (you do have backups, right?).

6. Recovery user accounts. Support toolkit already does that from what I gather, instead of duplicating its functionality in phpbb I would suggest contributing whatever functionality is missing from STK in this area to STK.

7. Python scripts. How do you intend to run them? If you have shell access you can just as easily reload your most recent database backup into the database.

[Go2GamerGuys]

I see all of your points. But the reason for a Python script is because EDS will be a standard in all phpBB3 forums, so if you were to use a free host and did not have access to the database, you could use a python script to run EDS. It is the only way I can think of to install and access something in phpBB3 without full database and MySQL database access.

If this were an add-on for STK, then STK should be made a standard in the first place.

[DavidIQ]

So what happens to servers that don't have Python installed?

[Danielx64]

So what happens to servers that don't have Python installed?

That and those who don't have ssh access?

[Go2GamerGuys]

Then you install python. You do not need to install python into your servers. You need a good copy of python 2.x on your own computer to run a python script, you need to retrieve the admins cookies and you would need the admins user name and password.

[Danielx64]

Then you install python. You do not need to install python into your servers. You need a good copy of python 2.x on your own computer to run a python script, you need to retrieve the admins cookies and you would need the admins user name and password.

Huh? I got no idea what your on about here, sorry dude.

[Go2GamerGuys]

You install a 2.x version of Python on your desktop. That is all you need to run Python scripts. You do not need python installed on your servers for it to work. Python works as a crawler, so once you enter the username and password of an admin account into a python script it can receive the cookies of that user and then access the forum as a crawler and make edits to it.
And this is not just an idea. This already works. Python scripts like the one I am describing are used in phpBB3 forum converters if you do not have access to the database or it is not a phpBB3.x conversion to another phpBB3.x.

[Danielx64]

So that mean that someone got to have Python installed on their local computer, no thanks.