[Discussion] Downtime and Server Compromise

Discussion of general topics related to the new version and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Forum rules
Discussion of general topics related to the new release and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Post Reply
thedark
Registered User
Posts: 3
Joined: Mon Feb 02, 2009 8:41 pm

Re: [Discussion] Downtime and Server Compromise

Post by thedark »

do we have an expected return date of phpbb.com

Paul
Infrastructure Team Leader
Infrastructure Team Leader
Posts: 373
Joined: Thu Sep 16, 2004 9:02 am
Contact:

Re: [Discussion] Downtime and Server Compromise

Post by Paul »

No, there is no ETA yet on when it will return. We are working as hard as possible to get it back.

jrobbio
Registered User
Posts: 1
Joined: Wed Feb 04, 2009 6:33 pm

Re: [Discussion] Downtime and Server Compromise

Post by jrobbio »

I've just read one of the posts with someone detailing how they managed the hack.

Is there any advice on how people who have upgraded from PHPbb 2 to PHPbb 3 could run a script on their database to update the hashes of users that haven't logged into the system since the update.

I agree with their suggestion that password changes for Administrators should require the intervention of another admin and not via password recovery.

I didn't understand their comments about sleeping better knowing that I am not worrying about the next way to break in. The fact they uploaded all the accounts makes me feel if they were feeling vindictive for some reason against PHPbb.

I had a look at the site that the exploit came from and I spotted that PHPbbBook 1.3 and PNphpBB2 1.2i have recent exploits too.

bolverk
I've been banned
Posts: 280
Joined: Mon Feb 02, 2009 5:39 pm

Re: [Discussion] Downtime and Server Compromise

Post by bolverk »

I am curious about something that perhaps the phpBB Management team can address at some point. Are you doing any sort of 'Lessons Learned' from this outage, other than the obvious "keep things patched" ;) Perhaps given the lengthy downtime involved in this one attack, setup a mirror site that can be brought online to keep certain things available like downloads, styles, mods, converters, packages other than the full install etc..

I understand that you are not a commercial entity and as such, have no responsibility to be available to the public, but perhaps some redundancy and disaster recovery planning might be in order.

Marshalrusty
Project Manager
Project Manager
Posts: 273
Joined: Thu Oct 27, 2005 1:45 am

Re: [Discussion] Downtime and Server Compromise

Post by Marshalrusty »

jrobbio wrote:Is there any advice on how people who have upgraded from PHPbb 2 to PHPbb 3 could run a script on their database to update the hashes of users that haven't logged into the system since the update.
As has been said earlier in this topic, it is not possible to update the hash. Hashing is a one way process, so you cannot get the plaintext value to rehash it in the new format (at least not consistently enough to create a script for all cases). There are, however, some ideas on how similar affects can be achieved. They are currently being discussed. Anyone on your board with more than regular permissions should have logged in on phpBB3. If they have not done so, then you should just change their password manually from the ACP.
jrobbio wrote:I agree with their suggestion that password changes for Administrators should require the intervention of another admin and not via password recovery.
I honestly don't see how that makes a difference. If an attacker can change the email on the account, he/she can change the password as well. Therefore, that suggestion is invalid. The attacker makes several other suggestions that are completely invalid.
jrobbio wrote:I didn't understand their comments about sleeping better knowing that I am not worrying about the next way to break in. The fact they uploaded all the accounts makes me feel if they were feeling vindictive for some reason against PHPbb.
He means that he can sleep better knowing that he no longer has to worry about how to break in. This has apparently been high on his priority list for some time.
jrobbio wrote:I had a look at the site that the exploit came from and I spotted that PHPbbBook 1.3 and PNphpBB2 1.2i have recent exploits too.
I don't know what either of those are.
bolverk wrote:I am curious about something that perhaps the phpBB Management team can address at some point. Are you doing any sort of 'Lessons Learned' from this outage, other than the obvious "keep things patched" ;) Perhaps given the lengthy downtime involved in this one attack, setup a mirror site that can be brought online to keep certain things available like downloads, styles, mods, converters, packages other than the full install etc..
We are very open to learning from our mistakes and this is precisely why such an in-depth investigation is being performed. Something you should keep in mind, however, is that the initial attack was performed before any patch was available. It was down to us having to see the exploit posted before him and patching it quickly ourselves. We were not, unfortunately, able to do this.

As far as the downtime, it has nothing to do with not having a mirror site. The attacker had access to the server for a 2 week period. This means that we would either have to revert to a 2 week old backup (and lose 2 weeks of information in the process) or run the full investigation that we are running now. This site is available for support while the main board remains offline.

You should also note that this all happened in the process of us already making changes. The timing was very unfortunate, as this would not have been possible to achieve in a few weeks.
bolverk wrote:I understand that you are not a commercial entity and as such, have no responsibility to be available to the public, but perhaps some redundancy and disaster recovery planning might be in order.
That's not out mentality at all. In this case, there is little we can do in terms of damage control because private information has already been posted. This is why the community has been advised to change their passwords, even if there is minimal risk to their specific account data. You are very wrong if you believe that we do not consider ourselves responsible for anything.

The server has redundancy, but as per above, that has nothing to do with why things are taking this long. We have the choice of either reverting to an old "safe" backup or sanitising gigabytes of information.

I hope that clears things up a bit. Please feel free to ask follow-up questions.

panhead
Registered User
Posts: 3
Joined: Fri Jan 14, 2005 8:52 pm

Re: [Discussion] Downtime and Server Compromise

Post by panhead »

I am also using the combination phpBB and phplist. Is updating phplist to the newest version enough to prevent this happening again or can/must I do more?

Thanks

phpbblove
Registered User
Posts: 1
Joined: Wed Feb 04, 2009 12:53 pm

Re: [Discussion] Downtime and Server Compromise

Post by phpbblove »

Its very unlucky that it was a zero day exploit, goodluck phpbb.com team with all the things you all must do,

one question though, what are you going to do with the hacker? Try to put him in jail?

Marshalrusty
Project Manager
Project Manager
Posts: 273
Joined: Thu Oct 27, 2005 1:45 am

Re: [Discussion] Downtime and Server Compromise

Post by Marshalrusty »

panhead wrote:I am also using the combination phpBB and phplist. Is updating phplist to the newest version enough to prevent this happening again or can/must I do more?
Updating to the latest version of PHPList fixes this vulnerability, yes.
phpbblove wrote:Its very unlucky that it was a zero day exploit, goodluck phpbb.com team with all the things you all must do,
Thanks, we're trying :)
phpbblove wrote:one question though, what are you going to do with the hacker? Try to put him in jail?
The attacker was able to cover his tracks very well.

panhead
Registered User
Posts: 3
Joined: Fri Jan 14, 2005 8:52 pm

Re: [Discussion] Downtime and Server Compromise

Post by panhead »

Thank you and good luck!

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: [Discussion] Downtime and Server Compromise

Post by EXreaction »

Marshalrusty wrote:As has been said earlier in this topic, it is not possible to update the hash. Hashing is a one way process, so you cannot get the plaintext value to rehash it in the new format (at least not consistently enough to create a script for all cases). There are, however, some ideas on how similar affects can be achieved. They are currently being discussed. Anyone on your board with more than regular permissions should have logged in on phpBB3. If they have not done so, then you should just change their password manually from the ACP.
There is the option of using rainbow tables or something to that affect to attempt to figure out what the users passwords were from the md5 hash. Then use that to make a new hash. If the md5 reverse can not be found in the rainbow table then just don't update it.

But I suppose that's all useless since the attacker already did that and would know the passwords.

The only thing I can see that would be acceptable would be to put some sort of lock or deactivate all accounts with the old style passwords and require them to activate their account through the email account they originally listed. If they've not logged on since phpbb.com changed to phpBB3 I doubt they will be logging on again any time soon, if ever anyways.

Post Reply