PHPBB3-11595 - API

Discuss requests for comments/changes posted in the Issue Tracker for the development of phpBB. Current releases are 3.2/Rhea and 3.3/Proteus.
Post Reply
User avatar
MichaelC
Development Team
Development Team
Posts: 889
Joined: Thu Jan 28, 2010 6:29 pm

Re: [RFC] API

Post by MichaelC »

EXreaction wrote:Generally, I do not think the amount of data transmitted is going to be all that large for a request like that, at least not within the limitations of phpBB itself, so I would say that the only option really needed for now is going to be the ability to select all forums (recursively) that are children of a certain forum id (0 would mean select all forums on the board).

If additional options are desired in the future, they can be added, but this should work for all cases (if the subforums are not needed, the info can just be ignored by whatever is processing the data).
+1
Formerly known as Unknown Bliss
psoTFX wrote: I went with Olympus because as I said to the teams ... "It's been one hell of a hill to climb"
No unsolicited PMs please except for quotes.

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor »

So I want to talk about authentication for a bit.

The authentication system
Users should never be required to give their username and password to applications because that would simply just be unsecure for a lot of reasons (password reuse, don't know which app is abusing your password if you use multiple and so on). Therefore I would suggest using a token system instead.

Tokens
An app would be required to use a token to access a users account, this could be done in two ways:
  • The dropbox/oauth way - application requests a token from the forum, opens the forum in a webbrowser with the token in the URL, user logs in and accepts the app auth request, the app checks if the token is valid
Allow me to demonstrate how ShareX use the Dropbox API:
When configuring ShareX to use Dropbox, you start with this page: http://i.imgur.com/kg9F66Y.png (note that I'm already logged in here, but doesn't matter). The button "Open authorize page..." generates a token and opens the dropbox authpage: http://i.imgur.com/570Oghn.png. After you press allow here, you have to go back and press "Complete authorization" in ShareX to finish the process.

This process is quite straightforward and easy to use for the user. It requires you to be able to use a browser on whatever device you are using, or are able to visit the authorization page in a browser on another device by entering the address.
  • The user generates a token in the UCP and pastes that into the application.
The upside with this is that the user could enter the token themselves, but I doesn't really see any usecases with this, you have to use a browser anyway, you just don't have to retype an URL on another device.

How should tokens be generated?
Tokens need to be unique and it shouldn't be possible for an attacker to derive them from known data. A hashed combination of username and timestamp for example would be a bad idea, as an attacker could just hash the username together with timestamps until he finds the token. Tokens should probably be generated on the forum side too and given to the app upon request.

Security issues
We need to make sure it isn't possible to guess the token easily. One could probably make a script that sends a lot of request using various topics and see if they manage to get a working token. What countermeasures could be taken to prevent that, how big of a threat would it be? One thing you could do is rate limiting, but then you need to figure out an amount that wouldn't be in the way for the user. Say you are allowed to do 1000 requests an hour, then it would take some time to be able to guess a token unless you are really lucky (the possibility would probably always be there whichever method one choses). Another probably much better way is to check if the same IP tries to fetch data using several different tokens in a short period of time. The amount and time period must then be chosen in a way to not affect people sharing the same IP, for example at a college's student wifi.

OAuth
I've been looking into OAuth a bit but I am uncertain if it's possible to use. From my observation of for example the Twitter OAuth API you need to register your app with Twitter to get an App token which is then combined with the customer token. This works for developers if there is a single service they are working around, but in the case of phpBB that isn't the case. It wouldn't be possible for a developer to request app tokens from every forum and the API loses a bit of it's point if the apps developed are site-specific. I'm pretty new to OAuth and I don't know if OAuth works without this app token or not. Also using OAuth would add another dependency, maybe it's easier for us to create a token system ourselves? The pros with app tokens is that you could control access a bit further in case an app is malicious, but I don't see it easy to implement unless there would be some kind of central repository where you register the app that each phpBB installation gathers info from (which probably isn't possible).

Token management
There would be a page in the UCP where all active tokens the user has approved is listed with a name of the app the user specifies. From here the user could revoke access for the app, in case it isn't needed or the app turned out to be malicious.

Permissions
An admin could decide if a specific user or group has access to the API. On some boards with ads the forum owners might dislike the idea of having an API without ads and could therefore offer API access to members who donate (is this even a problem?)
---
I probably forgot something in here, but I want to know what you think about this and figure out the best way to implement this ^^

User avatar
imkingdavid
Registered User
Posts: 1050
Joined: Thu Jul 30, 2009 12:06 pm

Re: [RFC] API

Post by imkingdavid »

Regarding authentication, it would be a good idea to talk to Hardolaf because he is currently 1) refactoring the authentication system, and 2) implementing OAuth and potentially other external authentication options. You two need to talk so that you don't end up doing double the work and having throw it out because it's already been done.

-----
As for the rest of my post, it's about 2 am so I'm sorry if it's incoherent.

We need to look at restricting applications both at the board level and at the user level. For instance, if an application needs to access a private forum's topics, it could be given a token to authenticate it to the board. If the application wants to automatically create a new post/topic when a certain event occurs using the authenticated user as the author, the user needs to authorize it on the user level.

Ideally, applications would be restricted first at the board level, and only after given board-level approval, allow user-level restrictions. In other words, a user shouldn't be able to authenticate into an arbitrary application that creates 5 topics a minute. The board admin should be able to select applications that can be used by the board, and of those applications the user should be able to select which ones can perform user-level actions with his account.

----

Tokens should be fairly lengthy but perhaps variable in length so that guessing them is more difficult.
I do custom MODs. PM for a quote!
View My: MODs | Portfolio
Please do NOT contact for support via PM or email.
Remember, the enemy's gate is down.

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor »

imkingdavid wrote: The board admin should be able to select applications that can be used by the board, and of those applications the user should be able to select which ones can perform user-level actions with his account.
Wouldn't that require the application to be registered to the board in some kind of way?

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor »

To get closer into the details, how much security do we want? Do we want to make sure that the API key cannot be stolen when sending a request over HTTP? Should we just don't care much about it and let the user remove the API key if the user notices it's abused? I don't know how big problem it is with hackers targeting users of an API to get their API key over HTTP.

Currently the system works like this: The server generates two keys, an authentication key and a signing key. These are sent to the user and could be seen by an attacker if the board uses HTTP. The user saves these and sends them back by opening a browser for the user to authenticate those keys.

When making a request, the user would include the auth key, a serial number and the parameters. Then the user would take the request (say api/auth/verify/auth_key/serial) and hash it using HMAC-SHA256 and add it to the end of the request. The server does the same thing and compares the hashes. This way no one can forge requests unless they got their hands on the signing token. The serial is used to stop replay attacks, the server keeps track of the last serial used by the user and if the user tries sending the same or lower serial again it's rejected.

The other option would be to just have an auth key and send that with the request, but then the API would be unsecure if someone get their hands on the auth key at some point.

My biggest concern is how complex the API is with this system, and I feel like I'm trying to solve a non-issue which would only lead to a secure but hard-to-use API. Is HMAC-SHA256 available everywhere?

What's your opinion on this?

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: [RFC] API

Post by EXreaction »

To me this sounds pretty nice, good work.

As far as I can see, a user would first need to log in before they can request an auth key, so their password would have already been transmitted over the unsecured network (or their session can be hijacked and a new key generated). If this is not correct let me know, but if their password can be stolen, stealing the API key is a non-issue.

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor »

I made a simple API implementation so far, to be able to test API calls more easily. https://github.com/galaxyAbstractor/php ... d7f4#L3L-1 the hashing does make a bit extra code to implement but it turned out to not be very complicated after all.

Also, people have suggested to me to use OAuth for authentication instead of my own hashing implementation. What's the opinion on that? I can't really wrap my head around OAuth and how it works and will work if it's even possible to use. I've checked a couple of OAuth server libraries but they would probably have to be modified a lot to be of use for a possible phpBB integration. It might almost be easier to implement that yourself so you know it'll work with phpBB?

User avatar
imkingdavid
Registered User
Posts: 1050
Joined: Thu Jul 30, 2009 12:06 pm

Re: [RFC] API

Post by imkingdavid »

Note that Hardolaf is currently working on implementing Oauth into the phpBB authentication system and has gotten pretty far. I suggest you talk to him to see about somehow integrating it if possible.
I do custom MODs. PM for a quote!
View My: MODs | Portfolio
Please do NOT contact for support via PM or email.
Remember, the enemy's gate is down.

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor »

imkingdavid wrote:Note that Hardolaf is currently working on implementing Oauth into the phpBB authentication system and has gotten pretty far. I suggest you talk to him to see about somehow integrating it if possible.
I did during the hackathon but iirc he was a bit against OAuth or negative against it for the API if I understood him correctly. We discussed a bit about using Diffie-Hellman instead to establish a shared secret but that was too complex and hard to implement so after discussing a bit with bantu and naderman I did something simpler with the hashing that is currently implemented.

User avatar
naderman
Consultant
Posts: 1727
Joined: Sun Jan 11, 2004 2:11 am
Location: Berlin, Germany
Contact:

Re: [RFC] API

Post by naderman »

It does seem like most APIs of big services are switching to oauth for authentication. I think it's important to keep in mind what kind of uses the API will see. I can see it used by custom code by the owner of a site in which case something simple is nice. But I can also see it being used by mobile apps for example, in which case oauth would be the better option. So maybe having either depending on what the goal is may be the best option?

Post Reply