PHPBB3-11595 - API

Discuss requests for comments/changes posted in the Issue Tracker for the development of phpBB. Upcoming releases are 3.2/Rhea and 3.3.
Post Reply
User avatar
MichaelC
Development Team
Development Team
Posts: 889
Joined: Thu Jan 28, 2010 6:29 pm

Re: [RFC] API

Post by MichaelC » Sat Nov 03, 2012 12:55 pm

Merged three very similar API RFCs.
Formerly known as Unknown Bliss
psoTFX wrote: I went with Olympus because as I said to the teams ... "It's been one hell of a hill to climb"
No unsolicited PMs please except for quotes.

User avatar
callumacrae
Infrastructure Team
Infrastructure Team
Posts: 1046
Joined: Tue Apr 27, 2010 9:37 am
Location: England
Contact:

Re: [RFC] API

Post by callumacrae » Sat Nov 03, 2012 3:26 pm

MichaelC wrote:Merged three very similar API RFCs.
Please could you unmerge the URL structure one? I was aware of this topic, but didn't want it to be as cluttered as the AJAX topic.
Made by developers, for developers!
My blog

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: [RFC] API

Post by EXreaction » Sat Nov 03, 2012 5:56 pm

MichaelC wrote:The world of websites just serving up templated HTML files is over. Now many different types of content need to be served.

Currently, to be able to get data from a forum on another site you either need to develop your own API or connect to the database.

DB Remote Connection - The most practical of the options would be remotely connecting to a DB, but this means that you have to write your own code to submit posts etc, which can cause issues when they are not running on the same version and/or mods are added. You also have the problem that this is fine when the site accessing is allowed DB credentials, but what if it is an end-users device connecting to your site? You don't want to give out DB access.

So, that leaves us with users developing their own API. The best format as it stands, formatting wise to output is in JSON. I propose we build in this functionality to phpBB.

There would be a json.php (or api.php) file in the root of your installation. It would be able to take many parameters such as json.php?view=forum&id=32&start=30&show=15. It would accept both put (requiring auth of course) and pull requests. You'd be able to take all the posts of a forum, or just a certian number (the above would show the topics 30 - 45 in forum 32).

This API can be used for mobile & tablet app developers. Full desktop application developers for Windows, Mac & Linux. It could also be used elsewhere on the sites to show the latest SITEXYZ announcements (I know many sites show phpBB's), or could show the stats of every forum in a network of forums distributed across forums. The possibilities are endless. Currently RSS is used for this purpose but ultimatly that isn't its main purpose and it only allows the pulling of data and not the pushing of data.

Thoughts?
This would be very nice to have, but I do not think JSON access should not be separated from the rest of the system. Doing that would lead to significant code duplication.

User avatar
MichaelC
Development Team
Development Team
Posts: 889
Joined: Thu Jan 28, 2010 6:29 pm

Re: [RFC] API

Post by MichaelC » Sat Nov 03, 2012 11:16 pm

Because of the way things are structured it makes things very hard to do that. Its cleaner to just implement it separately and then call appropriate methods (any code that needs copying where other bits doesn't I'll split into a new function).

Drupal had the same problem (talk mentions it here) but they are rewriting it to use a front controller. If we could move everything to a front controller you could just have:

Code: Select all

viewtopic:
  pattern:  /topic/{id}.{_format}
  defaults: { _controller: content_controller:topic_controller, _format: php }
  requirements:
      _format:  php|json
Then yourdomain.com/phpBB/topic/34 (optional .php ending) would retrieve a normal topic while yourdomain.com/phpBB/topic/34.json would return json, it all going to the same controller would make the code maintainable and no repeating.

Ultimately funneling the api to this file which then re-uses functions/methods is the easiest way for now unless we decide to break a lot of BC.
Formerly known as Unknown Bliss
psoTFX wrote: I went with Olympus because as I said to the teams ... "It's been one hell of a hill to climb"
No unsolicited PMs please except for quotes.

User avatar
callumacrae
Infrastructure Team
Infrastructure Team
Posts: 1046
Joined: Tue Apr 27, 2010 9:37 am
Location: England
Contact:

Re: [RFC] API

Post by callumacrae » Mon Nov 12, 2012 2:31 pm

I started work on this: https://github.com/phpbb/phpbb3/pull/1079

It's very WIP, and I'll probably end up rewriting it a few times.
Made by developers, for developers!
My blog

User avatar
keith10456
Registered User
Posts: 523
Joined: Sat Apr 22, 2006 10:29 pm
Contact:

Re: [RFC] API

Post by keith10456 » Tue Nov 20, 2012 8:31 pm

callumacrae wrote:I started work on this: https://github.com/phpbb/phpbb3/pull/1079

It's very WIP, and I'll probably end up rewriting it a few times.
Very excited about this :)

User avatar
MichaelC
Development Team
Development Team
Posts: 889
Joined: Thu Jan 28, 2010 6:29 pm

Re: [RFC] API

Post by MichaelC » Tue Nov 20, 2012 11:30 pm

keith10456 wrote:
callumacrae wrote:I started work on this: https://github.com/phpbb/phpbb3/pull/1079

It's very WIP, and I'll probably end up rewriting it a few times.
Very excited about this :)
Me and imkingdavid had both already started on it together and are quite a bit further along (plus most of that will need re-doing for controllers but the one we are working on is already using controllers). I'll make a WIP PR at some point.

I did tell Callum this but he still wanted to do that himself.
Formerly known as Unknown Bliss
psoTFX wrote: I went with Olympus because as I said to the teams ... "It's been one hell of a hill to climb"
No unsolicited PMs please except for quotes.

User avatar
callumacrae
Infrastructure Team
Infrastructure Team
Posts: 1046
Joined: Tue Apr 27, 2010 9:37 am
Location: England
Contact:

Re: [RFC] API

Post by callumacrae » Wed Nov 21, 2012 10:09 am

Oh, I thought you were talking about the controller >_<
Made by developers, for developers!
My blog

User avatar
galaxyAbstractor
Google Summer of Code Student
Posts: 17
Joined: Sun Sep 25, 2011 7:40 pm

Re: [RFC] API

Post by galaxyAbstractor » Sun Jun 09, 2013 2:54 pm

I've started looking into this and is planning and drawing up how it should work etc etc. Right now I'm testing around a bit, making a prototype to retrieve a list of all forums to get the idea of stuff.

Forums can have sub-forums and so on. I've been thinking if we should offer the result as a simple, linear array or a recursive array. Examples:

Linear example:

Code: Select all

{

    "status":"success",
    "response":[
        {
            "forum_id":"1",
            "parent_id":"0",
            "forum_name":"Your first category",
            "forum_desc":"",
            "forum_type":"0",
            "forum_posts":"1",
            "forum_topics":"1",
            "forum_last_post_id":"1",
            "forum_last_poster_id":"2",
            "forum_last_post_subject":"",
            "forum_last_post_time":"1370536389",
            "forum_last_poster_name":"galaxyAbstractor",
            "forum_last_poster_colour":"AA0000"
        },
        {
            "forum_id":"2",
            "parent_id":"1",
            "forum_name":"Your first forum",
            "forum_desc":"Description of your first forum.",
            "forum_type":"1",
            "forum_posts":"1",
            "forum_topics":"1",
            "forum_last_post_id":"1",
            "forum_last_poster_id":"2",
            "forum_last_post_subject":"Welcome to phpBB3",
            "forum_last_post_time":"1370536389",
            "forum_last_poster_name":"galaxyAbstractor",
            "forum_last_poster_colour":"AA0000"
        },
        {
            "forum_id":"3",
            "parent_id":"2",
            "forum_name":"Your first subforum",
            "forum_desc":"",
            "forum_type":"1",
            "forum_posts":"0",
            "forum_topics":"0",
            "forum_last_post_id":"0",
            "forum_last_poster_id":"0",
            "forum_last_post_subject":"",
            "forum_last_post_time":"0",
            "forum_last_poster_name":"",
            "forum_last_poster_colour":""
        },
        {
            "forum_id":"4",
            "parent_id":"1",
            "forum_name":"Your second forum",
            "forum_desc":"",
            "forum_type":"1",
            "forum_posts":"0",
            "forum_topics":"0",
            "forum_last_post_id":"0",
            "forum_last_poster_id":"0",
            "forum_last_post_subject":"",
            "forum_last_post_time":"0",
            "forum_last_poster_name":"",
            "forum_last_poster_colour":""
        },
        {
            "forum_id":"7",
            "parent_id":"0",
            "forum_name":"Test",
            "forum_desc":"",
            "forum_type":"2",
            "forum_posts":"0",
            "forum_topics":"0",
            "forum_last_post_id":"0",
            "forum_last_poster_id":"0",
            "forum_last_post_subject":"",
            "forum_last_post_time":"0",
            "forum_last_poster_name":"",
            "forum_last_poster_colour":""
        }
    ]

}
Recursive example:

Code: Select all

{

    "status":"success",
    "response":[
        {
            "forum_id":"1",
            "parent_id":"0",
            "forum_name":"Your first category",
            "forum_desc":"",
            "forum_type":"0",
            "forum_posts":"1",
            "forum_topics":"1",
            "forum_last_post_id":"1",
            "forum_last_poster_id":"2",
            "forum_last_post_subject":"",
            "forum_last_post_time":"1370536389",
            "forum_last_poster_name":"galaxyAbstractor",
            "forum_last_poster_colour":"AA0000",
            "forum_link":"",
            "subforums":[
                {
                    "forum_id":"2",
                    "parent_id":"1",
                    "forum_name":"Your first forum",
                    "forum_desc":"Description of your first forum.",
                    "forum_type":"1",
                    "forum_posts":"1",
                    "forum_topics":"1",
                    "forum_last_post_id":"1",
                    "forum_last_poster_id":"2",
                    "forum_last_post_subject":"Welcome to phpBB3",
                    "forum_last_post_time":"1370536389",
                    "forum_last_poster_name":"galaxyAbstractor",
                    "forum_last_poster_colour":"AA0000",
                    "forum_link":"",
                    "subforums":[
                        {
                            "forum_id":"3",
                            "parent_id":"2",
                            "forum_name":"Your first subforum",
                            "forum_desc":"",
                            "forum_type":"1",
                            "forum_posts":"0",
                            "forum_topics":"0",
                            "forum_last_post_id":"0",
                            "forum_last_poster_id":"0",
                            "forum_last_post_subject":"",
                            "forum_last_post_time":"0",
                            "forum_last_poster_name":"",
                            "forum_last_poster_colour":"",
                            "forum_link":""
                        }
                    ]
                },
                {
                    "forum_id":"4",
                    "parent_id":"1",
                    "forum_name":"Your second forum",
                    "forum_desc":"",
                    "forum_type":"1",
                    "forum_posts":"0",
                    "forum_topics":"0",
                    "forum_last_post_id":"0",
                    "forum_last_poster_id":"0",
                    "forum_last_post_subject":"",
                    "forum_last_post_time":"0",
                    "forum_last_poster_name":"",
                    "forum_last_poster_colour":"",
                    "forum_link":""
                }
            ]
        },
        {
            "forum_id":"7",
            "parent_id":"0",
            "forum_name":"Test",
            "forum_desc":"",
            "forum_type":"2",
            "forum_posts":"0",
            "forum_topics":"0",
            "forum_last_post_id":"0",
            "forum_last_poster_id":"0",
            "forum_last_post_subject":"",
            "forum_last_post_time":"0",
            "forum_last_poster_name":"",
            "forum_last_poster_colour":"",
            "forum_link":"http://phpbb.com"
        }
    ]

}
In the first example, it's just a list of all forums. The developer has to use parent_id and forum_id to see what forum is a subforum to what. In the second, he would need to check the subforum node of each forum to get the subforum of a forum.

Alternatively, one could have a function that retrieves only the toplevel forums of a specified forum id. If one were to give forum id 1, the function would return all forums with the parent id 1. Downside of that would be if one wants a list of all forums as one would have to check for subforums for each id which could lead to many API calls. An upside would be that it wont return a massive response on boards with many forums.

Of course, one could offer both (or all 3) methods to list forums depending on what request the developer does against the API.

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: [RFC] API

Post by EXreaction » Sun Jun 09, 2013 10:01 pm

Generally, I do not think the amount of data transmitted is going to be all that large for a request like that, at least not within the limitations of phpBB itself, so I would say that the only option really needed for now is going to be the ability to select all forums (recursively) that are children of a certain forum id (0 would mean select all forums on the board).

If additional options are desired in the future, they can be added, but this should work for all cases (if the subforums are not needed, the info can just be ignored by whatever is processing the data).

Post Reply