Re-thinking read/unread features

[kaiden11]

Hello! New here... I've tried to find if there are more appropriate/active places for me to start this discussion, so please forgive me if I'm not quite in the right place. The 4.0 board seems like a place for fairly specific project planning, I was looking more for just discussion around features / philosophies, or whether people have seen this before elsewhere.

About a year ago now, I figured out and implemented an "alternate" method for tracking read/unread information for forum posts. It is a form of run-length encoding/compression (or so I'm told), that is able to efficiently store per-post seen/unseen information while storing no more seen/unseen records than half of the number of total posts in the database per user (and usually much less). It works by maintaining "seen boundaries," where a record lists a lower and upper boundary of posts that have been seen. Upon page-load, a subroutine maintains this list of boundaries for the user, either creating new boundaries, growing old ones, or merging two boundaries. It works for data sets that have monotonically increasing integers as their primary keys (like phpBB).

You can read about it here: http://stackoverflow.com/questions/2288 ... 27#5045827

I say that phpBB "cheats" a little in the above link because they only store timestamps in their *_track tables rather than per-post information. However, as I understand it, with phpBB being a decidedly flat forum, and people very rarely reading out of order, and are usually reading through pages of posts, not single posts. More than per-forum or per-topic timestamps isn't likely to be needed.

In the forum that I designed this for, posts were threaded, and given some of the topics, reading out of order was entirely possible, if not encouraged. Therefore, per-post historical accuracy was important. In the year since I came up with my initial design, I've improved upon it, adding hooks to provide nice display features outside of the seen/unseen maintenance routine ("Which posts in this thread had I not read before accessing this page?"), and also to efficiently support the maintenance of multiple messages viewed at once.

Based on what I've seen while briefly looking around here, I know the read/unread feature comes up a lot. I appreciate phpBB's current simplicity, but given some of the use cases ("Mark as unread", reading backwards, sub-forums, sub-sub-forums, etc.), it seems... unwieldy? This is a common problem, and though my solution may not be absolutely perfect, I would think this would have been a better solved problem by now.

Has anyone seen this type of implementation before? If so, were there problems? Or, do you have particular issues with, or questions about my current implementation? And are you tired of this topic coming up?

Thanks for reading.

[kaiden11]

Additionally, if this topic is misplaced, I would appreciate it greatly if somebody could indicate where I should take this discussion. Thanks in advance.

[imkingdavid]

No, you're in the right place. It can just take a little while for people to respond around here somethimes.

Personally, I don't know much about how the read/unread tracking is done so I really can't make an intelligent comment on it at the moment. Perhaps later when I've had a chance to take a look at what is currently done and what you've suggested I can respond better.

[AmigoJack]

More than per-forum or per-topic timestamps isn't likely to be needed

I think the post-wise approach comes from the very fact that ordering the posts by time is not mandatory. Storing the read/unread info per time would then make it mandatory. Also think of two topics being merged, which interleaves source and target posts.

particular issues with, or questions about my current implementation

Such a RLE needs more CPU cycles, of course, which also results in a longer time each PHP script needs to run. One has to put up benchmarks on what is more efficient: directly INSERTing SQL datasets or a combination of data normalization and fewer SQL commands...

[kaiden11]

I think the post-wise approach comes from the very fact that ordering the posts by time is not mandatory. Storing the read/unread info per time would then make it mandatory. Also think of two topics being merged, which interleaves source and target posts.

I think I see what you mean, and I agree. Because phpBB will only ever order posts by timestamp, it can make assumptions on what it needs to store to determine read/unread status.

The issue I come across when dealing with forums that make the same assumption is this: When I click on the latest post in a thread I'm not familiar with (say, if you're trying to see what is being immediately discussed), the software makes the assumption that "you have seen this far in the thread, so you must have already seen the rest." That may not be true, but most times I don't care about the rest of the thread. But say if you're doing collaborative storytelling, and you want the forum software to "keep your place" by way of the read/unread feature. Ideally, the software would be able to track whether you've seen each element of the story, and not make you suffer if you happened to "read ahead." That you're forced to use the separate bookmarking mechanism (and have to do so manually), because of that initial assumption is frustrating to me as a user.

Such a RLE needs more CPU cycles, of course, which also results in a longer time each PHP script needs to run. One has to put up benchmarks on what is more efficient: directly INSERTing SQL datasets or a combination of data normalization and fewer SQL commands...

That's valid. I need to insert some debugging hooks and determine what portion of the rendering process is spent on this. I'm not sure what percentage would be considered "acceptable," but, eh, knowing is half the battle. For a given page load, I've got a minimum of 1 SELECT (to return the current boundaries for a user), and either a) 1 INSERT, b) 1 UPDATE, or c) 1 UPDATE and 1 DELETE, with the PHP side of things determining the minimal number of actions to perform. For the phpBB method, you've got the same 1 SELECT, and either a) 1 UPDATE, or b) INSERT. However, for the phpBB method, are you having to select and compare the different forum/subforum/topic levels?

[A_Jelly_Doughnut]

Has anyone seen this type of implementation before? If so, were there problems? Or, do you have particular issues with, or questions about my current implementation? And are you tired of this topic coming up?

I know UBB.Threads v. 6 and another threaded forum system I used a decade ago (and have now forgotten) had per-post read tracking. I don't know if the system used a data structure like yours, although it seems like a reasonable algorithm. In UBB.Threads, you could view in threaded or flat mode. In flat mode when you loaded a page, you marked all the posts on that page read. In threaded mode, posts were only marked read as you read each individual one.

Now, as far as this being a common discussion ... I believe you're the first person in 9 years I've seen post about such a system for phpBB. Yay for originality

[kaiden11]

I know UBB.Threads v. 6 and another threaded forum system I used a decade ago (and have now forgotten) had per-post read tracking. I don't know if the system used a data structure like yours, although it seems like a reasonable algorithm. In UBB.Threads, you could view in threaded or flat mode. In flat mode when you loaded a page, you marked all the posts on that page read. In threaded mode, posts were only marked read as you read each individual one.

(nods) I've got threaded "classic" and threaded "full." Classic view "collapses" the thread such that only the selected post is fully rendered, but the rest are in a truncated tree below it (with indicators as to position, read/unread, etc.). In full mode, it fully renders every post in the thread in the appropriate indented tree position. Classic mode marks posts as read 1 post at a time, while full mode marks all posts in the thread as read (with read/unread indicators representing which ones you hadn't read just before page load).

Now, as far as this being a common discussion ... I believe you're the first person in 9 years I've seen post about such a system for phpBB. Yay for originality

Ha, with all of the questions about this on StackOverflow from would-be forum coders, I would have figured this would have been brought up more often. In your 9 years, has this just been a "solved" problem that people didn't care to improve upon? Or are you saying it's the first time you've seen this "boundary" solution applied to this problem?

Either way, I'll have to research into how UBB.Threads does its work.

[A_Jelly_Doughnut]

Has this just been a "solved" problem that people didn't care to improve upon?

I assume this is the reason it hasn't been brought up. Could be we're just lacking creativity.