[RFC] validate topic view count using javascript

[xsawyerx]

A major problem that I have with a phpBB forum is that bots, spammers, crawlers, and other scripts keep bumping the view count. For a board I just started I already have a view counts of a few hundreds. It makes no sense. When a script that tries to harvest emails, post spam, etc. identifies its useragent as a standard form one (they obviously won't say "MyScriptCrawler/1.0") or accesses using various legitimate useragent strings, they aren't caught by the bots list and are always able to up the view count, making it not just completely pointless but affecting the website horribly. "These guys have just 30 posts across 2 topics and 9453 views?!"

I've searched online and found a lot of people with the same problem. The solution always misses the point: protect the forum with a password (but I want it to be indexed!), create an extra field to authenticate users (but I want it to be simple for everyone!), etc.

Instead of blocking or restricting the forum, we can simply check which users are able to run javascript, and if someone is able to do so, we should only THEN increment the view count. Although I'm suggesting this for topic view counts (cuz seriously, guys, it's freaking killing me!), this could be used for a range of other issues that involve the question "is this a real person behind the keyboard".

Meanwhile I'm forced to analyze all my web server logs, find any IP that comes from multiple useragents (10 or more) and classify them as spammers and add them to the bots list so they don't influence the view count.

[DavidIQ]

Ok...and dismiss the counts for those that have JavaScript disabled on purpose?

[Oleg]

Also, "view" means someone requested the page. It may have been a spammer, the action of requesting the page still counts as a view. You may want a count of "views by people with certain intentions" but generally speaking it is a different metric from just "views".

[xsawyerx]

The problem is two-fold:
1. The "views" is not necessarily helpful since it's not people actually "viewing" the website, but rather "fetching" the content. The fact someone fetched the content does not mean they actually viewed it. Perhaps they were scanning it for something, indexing the content, looking for emails, whatever the purpose, whether for good or bad.
2. When you have a starting board and only a few posts, having 25K in "views" while only 10 posts gives a bad impression of the board since people think of "views" as... well, someone *viewing the website*. This makes the view count actually damaging for some, and I've found others who had this exact problem.

Suppose counting only javascripts is not an adequate solution, how would you suggest solving this instead?
(I do realize this isn't a support forum, I'm just trying to see if there's very obvious I'm missing)

I've noticed most forums simply do not allow viewing the content unless you're registered and logged in, while others have removed the count completely. Am I missing something else that is super obvious?

I appreciate the patience and helpfulness.
S.

[brunoais]

If you want to give a forward with this I can only say:
Make a MOD for this. This is better used as a MOD than as a core feature.

[Oyabun1]

affecting the website horribly

This makes the view count actually damaging for some

I'm just not seeing why that would be so. Damaging how? Are you suggesting that a guest to your board will see a topic as having, say, 10K views and will then think "oh, that's too popular so I'm not looking at that", rather than, "10K views already - must be worth a look"?

[ecwpa]

I'm just not seeing why that would be so. Damaging how? Are you suggesting that a guest to your board will see a topic as having, say, 10K views and will then think "oh, that's too popular so I'm not looking at that", rather than, "10K views already - must be worth a look"?

From the author's perspective I can see how actual views could be useful. In the case you use a forum for lets say, tutorials, reviews, or opinions from a certain topic you might use that metric to know if its worth your time to write more about a popular topic as as reviews or a burning issue with tutorials.

I used to have a news site using phpBB as a backend and I had to add a query to +1 topics for impressions made outside the actual forum just to have an idea of how many people where reading our articles.


Edit: I found a bit weird that we could see there were people reading the news script outside phpBB by looking at the online users stats on the forums. But that wasn't reflected on topic views. I'm totally off topic, sorry.

[*Daniel]

Well you can stop the bots and such from raising the post counts by adding the bots into your database. There a script that add a whole pile of bots into the database and then you just need to edit viewtopic to check to see if it a bot getting that page.

You can also use this to stop say the topic author from raising the view count on that topic as well.

http://www.phpbb.com/customise/db/mod/p ... by_author/

[*Daniel]

Anyone forgot about this?

[Oleg]

I would say a view by a bot still counts as a view.