Älphäbet url support

General discussion of development ideas and the approaches taken in the 3.x branch of phpBB. The next feature release of phpBB 3 will be 3.2/Rhea followed by 3.3.
Forum rules
Please do not post support questions regarding installing, updating, or upgrading phpBB 3.1. If you need support for phpBB 3.1 please visit the 3.1.x Support Forum on phpbb.com.

If you have questions regarding writing extensions please post in Extension Writers Discussion to receive proper guidance from our staff and community.
User avatar
Lurttinen
Registered User
Posts: 78
Joined: Mon Feb 20, 2006 7:57 am
Contact:

Älphäbet url support

Post by Lurttinen » Tue Dec 29, 2009 4:28 pm

http://www.älypää.com <- phpBB3 fails in this area. These should work in phpBB4 :)
This is becoming a real problem when users are forced to use tinyurls or encode it manually.

Example;
http://www.xn--lyp-plada.com for http://www.älypää.com
or
http://en.wikipedia.org/wiki/Matti_Nyk%C3%A4nen for http://en.wikipedia.org/wiki/Matti_Nykänen

code reader
Registered User
Posts: 653
Joined: Wed Sep 21, 2005 3:01 pm

Re: Älphäbet url support

Post by code reader » Tue Dec 29, 2009 7:04 pm

this seems like a phpbb3 issue.
serious issue, no argument, but i don't think this is the right place to discuss it.
i think a bug report should be filled.
-- and btw: this looks like one of the cases where a phpbb2 patch would not be out of place also.

peace.

User avatar
DavidIQ
Customisations Team Leader
Customisations Team Leader
Posts: 1732
Joined: Thu Mar 02, 2006 4:29 pm
Location: Earth
Contact:

Re: Älphäbet url support

Post by DavidIQ » Tue Dec 29, 2009 7:15 pm

code reader wrote:-- and btw: this looks like one of the cases where a phpbb2 patch would not be out of place also.
The only reason a phpBB2 patch would be officially released is if there is some security vuln. found and this hardly qualifies...
Image

User avatar
ToonArmy
Registered User
Posts: 335
Joined: Fri Mar 26, 2004 7:31 pm
Location: Bristol, UK
Contact:

Re: Älphäbet url support

Post by ToonArmy » Tue Dec 29, 2009 7:54 pm

DavidIQ wrote:
code reader wrote:-- and btw: this looks like one of the cases where a phpbb2 patch would not be out of place also.
The only reason a phpBB2 patch would be officially released is if there is some security vuln. found and this hardly qualifies...
Not even that.

phpBB 3.0 doesn't support IDN URIs we might add support in 3.x but 4.0 certainly will.
Chris SmithBlogXMOOhlohArea51WikiNo support via PM/IM
Image

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: Älphäbet url support

Post by EXreaction » Tue Dec 29, 2009 8:06 pm

Lurttinen wrote:http://www.älypää.com <- phpBB3 fails in this area. These should work in phpBB4 :)
This is becoming a real problem when users are forced to use tinyurls or encode it manually.

Example;
http://www.xn--lyp-plada.com for http://www.älypää.com
or
http://en.wikipedia.org/wiki/Matti_Nyk%C3%A4nen for http://en.wikipedia.org/wiki/Matti_Nykänen
If you are interested in trying some new regex out, I generated some that seems to work.

Open includes/functions.php

Find the get_preg_expression() function

In it there are returns in every case, for the url, www_url, and relative_url, comment out the returns and put these in their place:

Code: Select all

return "[a-z][a-z\d+\-.]*:/{2}(?:(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})+|[0-9.]+|\[[a-z0-9.]+:[a-z0-9.]+:[a-z0-9.:]+\])(?::\d*)?(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";

return "www\.(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})+(?::\d*)?(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";

return "(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";
I've not extensively tested it however, but didn't notice any problems (can't think of any security issues either).

User avatar
Lurttinen
Registered User
Posts: 78
Joined: Mon Feb 20, 2006 7:57 am
Contact:

Re: Älphäbet url support

Post by Lurttinen » Tue Dec 29, 2009 10:47 pm

code reader wrote: i think a bug report should be filled.
Did you read my post and pay any attention to the "Not a bug" i linked to? ;)
ToonArmy wrote: phpBB 3.0 doesn't support IDN URIs we might add support in 3.x but 4.0 certainly will.
Glad to hear that. :)

sajaki
Registered User
Posts: 86
Joined: Mon Jun 21, 2010 8:28 pm

Re: Älphäbet url support

Post by sajaki » Mon Dec 06, 2010 10:37 pm

hi,

Is this feature still deferred to 4.0 or will it already be included in 3.2 or maybe even in 3.1 (3.1 i think not, haven't seen an rfc for it) ?

http://tracker.phpbb.com/browse/PHPBB3-9779
http://tracker.phpbb.com/browse/PHPBB3-3981

Given that http://www.icann.org/en/topics/idn/fast-track/, more and more domains will appear with non latin domain names, giving this problem increasing importance.

this rfc http://www.ietf.org/rfc/rfc1738.txt is the old.
this is the new : http://www.faqs.org/rfcs/rfc3490.html

ps frankly i think this evolution may well wreck the web as we know it.

intd
Registered User
Posts: 1
Joined: Mon Jan 30, 2012 5:48 am

Re: Älphäbet url support

Post by intd » Mon Jan 30, 2012 5:49 am

The code posted above does not work with Cyrillic urls with uppercase, such as http://ru.wikipedia.org/wiki/Русский_язык

Stoker
Registered User
Posts: 39
Joined: Sat Jun 20, 2009 8:15 pm

Re: Älphäbet url support

Post by Stoker » Sat Aug 18, 2012 3:53 pm

EXreaction wrote:
Lurttinen wrote:http://www.älypää.com <- phpBB3 fails in this area. These should work in phpBB4 :)
This is becoming a real problem when users are forced to use tinyurls or encode it manually.

Example;
http://www.xn--lyp-plada.com for http://www.älypää.com
or
http://en.wikipedia.org/wiki/Matti_Nyk%C3%A4nen for http://en.wikipedia.org/wiki/Matti_Nykänen
If you are interested in trying some new regex out, I generated some that seems to work.

Open includes/functions.php

Find the get_preg_expression() function

In it there are returns in every case, for the url, www_url, and relative_url, comment out the returns and put these in their place:

Code: Select all

return "[a-z][a-z\d+\-.]*:/{2}(?:(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})+|[0-9.]+|\[[a-z0-9.]+:[a-z0-9.]+:[a-z0-9.:]+\])(?::\d*)?(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";

return "www\.(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})+(?::\d*)?(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";

return "(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*(?:/(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@|]+|%[\dA-F]{2})*)*(?:\?(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?(?:\#(?:[\x80-\xFFa-z0-9\-._~!$&'()*+,;=:@/?|]+|%[\dA-F]{2})*)?";
I've not extensively tested it however, but didn't notice any problems (can't think of any security issues either).
Using 3.0.10 the above edit produces this error:

Code: Select all

General error
SQL ERROR [ mysql4 ]

Incorrect string value: '\xA5l-65....' for column 'post_text' at row 1 [1366]
However this did work in 3.0.9 :?:

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: Älphäbet url support

Post by EXreaction » Sat Aug 18, 2012 8:34 pm

If it worked previously, but not in 3.0.10, I would check the install/database_update.php script to see what happened. Seems to me like that could be related to some sort of schema change on the post_text column.

Post Reply