[RFC] Support for IDN (IRI)

Note: We are moving the topics of this forum and it will be deleted at some point

Publish your own request for comments/change or patches for the next version of phpBB. Discuss the contributions and proposals of others. Upcoming releases are 3.2/Rhea and 3.3.
User avatar
rxu
Registered User
Posts: 124
Joined: Tue Apr 04, 2006 4:28 pm
Contact:

[RFC] Support for IDN (IRI)

Post by rxu » Sun Mar 04, 2012 4:07 am

I didn't find any similar RFC, but just a discussion topic of Älphäbet url support for 4.0.
Since October, 2009 ICANN has approved the creation of internationalized country code top-level domains (IDN ccTLDs), and first of then were installed in the DNS root zone on May, 2010. Now IDNs are actually in use already.

Thus, support for IDN (which is a part of IRI - International resource identifier) seems to be a mandatory for any web application, including bulletin board.
It would be nice to implement it as soon as possible to make it included into 3.1a.

Some useful links:
Internationalized domain name - Wikipedia
Internationalized Resource Identifiers (IRIs) - W3
rfc5891 - Internationalized Domain Names in Applications (IDNA): Protocol
rfc5892 - The Unicode Code Points and Internationalized Domain Names for Applications (IDNA)

EDIT: The derivation of valid characters in terms of Unicode properties for IDNA2008 - Internationalized Domain Names (IDN) FAQ

EDIT2:
Unicode Technical Standard #46. Unicode IDNA Compatibility Processing
Unicode Character Database (txt file)

EDIT3:
Ticket: https://tracker.phpbb.com/browse/PHPBB3-12926
Pull request: https://github.com/phpbb/phpbb/pull/3189
Last edited by rxu on Thu Jan 22, 2015 3:29 pm, edited 3 times in total.
Image

Oleg
Posts: 1150
Joined: Tue Feb 23, 2010 2:38 am
Contact:

Re: [RFC] Support for IDN (IRI)

Post by Oleg » Wed Mar 07, 2012 1:00 pm

We should support IDN I suppose.

ecwpa
Registered User
Posts: 181
Joined: Mon Jan 24, 2005 2:10 am
Contact:

Re: [RFC] Support for IDN (IRI)

Post by ecwpa » Wed Mar 07, 2012 1:41 pm

+1

It was weird from the begining not being able to use iconic characters like Ñ in urls for languages like Spanish, the name itself, España wasn't even posible within urls. Now that this finally changed phpBB should suppport it.
Slightly better English than it was in 2005, still improving :D

User avatar
imkingdavid
Registered User
Posts: 1050
Joined: Thu Jul 30, 2009 12:06 pm

Re: [RFC] Support for IDN (IRI)

Post by imkingdavid » Wed Mar 07, 2012 3:50 pm

+1 from me as well
I do custom MODs. PM for a quote!
View My: MODs | Portfolio
Please do NOT contact for support via PM or email.
Remember, the enemy's gate is down.

User avatar
bantu
3.0 Release Manager
3.0 Release Manager
Posts: 557
Joined: Thu Sep 07, 2006 11:22 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC] Support for IDN (IRI)

Post by bantu » Wed Mar 07, 2012 5:02 pm

Oleg wrote:We should support IDN I suppose.
+1

User avatar
rxu
Registered User
Posts: 124
Joined: Tue Apr 04, 2006 4:28 pm
Contact:

Re: [RFC] Support for IDN (IRI)

Post by rxu » Sun Sep 16, 2012 2:38 pm

Hmm. Valid IDN characters should be NFKC Case folded, while phpBB3 messages are normalized by NFC.
So we 'd need to add NFKC casefolded message content to the parser to process IDNs, unless I've missed the way of derivation of IDN valid characters.
Image

User avatar
rxu
Registered User
Posts: 124
Joined: Tue Apr 04, 2006 4:28 pm
Contact:

Re: [RFC] Support for IDN (IRI)

Post by rxu » Fri Jun 14, 2013 2:19 pm

It looks like IDN can be checked for validity pretty easy, using function idn_to_ascii() with the IDNA_USE_STD3_RULES flag. For example:

Code: Select all

idn_to_ascii('täst.de', IDNA_USE_STD3_RULES);
This code returns domain name in its IDNA ASCII form.
But this code:

Code: Select all

idn_to_ascii('-täst.de', IDNA_USE_STD3_RULES);
returns false since the leading hyphen is not allowed by domain name ASCII rules.

So, when false, IDN is invalid, and otherwise it's valid. This should simplify the whole implementeation, but requires support of Internationalization Functions module by PHP (intl).
Image

User avatar
spello
Registered User
Posts: 26
Joined: Fri Aug 31, 2012 12:13 pm
Contact:

Re: [RFC] Support for IDN (IRI)

Post by spello » Sun Feb 23, 2014 8:44 pm

+1

Wrongly displayed IDN URLs are really annoying.

User avatar
bantu
3.0 Release Manager
3.0 Release Manager
Posts: 557
Joined: Thu Sep 07, 2006 11:22 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC] Support for IDN (IRI)

Post by bantu » Sun Feb 23, 2014 11:33 pm

There are security implications that must be considered when implementing IDN support. See http://en.wikipedia.org/wiki/IDN_homograph_attack

User avatar
EXreaction
Registered User
Posts: 1555
Joined: Sat Sep 10, 2005 2:15 am

Re: [RFC] Support for IDN (IRI)

Post by EXreaction » Mon Feb 24, 2014 3:08 am

That issue seems entirely irrelevant to phpBB, it's up to browsers to handle IDN properly. It's not like us not supporting it is going to prevent someone from trying these attacks against people, even through phpBB.

Code: Select all

[url=myASCIIDomainNameWhichRedirectsToPayPalLookAlike]http://www.paypal.com[/url]
I'd bet that > 90% of people would be fooled simply by doing that (people look where the URL of the current page is more often than where a link is pointing to and after they click it's too late).

Post Reply