Duplicate keys in $lang array

Discussion of general topics related to the new version and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Forum rules
Discussion of general topics related to the new release and its place in the world. Don't discuss new features, report bugs, ask for support, et cetera. Don't use this to spam for other boards or attack those boards!
Post Reply
code reader
Registered User
Posts: 653
Joined: Wed Sep 21, 2005 3:01 pm

Duplicate keys in $lang array

Post by code reader »

hi all.
i know we are not supposed to report bugs.
however, i wanted to add a small contribution to the development cycle.
as you know, the associative array $lang is built in several files now, using the construct
$lang += array(
'KEY' => 'VALUE',
...
);
it is very possible to have the same KEY used, unwittingly, in several files.
i tried to detect it, and, lo and behold, there are quite a few (well, 12, as of now) duplicates.
as a service to the community, i am posting here both the little script that detects the duplicates, and the results:

the script:

Code: Select all

<?php
$modules = array('help_faq', 'mcp', 'search', 'ucp', 'common', 'groups', 'memberlist', 'viewforum', 'help_bbcode', 'posting', 'viewtopic');
$kkeys = array();
$keys = array();
$langdir = dirname(__FILE__);
foreach($modules as $module) {
	require("$langdir/$module.php");
	if (isset($lang)) {
		$kkeys[$module] = $lang;
		$keys[] = $module;
		unset($lang);
	}
}
$duplicates = array();
while ($module = array_shift($keys)) {
	$keys_1 = array_keys($kkeys[$module]);
	reset($keys);
	foreach ($keys as $other_module) {
		$keys_2 = array_keys($kkeys[$other_module]);
		foreach(array_intersect($keys_1, $keys_2) as $dup)
			$duplicates[] = "$dup, $module.php, $other_module.php";
	}
}
print_r($duplicates);
?>
the results:

Code: Select all

Array
(
    [0] => POST_DETAILS, mcp.php, common.php
    [1] => DELETE_POST_CONFIRM, mcp.php, posting.php
    [2] => FORK_TOPIC, mcp.php, viewtopic.php
    [3] => UNLOCK_TOPIC, mcp.php, viewtopic.php
    [4] => DELETE_ALL, ucp.php, common.php
    [5] => UNREAD_MESSAGES, ucp.php, common.php
    [6] => RECIPIENT, ucp.php, memberlist.php
    [7] => GROUP_LEADER, ucp.php, memberlist.php
    [8] => DELETE_MESSAGE, ucp.php, posting.php
    [9] => DELETE_MESSAGE_CONFIRM, ucp.php, posting.php
    [10] => MESSAGE_DELETED, ucp.php, posting.php
    [11] => DAYS, common.php, posting.php
)
comments:
  1. in order for this script to work "as is", it have to be placed in the language directory (eg, pbpBB2/language/en)
  2. it is possible to compute the modules list programatically. i didnt do it, for simplicity
  3. in addition, there are some minor problems with the $lang['datetime'] array:
    1. 'TODAY' and 'YESTERDAY' are set twice
    2. there is a 'short month names' and 'long month names' parts. unfortunately, the month of May has the same key for both. right now the developers removed may from the short month-names part, but, in languages where the name of the fifth month contains more than 3 letters, there will be a slight problem. i would suggest to use two 0..11 arrays, namely short_month_name and long_month_names, rather than the associative way index on the english names, as it is done now.
  4. in the future, a similar script should be written, to help authors of language-paks to detect missing translations
  5. unfortunately, i couldnt think of a script to detect multiple uses of the same KEY in the same file
i hope the mods will not take to my infraction of the "no bug reports" rule too harshly.
APTX
Registered User
Posts: 680
Joined: Thu Apr 24, 2003 12:07 pm

Re: Duplicate keys in $lang array

Post by APTX »

Let's call it a hint :mrgreen: .
Don't give me my freedom out of pity!
User avatar
DavidMJ
Registered User
Posts: 932
Joined: Thu Jun 16, 2005 1:14 am
Location: Great Neck, NY

Re: Duplicate keys in $lang array

Post by DavidMJ »

You do know that not all of those files will be included at once right? That being said, the keys might not have identical values. However, ones that do have identical values may be offloaded into common.php.
Freedom from fear
APTX
Registered User
Posts: 680
Joined: Thu Apr 24, 2003 12:07 pm

Re: Duplicate keys in $lang array

Post by APTX »

I had a little struggle with myself. I wanted to write something similar to "there is a logical explanation why it is that way in pre-alpha software..." but the hint thing won in the end.
Don't give me my freedom out of pity!
code reader
Registered User
Posts: 653
Joined: Wed Sep 21, 2005 3:01 pm

Re: Duplicate keys in $lang array

Post by code reader »

DavidMJ wrote: You do know that not all of those files will be included at once right? That being said, the keys might not have identical values. However, ones that do have identical values may be offloaded into common.php.
  • if you pay attention, you will notice that several of the duplicates are between "common" and another module
  • when the values differ intentionally, different keys should be used
  • the right thing to do, if you need a language literal thats defined in a language module that is not included, is to include this language module, not duplicate the key => value pair.
these issues are far from critical. the reason i think its a good idea to fix them now, rather than in the first, second or third beta cycle is because people want to get an early start on translation, and issues in the "$lang" area multiply with the languages.
as a side, i want to contribute the following little script, to help check that a "language pack" is complete.

Code: Select all

<?php
/* the following script tests a "language pack" and compares it to the english. it verifies that each module contains the same keys as the english module. 
the file should be placed in the "language" directory, above the 'en' ...
*/

$languages = array('fr', 'de', 'he');   // you should change this array to reflect the language(s) you want tested
$modules = array('help_faq', 'mcp', 'search', 'ucp', 'common', 'groups', 'memberlist', 'viewforum', 'help_bbcode', 'posting', 'viewtopic'); // this should be updated when more modules are to be presented.
				
function create_language_keys($language) {
	global $modules, $lang;
	$l_keys = array();
	$langdir = dirname(__FILE__) . "/$language";
	foreach($modules as $module) {
		require("$langdir/$module.php");
		if (isset($lang)) {
			$l_keys[$module] = array_keys($lang);
			unset($lang);
		}
	}
	return $l_keys;
}				

$missing = array();
$unnecessary = array();
$english_keys = create_language_keys('en');
foreach($languages as $language) {
	$language_keys = create_language_keys($language);
	$missing[$language] = array();
	$unnecessary[$language] = array();
	foreach(array_keys($english_keys) as $module) {
		$temp = array_diff($english_keys[$module], $language_keys[$module]);
		if (sizeof($temp) > 0)
			$missing[$language][$module] = $temp;
		$temp = array_diff($language_keys[$module], $english_keys[$module]);
		if (sizeof($temp) > 0)
			$unnecessary[$language][$module] = $temp;
	}
}
echo("<br />Missing language keys<br>\n");
print_r($missing);
echo("\n\n<br /><br /><br />Unnecessary language keys<br>\n");
print_r($unnecessary);
?>

User avatar
Acyd Burn
Posts: 1838
Joined: Tue Oct 08, 2002 5:18 pm
Location: Behind You
Contact:

Re: Duplicate keys in $lang array

Post by Acyd Burn »

No offence, but we already have similar scripts locally. ;)

Image
code reader
Registered User
Posts: 653
Joined: Wed Sep 21, 2005 3:01 pm

Re: Duplicate keys in $lang array

Post by code reader »

Acyd Burn wrote: No offence, but we already have similar scripts locally. ;)
i wont bother you any more, just for this once: may i request that these script(s) will be made public (i am talking about the "language pack verification"), as most of the translators are not part of the team?
thx.
User avatar
Acyd Burn
Posts: 1838
Joined: Tue Oct 08, 2002 5:18 pm
Location: Behind You
Contact:

Re: Duplicate keys in $lang array

Post by Acyd Burn »

We haven't opened the translation process yet, but once started we will give the translators some tools to check their language pack integrity (as is now the case for the 2.0.x language packs - the checking is done after uploading the language pack, it reports missing strings, dirs...).

At the moment it is recommended to not start translating (those who wish [and doing] are doing so on their own at the moment).

Image
Post Reply