Mantis Bugtracker          
testlink.org
Main | My View | View Issues | Change Log | My Account
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007959TestLinkLocalizationpublic2017-06-01 23:022017-07-13 19:55
ReporterMr.Bricodage 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version1.9.16 (2016 Q4) 
Fixed in Version1.9.17 (2017 Q2) 
Summary0007959: en_GB\strings.txt file cleanup
Description2565 variables are presents in locale\en_GB\strings.txt.

Some variables are declared twice, others are declared but not used anymore in testlink code.

To reduce the translation and maintenance effort, this file should be cleaned :
- duplicates entries should be unique (19 hits),
- not used entries should be deleted (424 hits)
Steps To Reproduce- For strings that are declared twice : search in en_GB\strings.txt the strings provided in additional Information
- For unused strings : search in testlink-code folder for any of the strings provided as attachment : no matches (except in locale folder)
Additional InformationHow I found strings that are present several times :
in locale\en_GB\
   # sort strings.txt | uniq -cd
      2 add_issue_note
      2 audit_security_user_right_missing
      2 caption_nav_filters
      2 issue_status_assigned
      2 issue_status_closed
      2 issue_status_confirmed
      2 issue_status_resolved
      2 related_to
      2 req_created
      2 requirement_document_id
      2 select_at_least_one_req
      2 tcase_doesnot_exist
      2 title_plugins
      2 unassigned
      2 warning_empty_pwd
      2 warning_enter_at_least1
      2 warning_enter_at_least2
      2 warning_enter_less1
      2 warning_enter_less2


How I found unused strings

0. Copy locale\en_GB\strings.txt from TestLink repository to work area
    # cp /drives/d/Mr.B/GitHub/testlink-code/lang/en_GB\strings.txt ~
1. Remove all lines that don't start with $TLS_ (comments, header, blank lines, translations done on severals lines, ...)
    # sed -i '/^\$TLS_/!d' strings.txt
2. Remove all translations (all data after equals)
        # sed -i 's:=.*$::g' strings.txt
3. Remove $TLS_ prefix (5 first characters of each line)
    # sed -i 's/^....//' strings.txt
4. Remove residual spaces and tabs at the end of each line
    # sed -i 's/[[:blank:]]*$//' strings.txt
4. Remove short text values (7 characters or less) that can be identified easily in an automatic way
    # sed -i '/........./!d' strings.txt
5. Carriage return management
    If CRLF, \r\n replaced by \n by any text editor
6. sort strings
    # sort strings.txt -o strings.txt
      
7. Search for each string present in the string.txt file in the TestLink folder (except in unwanted directories). Sort and filter results to ease the comparaison with source
    # grep -hoFf strings.txt -r /drives/d/Mr.B/GitHub/testlink-code/ --exclude-dir=locale --exclude-dir=.git --exclude-dir=docs --exclude-dir=third_party | sort | uniq > strings_found.txt

8. Compare files strings.txt (declared variables) and strings_found.txt (used variables) to found and export unused variables
    # comm -23 strings.txt strings_found.txt > strings_not_used.txt


I can open a PR with all unused strings deleted from locale\en_GB\strings.txt if needed.
TagsNo tags attached.
Database (MySQL,Postgres,etc)Any
BrowserAny
PHP VersionAny
TestCaseID
QA Team - Task Workflow StatusTBD
Attached Filestxt file icon strings_not_used.txt [^] (8,389 bytes) 2017-06-01 23:02 [Show Content]

- Relationships

-  Notes
(0026439)
fman (administrator)
2017-06-02 07:45

 thanks for your help.
please proceed opening the PR
(0026444)
Mr.Bricodage (updater)
2017-06-05 10:47

 After investigation, some strings are generated in testlink code.

Example :
$guiObj->title = lang_get('list_of_' . $verbose_status);

that is linked to :
$TLS_list_of_blocked
$TLS_list_of_failed
$TLS_list_of_not_run

which were detected as 'not_used' by the script.

Need more time to open the PR, I have to find an other way to automate the detection.
(0026447)
fman (administrator)
2017-06-05 12:22

 OK; thanks again
(0026504)
Mr.Bricodage (updater)
2017-06-20 19:55

 New process that will be used in PR

0. Copy en_GB\strings.txt from TestLink repository to work area
    # cp /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt ~/Desktop/strings_modified.txt
1. Remove all lines that don't start with $TLS_ (comments, header, blank lines, ...)
    # sed -i '/^\$TLS_/!d' strings_modified.txt
2. Remove all translations (all data after equals)
    # sed -i 's:=.*$::g' strings_modified.txt
3. Remove $TLS_ prefix (5 first characters of each line)
    # sed -i 's/^.....//' strings_modified.txt
4. Remove spaces and tabs at the end of each line
    # sed -i 's/[[:blank:]]*$//' strings_modified.txt
4. Remove short text values (less that 8 characters) that can be searched in an automatic way
    # sed -i '/........./!d' strings_modified.txt
5. Carriage return management
    If CRLF, \r\n replaced by \n by text editor option
6. sort strings_modified to find
    # sort strings_modified.txt -o strings_modified.txt

7. Looking for duplicates in variables
    # sort strings_modified.txt | uniq -cd
    If values are found, delete duplicates in TestLink file.
        rules :
        - if descriptions are identical, keep the variable in the best section
        - if descriptions are different, keep the last occurence (the one currently used by TestLink)

    
8. Search for each row present in the file in the TestLink folder, except in specifics directories. Sort and filter results to help the comparaison with source
    # grep -hoFf strings_modified.txt -r /drives/d/Mr.B/GitHub/testlink-code/ --exclude-dir=locale --exclude-dir=.git --exclude-dir=docs --exclude-dir=third_party | sort | uniq > strings_used.txt

9. Compare adapted TestLink file content "strings_modified.txt" and "strings_used.txt" (used variables) and export unused variables
    # comm -23 strings_modified.txt strings_used.txt > strings_not_used.txt

    
10. Search for label generated by concatenation in the code and manually add entries that match pattern to "strings_used.txt"

    10.1 Example : lang_get("action_{$argsObj->doAction}_user")
        Regex : (lang_get\()+(.)+\}+(.)+\)
        1 Result :
            lang_get("action_{$argsObj->doAction}_user")

    10.2 Example : lang_get("req_title_" . $status)
        Regex : (lang_get)+(.)+\.+(.)+\)
        12 Results :
            lang_get('API_' . $ret['API_error_code'])
            lang_get('test_status_' . $suffix)
            lang_get('testCaseStatus_' . $human);
            lang_get('bad_password_' . $attr);
            lang_get('issue_status_' . $tlStatus); = (Mantis connector)
            lang_get('list_of_' . $verbose_status)
            lang_get('no_' . $statusVerbose . '_with_tester')
            lang_get('info_' . $resultsCfg['code_status'][$args->type] .'_tc_report');
            lang_get("req_title_" . $status)
            lang_get('testCaseStatus_' . $vv)
            lang_get('container_title_' . $args->feature);
            lang_get('btn_reorder_testcases' . $lblkey);
            lang_get('access_' . $accessKey) (private/public)
    
11. False positive test : search for "strings_not_used" content in TestLink, should return no match.
grep -wnFf strings_not_used.txt -r /drives/d/Mr.B/GitHub/testlink-code/ --exclude-dir=locale --exclude-dir=.git --exclude-dir=docs --exclude-dir=third_party -exclude-from=CHANGELOG | sort | uniq

12. Restore prefix $TLS_ for all strings listed in strings_not_used.txt
    # sed -i -e 's/^/$TLS_/' strings_not_used.txt
    
13. Color strings to be deleted. If pattern matches on the right side of the equal sign in strings.txt file, it means that variables are used locally in strings.txt and should be deleted manually from strings_not_used.txt (examples : $TLS_btn_update, $TLS_imp_result_ko and $TLS_no_right_to_delete_executed_tc are used in others variables)
    # grep -wnFf strings_not_used.txt /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt
    
14. Count how many lines will be deleted
    # grep -Ff strings_not_used.txt /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt | wc -l

15. Count how many lines will be deleted if duplicates (step 7) have been fixed (result should be identical to step 13)
    # grep -Ff strings_not_used.txt /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt | sort | uniq | wc -l

16. Match whole word only
    16.1 force pattern in source
        grep return partial matching. Example :
        grep btn_upload /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt
            $TLS_btn_upload = "Upload";
            $TLS_btn_upload_file = "Upload file";
            
        The pattern [:space:]=[:space:] must be used for each variable definition between variable and content in en_GB\strings.txt
        # grep -vnF ' = ' /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt
        and fix all lines that are listed to use [:space:]=[:space:]

    16.2 Add suffix for all strings listed in strings_not_used.txt
        # sed -i -e 's/$/ = /' strings_not_used.txt

17. Remove "$TLS_related_to" value.
    defined in configuration ($tlCfg->testcase_cfg->relations->type_labels)

18. Don't delete values that are used by default by TestLink install
    Remove "$TLS_no_rights", "$TLS_no_rights" and "$TLS_no_rights" values from strings_not_used.txt
    
19. table label defined with variable.
    Example : usersAssign.tpl, r.167 : lang_get s="th_roles_$featureVerbose"
    $TLS_th_roles_testproject = "Test Project Role";

    Regex : lang_get ([a-zA-Z])+=([ "])([a-zA-Z_])+\$([a-zA-Z_])+(["])
    1 match : usersAssign.tpl, r.167 : lang_get s="th_roles_$featureVerbose"

20. Delete all items that are present in strings_not_used.txt
    # grep -Gf strings_not_used.txt /drives/d/Mr.B/GitHub/testlink-code/locale/en_GB/strings.txt > final_strings.txt
(0026505)
Mr.Bricodage (updater)
2017-06-20 20:03

 PR : https://github.com/TestLinkOpenSourceTRMS/testlink-code/pull/105 [^]
(0026598)
Mr.Bricodage (updater)
2017-07-12 19:48

 PR merged, status should be marked as resolved

- Issue History
Date Modified Username Field Change
2017-06-01 23:02 Mr.Bricodage New Issue
2017-06-01 23:02 Mr.Bricodage File Added: strings_not_used.txt
2017-06-02 07:45 fman Note Added: 0026439
2017-06-05 10:47 Mr.Bricodage Note Added: 0026444
2017-06-05 12:22 fman Note Added: 0026447
2017-06-20 19:55 Mr.Bricodage Note Added: 0026504
2017-06-20 20:03 Mr.Bricodage Note Added: 0026505
2017-07-12 19:48 Mr.Bricodage Note Added: 0026598
2017-07-13 19:55 fman QA Team - Task Workflow Status => TBD
2017-07-13 19:55 fman Status new => resolved
2017-07-13 19:55 fman Fixed in Version => 1.9.17 (2017 Q2)
2017-07-13 19:55 fman Resolution open => fixed
2017-07-13 19:55 fman Assigned To => fman


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker