MantisBT - TestLink
View Issue Details
0005353TestLinkReportspublic2012-11-19 11:212012-11-22 20:36
twelve 
fman 
normalmajoralways
closedfixed 
1.9.4 (2012 Q3 - bug fixing) 
1.9.5 (2012 Q4 - bug fixing) 
MySQL 5.5
5.3
READY FOR TESTING
0005353: Not run count on General Test Plan Metrics is wrong
This bug is about the same problem as 0005272. However not only the "Overall Build Status" is wrong (now fixed), but also the reports below (Results by top level Test Suites, by Platform, by Priority, by Keyword) show wrong numbers for "Not run".
Passed, Failed and Blocked are calculated only for the active builds, but Not run is calculated for ALL builds.
That means that Passed + Failed + Blocked + Not run will not sum up to Total, if you have disabled builds in the testplan.
1. Create a testplan
2. Create 2 builds, Build1 and Build2
3. Assign 3 testcases to the testplan (with a platform so that you have some data for the by-platform-report)
4. Execute one testcase with result Passed and one testcase with result Failed for Build1
5. Disable Build1
6. Open the General Test Plan Metrics

You would expect that all testcases are displayed as Not run for Build2. However what for example the "Report by platform" actually displays is Total: 3, Not run: 1, Passed: 0, Failed: 0, Blocked: 0.
1+0+0+0 is not 3
I have attached a patched version of the file tlTestPlanMetrics.class.php that seems to fix this problem. However im not a SQL expert, so this may be wrong.
The patch is against the current GIT version.
QA-TEAM TASK, TO BE FIXED on 2.0
related to 0005272closed fman General Test Plan Metrics / Overall Build Status / Not Run is not for build but global. 
? tlTestPlanMetrics.class.php (83,468) 2012-11-19 11:21
http://mantis.testlink.org/file_download.php?file_id=2969&type=bug
xml testplan-export.xml (1,067) 2012-11-21 09:52
http://mantis.testlink.org/file_download.php?file_id=2970&type=bug
xml testsuite-export.xml (1,259) 2012-11-21 09:52
http://mantis.testlink.org/file_download.php?file_id=2971&type=bug
Issue History
2012-11-19 11:21twelveNew Issue
2012-11-19 11:21twelveFile Added: tlTestPlanMetrics.class.php
2012-11-19 15:39fmanNote Added: 0017843
2012-11-19 15:39fmanStatusnew => feedback
2012-11-19 15:41fmanNote Added: 0017845
2012-11-19 15:42fmanNote Edited: 0017843bug_revision_view_page.php?bugnote_id=17843#r2210
2012-11-19 15:42fmanRelationship addedrelated to 0005272
2012-11-21 09:52twelveNote Added: 0017863
2012-11-21 09:52twelveStatusfeedback => new
2012-11-21 09:52twelveFile Added: testplan-export.xml
2012-11-21 09:52twelveFile Added: testsuite-export.xml
2012-11-21 09:55twelveNote Edited: 0017863bug_revision_view_page.php?bugnote_id=17863#r2214
2012-11-21 19:20fmanNote Added: 0017866
2012-11-21 21:45fmanNote Added: 0017868
2012-11-21 21:45fmanTask Workflow StatusTBD => READY FOR TESTING
2012-11-21 21:46fmanTag Attached: TO BE FIXED on 2.0
2012-11-21 21:46fmanTag Attached: QA-TEAM TASK
2012-11-21 21:46fmanAssigned To => fman
2012-11-21 21:46fmanStatusnew => feedback
2012-11-22 12:52twelveNote Added: 0017874
2012-11-22 12:52twelveStatusfeedback => assigned
2012-11-22 16:18twelveNote Added: 0017876
2012-11-22 16:20twelveNote Edited: 0017876bug_revision_view_page.php?bugnote_id=17876#r2216
2012-11-22 20:36fmanStatusassigned => closed
2012-11-22 20:36fmanResolutionopen => fixed
2012-11-22 20:36fmanFixed in Version => 1.9.5 (2012 Q4 - bug fixing)

Notes
(0017843)
fman   
2012-11-19 15:39   
(edited on: 2012-11-19 15:42)
More detailed information is needed.
What about test execution assignment on any build ?

We appreciate patches but is much better if you provide detailed (very detailed) steps to reproduce ( i.e. complete test cases).
We need thing like:
1. Create a testplan TPLAN X
2. Create 2 builds, Build1 and Build2
3. Create test spec with following structure:
TSUITE 1, with TS-C01, TS-C02 ...

SEE WORK DONE on related issue on providing a really clear test case

Is also important that we provide TESTLINK XML exported files that will easy the work of dev team.

We will wait for this info before act.

Thanks

(0017845)
fman   
2012-11-19 15:41   
>> However not only the "Overall Build Status" is wrong (now fixed)
Then no need to say is wrong because has been fixed => this sentence IMHO adds only confusion
(0017863)
twelve   
2012-11-21 09:52   
(edited on: 2012-11-21 09:55)
Creation:
1. Create Project Testproject1, no advanced features needed
2. Create Testplan TP1
3. Create Builds B1 and B2
4. Create Platform Plat1
5. Create Testsuite TS1
6. Create Testcase TC1, TC2, TC3
7. Add Plat1 to TP1
8. Add TC1 + TC2 + TC3 to TP1 / Plat1
9. Assign a user to TC1 + TC2 + TC3 for both builds

Execution:
10. Select Build B1
11. Execute TC1 with result Passed
12. Execute TC2 with result Failed

Reporting:
13. Disable Build B1
14. Open General Test Plan Metrics Report
15. Look at Results by platform

Plat1 now shows: Total 3, Not run 1, Failed 0, Passed 0, Blocked 0
This is wrong, because only Build B2 is active and for B2 we did not execute any testcases at all. So it should display: Total 3, Not run 3, Failed 0, Passed 0, Blocked 0. Then also the numbers add up, Not run + Failed + Passed + Blocked = Total.

The Report by top level suite, Report by priority and Report by keyword also have the problem.

(0017866)
fman   
2012-11-21 19:20   
Ok, now we can try to investigate the issue
(0017868)
fman   
2012-11-21 21:45   
TestLink 1.9.4 fix
http://gitorious.org/testlink-ga/testlink-code/commit/7d04bbb0c82d9c6710afe92b1639abbcfabe8a88 [^]

Please try with this and provide feedback
(0017874)
twelve   
2012-11-22 12:52   
Looks good to me. Thanks
(0017876)
twelve   
2012-11-22 16:18   
(edited on: 2012-11-22 16:20)
One more thing: In function helperBuildSQLExecCounters() in lib/functions/tlTestPlanMetrics.class.php " AND E.platform_id = TPTCV.platform_id " is added twice. Not sure if something is missing there.