Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005353TestLinkReportspublic2012-11-19 11:212012-11-22 20:36
Reportertwelve 
Assigned Tofman 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.9.4 (2012 Q3 - bug fixing) 
Fixed in Version1.9.5 (2012 Q4 - bug fixing) 
Summary0005353: Not run count on General Test Plan Metrics is wrong
DescriptionThis bug is about the same problem as 0005272. However not only the "Overall Build Status" is wrong (now fixed), but also the reports below (Results by top level Test Suites, by Platform, by Priority, by Keyword) show wrong numbers for "Not run".
Passed, Failed and Blocked are calculated only for the active builds, but Not run is calculated for ALL builds.
That means that Passed + Failed + Blocked + Not run will not sum up to Total, if you have disabled builds in the testplan.
Steps To Reproduce1. Create a testplan
2. Create 2 builds, Build1 and Build2
3. Assign 3 testcases to the testplan (with a platform so that you have some data for the by-platform-report)
4. Execute one testcase with result Passed and one testcase with result Failed for Build1
5. Disable Build1
6. Open the General Test Plan Metrics

You would expect that all testcases are displayed as Not run for Build2. However what for example the "Report by platform" actually displays is Total: 3, Not run: 1, Passed: 0, Failed: 0, Blocked: 0.
1+0+0+0 is not 3
Additional InformationI have attached a patched version of the file tlTestPlanMetrics.class.php that seems to fix this problem. However im not a SQL expert, so this may be wrong.
The patch is against the current GIT version.
TagsQA-TEAM TASK, TO BE FIXED on 2.0
Database (MySQL,Postgres,etc)MySQL 5.5
Browser
PHP Version5.3
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Files? file icon tlTestPlanMetrics.class.php [^] (83,468 bytes) 2012-11-19 11:21
xml file icon testplan-export.xml [^] (1,067 bytes) 2012-11-21 09:52
xml file icon testsuite-export.xml [^] (1,259 bytes) 2012-11-21 09:52

- Relationships
related to 0005272closedfman General Test Plan Metrics / Overall Build Status / Not Run is not for build but global. 

-  Notes
(0017843)
fman (administrator)
2012-11-19 15:39
edited on: 2012-11-19 15:42

More detailed information is needed.
What about test execution assignment on any build ?

We appreciate patches but is much better if you provide detailed (very detailed) steps to reproduce ( i.e. complete test cases).
We need thing like:
1. Create a testplan TPLAN X
2. Create 2 builds, Build1 and Build2
3. Create test spec with following structure:
TSUITE 1, with TS-C01, TS-C02 ...

SEE WORK DONE on related issue on providing a really clear test case

Is also important that we provide TESTLINK XML exported files that will easy the work of dev team.

We will wait for this info before act.

Thanks

(0017845)
fman (administrator)
2012-11-19 15:41

>> However not only the "Overall Build Status" is wrong (now fixed)
Then no need to say is wrong because has been fixed => this sentence IMHO adds only confusion
(0017863)
twelve (reporter)
2012-11-21 09:52
edited on: 2012-11-21 09:55

Creation:
1. Create Project Testproject1, no advanced features needed
2. Create Testplan TP1
3. Create Builds B1 and B2
4. Create Platform Plat1
5. Create Testsuite TS1
6. Create Testcase TC1, TC2, TC3
7. Add Plat1 to TP1
8. Add TC1 + TC2 + TC3 to TP1 / Plat1
9. Assign a user to TC1 + TC2 + TC3 for both builds

Execution:
10. Select Build B1
11. Execute TC1 with result Passed
12. Execute TC2 with result Failed

Reporting:
13. Disable Build B1
14. Open General Test Plan Metrics Report
15. Look at Results by platform

Plat1 now shows: Total 3, Not run 1, Failed 0, Passed 0, Blocked 0
This is wrong, because only Build B2 is active and for B2 we did not execute any testcases at all. So it should display: Total 3, Not run 3, Failed 0, Passed 0, Blocked 0. Then also the numbers add up, Not run + Failed + Passed + Blocked = Total.

The Report by top level suite, Report by priority and Report by keyword also have the problem.

(0017866)
fman (administrator)
2012-11-21 19:20

Ok, now we can try to investigate the issue
(0017868)
fman (administrator)
2012-11-21 21:45

TestLink 1.9.4 fix
http://gitorious.org/testlink-ga/testlink-code/commit/7d04bbb0c82d9c6710afe92b1639abbcfabe8a88 [^]

Please try with this and provide feedback
(0017874)
twelve (reporter)
2012-11-22 12:52

Looks good to me. Thanks
(0017876)
twelve (reporter)
2012-11-22 16:18
edited on: 2012-11-22 16:20

One more thing: In function helperBuildSQLExecCounters() in lib/functions/tlTestPlanMetrics.class.php " AND E.platform_id = TPTCV.platform_id " is added twice. Not sure if something is missing there.


- Issue History
Date Modified Username Field Change
2012-11-19 11:21 twelve New Issue
2012-11-19 11:21 twelve File Added: tlTestPlanMetrics.class.php
2012-11-19 15:39 fman Note Added: 0017843
2012-11-19 15:39 fman Status new => feedback
2012-11-19 15:41 fman Note Added: 0017845
2012-11-19 15:42 fman Note Edited: 0017843 View Revisions
2012-11-19 15:42 fman Relationship added related to 0005272
2012-11-21 09:52 twelve Note Added: 0017863
2012-11-21 09:52 twelve Status feedback => new
2012-11-21 09:52 twelve File Added: testplan-export.xml
2012-11-21 09:52 twelve File Added: testsuite-export.xml
2012-11-21 09:55 twelve Note Edited: 0017863 View Revisions
2012-11-21 19:20 fman Note Added: 0017866
2012-11-21 21:45 fman Note Added: 0017868
2012-11-21 21:45 fman Task Workflow Status TBD => READY FOR TESTING
2012-11-21 21:46 fman Tag Attached: TO BE FIXED on 2.0
2012-11-21 21:46 fman Tag Attached: QA-TEAM TASK
2012-11-21 21:46 fman Assigned To => fman
2012-11-21 21:46 fman Status new => feedback
2012-11-22 12:52 twelve Note Added: 0017874
2012-11-22 12:52 twelve Status feedback => assigned
2012-11-22 16:18 twelve Note Added: 0017876
2012-11-22 16:20 twelve Note Edited: 0017876 View Revisions
2012-11-22 20:36 fman Status assigned => closed
2012-11-22 20:36 fman Resolution open => fixed
2012-11-22 20:36 fman Fixed in Version => 1.9.5 (2012 Q4 - bug fixing)



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker