Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005272TestLinkReportspublic2012-10-06 22:002013-05-26 20:06
Reporterdenjolras 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.9.4 (2012 Q3 - bug fixing) 
Fixed in Version1.9.5 (2012 Q4 - bug fixing) 
Summary0005272: General Test Plan Metrics / Overall Build Status / Not Run is not for build but global.
DescriptionIn General Test Plan Metrics,
The Overall Build Status show a list of the build.
for each we can see the Assigned, Passed, Failed, Block and Not RUN

The "Not Run" is not for the build, but the global value (ie never run)

Of course, given this bug, the addition of Passed+Failed+Blocked/Not run does not = Assigned.
Steps To ReproduceI use platform, have multiple build.
I have an extra status "w" ( Passed with warning )
Version 1.9.0.4 with all fixes.
Additional Informationyou can see that the addition of f+p+w+b+n is not = qty

mysql> /* Class:tlTestPlanMetrics - Method: getExecCountersByBuildExecStatus UNION WITH ALL CLAUSE */ SELECT build_id,status, count(0) AS exec_qty FROM (/* Class:tlTestPlanMetrics - Method: getExecCountersByBuildExecStatus sqlUnionAB - executions */ SELECT UA.build_id, TPTCV.tcversion_id, TPTCV.platform_id, COALESCE(E.status,'n') AS status FROM testplan_tcversions TPTCV JOIN user_assignments UA ON UA.feature_id = TPTCV.id AND UA.build_id IN (757,782,793,802,808) AND UA.type = 1 /* GO FOR Absolute LATEST exec ID ON BUILD and PLATFORM */ JOIN ( SELECT EE.tcversion_id,EE.testplan_id,EE.platform_id,EE.build_id, MAX(EE.id) AS id FROM executions EE WHERE EE.testplan_id=112983 AND EE.build_id IN (757,782,793,802,808) GROUP BY EE.tcversion_id,EE.testplan_id,EE.platform_id,EE.build_id ) AS LEBBP ON LEBBP.testplan_id = TPTCV.testplan_id AND LEBBP.platform_id = TPTCV.platform_id AND LEBBP.tcversion_id = TPTCV.tcversion_id AND LEBBP.testplan_id = 112983 /* Get execution status WRITTEN on DB */ JOIN executions E ON E.id = LEBBP.id AND E.build_id = UA.build_id WHERE TPTCV.testplan_id=112983 AND UA.build_id IN (757,782,793,802,808) UNION ALL /* Class:tlTestPlanMetrics - Method: getExecCountersByBuildExecStatus sqlUnionBB - NOT RUN */ SELECT UA.build_id, TPTCV.tcversion_id, TPTCV.platform_id, COALESCE(E.status,'n') AS status FROM testplan_tcversions TPTCV JOIN user_assignments UA ON UA.feature_id = TPTCV.id AND UA.build_id IN (757,782,793,802,808) AND UA.type = 1 /* Get REALLY NOT RUN => BOTH LE.id AND E.id ON LEFT OUTER see WHERE */ LEFT OUTER JOIN ( SELECT EE.tcversion_id,EE.testplan_id,EE.platform_id,EE.build_id, MAX(EE.id) AS id FROM executions EE WHERE EE.testplan_id=112983 AND EE.build_id IN (757,782,793,802,808) GROUP BY EE.tcversion_id,EE.testplan_id,EE.platform_id,EE.build_id ) AS LEBBP ON LEBBP.testplan_id = TPTCV.testplan_id AND LEBBP.platform_id = TPTCV.platform_id AND LEBBP.tcversion_id = TPTCV.tcversion_id AND LEBBP.testplan_id = 112983 LEFT OUTER JOIN executions E ON E.tcversion_id = TPTCV.tcversion_id AND E.testplan_id = TPTCV.testplan_id AND E.platform_id = TPTCV.platform_id /* FILTER BUILDS in set on target test plan (not alway can be applied) */ WHERE TPTCV.testplan_id=112983 AND UA.build_id IN (757,782,793,802,808) /* Get REALLY NOT RUN => BOTH LE.id AND E.id NULL */ AND E.id IS NULL AND LEBBP.id IS NULL ) AS SQBU GROUP BY build_id,status ;
+----------+--------+----------+
| build_id | status | exec_qty |
+----------+--------+----------+
| 757 | f | 1 |
| 757 | n | 11 |
| 757 | p | 46 |
| 782 | b | 7 |
| 782 | f | 24 |
| 782 | n | 11 |
| 782 | p | 584 |
| 782 | w | 28 |
| 793 | b | 1 |
| 793 | f | 3 |
| 793 | n | 11 |
| 793 | p | 25 |
| 793 | w | 1 |
| 802 | b | 21 |
| 802 | f | 3 |
| 802 | n | 11 |
| 802 | p | 201 |
| 802 | w | 14 |
| 808 | b | 4 |
| 808 | n | 14 |
| 808 | p | 22 |
+----------+--------+----------+
21 rows in set (0.27 sec)

mysql> SELECT COUNT(0) AS qty, UA.build_id FROM user_assignments UA JOIN testplan_tcversions TPTCV ON TPTCV.id = UA.feature_id WHERE UA. build_id IN ( 757,782,793,802,808 ) AND UA.type = 1 GROUP BY build_id ;
+-----+----------+
| qty | build_id |
+-----+----------+
| 850 | 757 |
| 850 | 782 |
| 265 | 793 |
| 256 | 802 |
| 80 | 808 |
+-----+----------+
5 rows in set (0.01 sec)
TagsTO BE FIXED on 2.0
Database (MySQL,Postgres,etc)MySQL
Browser
PHP Version
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Filesxml file icon testproject-deep.xml [^] (3,073 bytes) 2012-10-08 05:36
jpg file icon Not run.jpg [^] (89,559 bytes) 2012-10-08 05:37


jpg file icon extra_prints.jpg [^] (144,207 bytes) 2012-10-09 05:13


jpg file icon events.jpg [^] (177,538 bytes) 2012-10-09 05:13

- Relationships
related to 0005353closedfman Not run count on General Test Plan Metrics is wrong 
child of 0005193closedfman Availables hot-fixes for 1.9.4 & How To get full fixed package from gitorious 

-  Notes
(0017619)
denjolras (reporter)
2012-10-06 22:17

as a dirty workaround, I have modify my getOverallBuildStatusForRender
to recompute the not run:

just after
$rf = &$renderObj->info[$buildID]['details'];

I've added :
                                // My workaound
                                $metrics['with_tester'][$buildID]['n']['exec_qty']=$metrics['total'][$buildID]['qty'];
                                foreach($code_verbose as $statusCode => $statusVerbose)
                                {
                                        $rf[$statusVerbose] = array('qty' => 0, 'percentage' => 0);
                                        $rf[$statusVerbose]['qty'] = $metrics['with_tester'][$buildID][$statusCode]['exec_qty'];

                                        $metrics['with_tester'][$buildID]['n']['exec_qty'] -= $statusVerbose == 'not_run' ? 0 : $rf[$statusVerbose]['qty'];
                                }
(0017620)
fman (administrator)
2012-10-07 07:41

@denjolras
It's ok if you provide a workaround, but is absolutely more useful for us if you can provided a very detailed steps to reproduce, in this way

1. create test project
2. create test suite A, B, C
3. create following test cases ...

Then is very important that you export as much as possible elements on our XML format, this way we can re-create your environment and retest.

Another detail that is useful whilw reporting, please do not create kilometric notes as you have done with the query, put all this info in a text file or something like that and upload.
This make browsing the issue easier.
(0017621)
sujataverma3 (reporter)
2012-10-08 05:34

I tried to reproduce the bug, and was able to reproduce.

Steps to reproduce :

1) Create test project A

2) Create Test plan A

3) Create 2 builds -> 501122, 502233

3) Create 4 test cases in Test suite A

4) Assign all 4 test cases for both Builds.

5) Go to General test plan metrics

 -> Over all build status shows -> 4 not run for both the build -> correct

6) Now execute -> test cases for one build 501122-> 1 Passed, 1 failed, 1 blocked

7) Go to general test plan metrics

 -> overall build status shows
 
    not run = 1 for both the builds -> wrong behavior. (As mentioned in the bug it is taking "never run" , and is not on per build basis)

Please see the attached screenshot for more clarity.

I will add this test case in demo QA repository.
(0017622)
fman (administrator)
2012-10-08 09:02

@sujataverma3
Thanks a lot!!!
(0017626)
fman (administrator)
2012-10-08 19:15

issue is generated by missing clauses regarding UA.build_id
The code was in place but was commented, with some concern.
I've going to commit.
@sujataverma3
I would appreciate if we can do a set of extensive (as much as your time allows) test.
Same for denjolras
    
we need to test with:
NO PLATFORM - Multiple builds => this has been tested.
NO PLATFORM - ONE BUIDL
ONE PLATFORM - Multiple builds
ONE PLATFORM - ONE BUILD
(0017627)
fman (administrator)
2012-10-08 19:17

TestLink 1.9.5
http://gitorious.org/testlink-ga/testlink-code/commit/c5a033e6c35d3b73ef7ba9ce49af9ae23d3e89e0 [^]
(0017630)
sujataverma3 (reporter)
2012-10-09 05:13

Issue is not observed for No Platform- 2 builds, but some extra messages are displayed when we click on General test plan metrics. I am attaching the event log and screenshot.
(0017638)
fman (administrator)
2012-10-09 18:13

@sujataverma3
ok will remove debug info
would you mind to test other scenarios ?
same request to reporter
(0017643)
sujataverma3 (reporter)
2012-10-10 04:14

No platform - 1 build -> No issue

No Platform -> multiple builds ( 4 builds) -> no issues

1 Platform -> 1 build -> No issue

1 Platform -> multiple builds ( some issue ) -> trying to reproduce , will update
(0017644)
sujataverma3 (reporter)
2012-10-10 04:40

Ok, there is another issue and which is independent of platforms, will file bug separately.

For this issue

1 Platform -> 3 builds -> properly "not run" status updated

2 Platform -> 1 build -> no issue

2 platform -> 2 build -> over all build status shows correct
(0017645)
fman (administrator)
2012-10-10 05:56

OK, last check. does this means I can set this issue to resolved ?
(0017647)
sujataverma3 (reporter)
2012-10-10 08:02

Yes, for all combination,I did not see the issue and "Not run" status is shown properly on build basis.

- Issue History
Date Modified Username Field Change
2012-10-06 22:00 denjolras New Issue
2012-10-06 22:17 denjolras Note Added: 0017619
2012-10-07 07:41 fman Note Added: 0017620
2012-10-08 05:34 sujataverma3 Note Added: 0017621
2012-10-08 05:36 sujataverma3 File Added: testproject-deep.xml
2012-10-08 05:37 sujataverma3 File Added: Not run.jpg
2012-10-08 09:02 fman Note Added: 0017622
2012-10-08 19:15 fman Note Added: 0017626
2012-10-08 19:16 fman Product Version => 1.9.4 (2012 Q3 - bug fixing)
2012-10-08 19:16 fman Fixed in Version => 1.9.5 (2012 Q4 - bug fixing)
2012-10-08 19:17 fman Note Added: 0017627
2012-10-08 19:17 fman Tag Attached: TO BE FIXED on 2.0
2012-10-08 19:18 fman Relationship added child of 0005193
2012-10-08 19:45 fman Assigned To => fman
2012-10-08 19:45 fman Status new => feedback
2012-10-08 20:55 fman Task Workflow Status TBD => READY FOR TESTING
2012-10-09 05:13 sujataverma3 Note Added: 0017630
2012-10-09 05:13 sujataverma3 File Added: extra_prints.jpg
2012-10-09 05:13 sujataverma3 File Added: events.jpg
2012-10-09 18:13 fman Note Added: 0017638
2012-10-10 04:14 sujataverma3 Note Added: 0017643
2012-10-10 04:40 sujataverma3 Note Added: 0017644
2012-10-10 05:56 fman Note Added: 0017645
2012-10-10 08:02 sujataverma3 Note Added: 0017647
2012-10-12 18:45 fman Status feedback => resolved
2012-10-12 18:45 fman Resolution open => fixed
2012-11-19 15:42 fman Relationship added related to 0005353
2013-05-26 20:06 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker