|Anonymous | Login | Signup for a new account||2020-09-28 19:44 UTC|
|Main | My View | View Issues | Change Log | My Account|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0005003||TestLink||Reports||public||2012-05-04 17:14||2012-09-01 19:59|
|Product Version||1.9.3 (2011 Q3 - bug fixing)|
|Fixed in Version||1.9.4 (2012 Q3 - bug fixing)|
|Summary||0005003: General Test Plan Metrics pass/fail/blocked numbers include unassigned test cases|
|Description||Sorry if this is a duplicate. I've searched the issue list for a while and have read related issues 0004695 and 0004734, but this particular issue doesn't appear to be mentioned anywhere that I can see.|
In the General Test Plan Metrics page, under overall build status, the number of passed/failed and blocked test cases includes unassigned test cases. This can lead to greater than 100% of tests being complete, since the total number of test cases just includes assigned ones.
|Steps To Reproduce||1) Create a new test project, test plan, test suite and build|
2) Make two test cases
3) assign one of them to a user
4) pass both of them
5) view general test plan metrics
It will show that there is one assigned test case, but that 2 have passed, and so 200% have passed. The completed column at the end shows 100% so I assume it only includes assigned test cases, but the other columns include unassigned ones. Regardless of whether you only measure assigned or not, I think the metrics should be consistent in order to have meaningful numbers.
|Tags||No tags attached.|
|QA Team - Task Workflow Status||TBD|
|Attached Files|| Screenshot at 2012-05-04 13:00:14.png [^] (48,957 bytes) 2012-05-04 17:14
>> Regardless of whether you only measure assigned or not, I think the metrics
>> should be consistent in order to have meaningful numbers.
unfortunatelly this sentence is to generic, that do not help.
Please make the effort of explaining clearly.
When reporting an issue or feature request, use same level of detail you will use on your regular work.
We are planning to do continue on 1.9.4 with same logic:
On General test plan metrics ONLY PIECE of data that will consider TESTER ASSIGNMENT will be Overall Builds Section.
All other section that summarize data, will consider Testcase WITH and WITHOUT Tester assignment.
This kind of summarize vision (by platform, but top level suites, etc), seems to provide figures that create LESS confusion, when calculated as we have done till today.
If possible we will try to add an option on GUI, to allow user following choice:
1. CONSIDER on ALL METRICS only Test cases that HAS TESTER ASSIGNMENT
The issue is that the metric is not only considering cases with tester assignment. In the steps to reproduce that I listed, one test that is assigned will pass and one test that is unassigned will pass, however the general test plan metrics will show two passes even though only one assigned test has passed. It is correctly showing the total number as only including assigned, meaning that 2/1 test cases have passed, which is nonsensical and useless.
I'm not talking about consistency between sections, I'm talking about consistency within the overall builds section itself.
|2012-05-04 17:14||DanielB||New Issue|
|2012-05-04 17:14||DanielB||File Added: Screenshot at 2012-05-04 13:00:14.png|
|2012-05-04 18:50||fman||Note Added: 0016639|
|2012-05-04 18:55||DanielB||Note Added: 0016640|
|2012-05-04 20:01||fman||Relationship added||duplicate of 0004989|
|2012-07-28 15:08||fman||Task Workflow Status||=> TBD|
|2012-07-28 15:08||fman||Status||new => resolved|
|2012-07-28 15:08||fman||Fixed in Version||=> 1.9.4 (2012 Q3 - bug fixing)|
|2012-07-28 15:08||fman||Resolution||open => fixed|
|2012-07-28 15:08||fman||Assigned To||=> fman|
|2012-09-01 19:59||fman||Note Added: 0017317|
|2012-09-01 19:59||fman||Status||resolved => closed|
|Copyright © 2000 - 2020 MantisBT Team|