Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005003TestLinkReportspublic2012-05-04 17:142012-09-01 19:59
ReporterDanielB 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSCentOSOS Version5.8
Product Version1.9.3 (2011 Q3 - bug fixing) 
Fixed in Version1.9.4 (2012 Q3 - bug fixing) 
Summary0005003: General Test Plan Metrics pass/fail/blocked numbers include unassigned test cases
DescriptionSorry if this is a duplicate. I've searched the issue list for a while and have read related issues 0004695 and 0004734, but this particular issue doesn't appear to be mentioned anywhere that I can see.

In the General Test Plan Metrics page, under overall build status, the number of passed/failed and blocked test cases includes unassigned test cases. This can lead to greater than 100% of tests being complete, since the total number of test cases just includes assigned ones.
Steps To Reproduce1) Create a new test project, test plan, test suite and build
2) Make two test cases
3) assign one of them to a user
4) pass both of them
5) view general test plan metrics

It will show that there is one assigned test case, but that 2 have passed, and so 200% have passed. The completed column at the end shows 100% so I assume it only includes assigned test cases, but the other columns include unassigned ones. Regardless of whether you only measure assigned or not, I think the metrics should be consistent in order to have meaningful numbers.
TagsNo tags attached.
Database (MySQL,Postgres,etc)mysql
Browserfirefox
PHP Version2.3
TestCaseID
QA Team - Task Workflow StatusTBD
Attached Filespng file icon Screenshot at 2012-05-04 13:00:14.png [^] (48,957 bytes) 2012-05-04 17:14

- Relationships
duplicate of 0004989closedfman Reports - Overall Build Status - refactoring and final business logic. 

-  Notes
(0016639)
fman (administrator)
2012-05-04 18:50

>> Regardless of whether you only measure assigned or not, I think the metrics
>> should be consistent in order to have meaningful numbers.

unfortunatelly this sentence is to generic, that do not help.
Please make the effort of explaining clearly.
When reporting an issue or feature request, use same level of detail you will use on your regular work.


We are planning to do continue on 1.9.4 with same logic:

On General test plan metrics ONLY PIECE of data that will consider TESTER ASSIGNMENT will be Overall Builds Section.

All other section that summarize data, will consider Testcase WITH and WITHOUT Tester assignment.

This kind of summarize vision (by platform, but top level suites, etc), seems to provide figures that create LESS confusion, when calculated as we have done till today.

If possible we will try to add an option on GUI, to allow user following choice:

1. CONSIDER on ALL METRICS only Test cases that HAS TESTER ASSIGNMENT
(DEFAULT NO)
(0016640)
DanielB (reporter)
2012-05-04 18:55

The issue is that the metric is not only considering cases with tester assignment. In the steps to reproduce that I listed, one test that is assigned will pass and one test that is unassigned will pass, however the general test plan metrics will show two passes even though only one assigned test has passed. It is correctly showing the total number as only including assigned, meaning that 2/1 test cases have passed, which is nonsensical and useless.

I'm not talking about consistency between sections, I'm talking about consistency within the overall builds section itself.
(0017317)
fman (administrator)
2012-09-01 19:59

1.9.4 released

- Issue History
Date Modified Username Field Change
2012-05-04 17:14 DanielB New Issue
2012-05-04 17:14 DanielB File Added: Screenshot at 2012-05-04 13:00:14.png
2012-05-04 18:50 fman Note Added: 0016639
2012-05-04 18:55 DanielB Note Added: 0016640
2012-05-04 20:01 fman Relationship added duplicate of 0004989
2012-07-28 15:08 fman Task Workflow Status => TBD
2012-07-28 15:08 fman Status new => resolved
2012-07-28 15:08 fman Fixed in Version => 1.9.4 (2012 Q3 - bug fixing)
2012-07-28 15:08 fman Resolution open => fixed
2012-07-28 15:08 fman Assigned To => fman
2012-09-01 19:59 fman Note Added: 0017317
2012-09-01 19:59 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker