Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006740TestLinkReports - Req. Based Reportpublic2014-11-11 21:352014-12-31 14:22
Reportersbill 
Assigned Tofman 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.9.12 (2014 Q3) 
Fixed in Version1.9.13 (2015 #1) 
Summary0006740: Req Based Report shows a rolled up status of "Partial Pass" when all test cases are not run
DescriptionIn the req based report is generated, the report lists all test cases for the selected requirements even those that are not part of the selected test plan.

When some test cases are not part of the test plan and all of the test cases have not been run, a status of Partially Passed is given. I was expected a not run status.


Steps To Reproduce1) define a requirement r-1
2) associate test cases t-1 and t-2 to this requirement
3) create a test plan tp
4) add only one test case (t-1) to the test plan tp
5) run the requirement based report for this test plan

I was able to reproduce in the demo environment as well, project: RQT: ReqCoverageTest
TagsNo tags attached.
Database (MySQL,Postgres,etc)mySQL
BrowserFirefox
PHP Version
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Filespng file icon reqreport.png [^] (17,340 bytes) 2014-11-11 21:35

- Relationships
child of 0006609closedfman Availables hot-fixes for 1.9.12 & How To get full fixed package from gitorious 

-  Notes
(0022118)
fman (administrator)
2014-11-12 11:18

please get latest code from gitorious (branch 1.9), do fresh install on sandbox, retest and provide feedback
(0022129)
fman (administrator)
2014-11-13 09:59

any news?
(0022130)
sbill (reporter)
2014-11-13 13:36

I installed 1.9.13[dev] and get the same results as described in my notes
(0022154)
fman (administrator)
2014-11-14 21:06
edited on: 2014-11-14 21:07

1. create test project - tproj01
2. create test suite TS100
3. create (inside TS100) two test cases TC1A and TC1B
4. create req. spec - RSPEC-100
5. create (inside RSPEC-100), req. REQ-01
6. create test plan TPLAN-A
7. create Build 1.0 for TPLAN-A

(0022155)
fman (administrator)
2014-11-14 21:19

>> In the req based report is generated, the report lists all test cases for the >> selected requirements even those that are not part of the selected test plan.
IMHO There is problem if you consider or not the second test case
REQ has been related to TWO test cases, then if just ONE has been linked to the test plan, REQ anyway has to consider status of BOTH test cases.
What need to be clarified is what linked test cases means on this report, because
test cases are present NOT BECAUSE they are present on TEST PLAN,
but because BOTH have been assigned to the REQ
Probably also is need a notice near to TEST CASES that ARE NOT linked to Test Plan.

REQUIREMENT are NOT assigned IN DIRECT WAY to a test plan but in INDIRECT way through Test cases.
(0022158)
fman (administrator)
2014-11-15 08:40

Work in progress to provide a RIGHT req status indication
(0022185)
sbill (reporter)
2014-11-17 21:59

I saw with the latest build the not in test plan message. It is not clear what is most wanted or expected. In one context, we are waying in the context of a particular test plan, the report now shows the results for the requirements mentioned in the test plan.

However from the point of view of the requirements, it is incomplete since you can't control which requirements you are interested in. If you want a requirement coverage report, from a requirement basis, then you would need to select a requirement spec and see the results based on that, giving similiar output as you do now. I would say you would probably also need to select the test plans that give the context to the results. This is another report however and another enhancement request.

In short then, I would say the new implementation is correct, in that you ignore the test execution results for test cases not in the selected test plan. Readers would just need to understand that the report shows the status of requirements only on the basis of the test cases covered by the test plan.
(0022190)
fman (administrator)
2014-11-18 06:08

ok.

- Issue History
Date Modified Username Field Change
2014-11-11 21:35 sbill New Issue
2014-11-11 21:35 sbill File Added: reqreport.png
2014-11-12 11:18 fman Note Added: 0022118
2014-11-13 09:59 fman Note Added: 0022129
2014-11-13 09:59 fman Assigned To => fman
2014-11-13 09:59 fman Status new => feedback
2014-11-13 13:36 sbill Note Added: 0022130
2014-11-13 13:36 sbill Status feedback => assigned
2014-11-14 21:06 fman Note Added: 0022154
2014-11-14 21:07 fman Note Edited: 0022154 View Revisions
2014-11-14 21:14 fman Note View State: 0022154: public
2014-11-14 21:19 fman Note Added: 0022155
2014-11-15 08:40 fman Note Added: 0022158
2014-11-15 14:28 fman Status assigned => feedback
2014-11-15 14:28 fman QA Team - Task Workflow Status => READY FOR TESTING
2014-11-15 14:29 fman Relationship added child of 0006609
2014-11-17 21:59 sbill Note Added: 0022185
2014-11-17 21:59 sbill Status feedback => assigned
2014-11-18 06:08 fman Note Added: 0022190
2014-11-18 06:08 fman Status assigned => resolved
2014-11-18 06:08 fman Fixed in Version => 1.9.13 (2015 #1)
2014-11-18 06:08 fman Resolution open => fixed
2014-12-31 14:22 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker