Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0003451TestLinkasimon_thesispublic2010-05-11 20:492010-08-31 20:28
ReporterFalcon1 
Assigned Toasimon 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionopen 
PlatformOSOS Version
Product Version1.9 Beta 4 
Fixed in Version1.9 Beta 6 
Summary0003451: In the "Test reports and Metrics" -> "Test report" the "Last Result" is always "Not Run"
DescriptionI created a Test plan and added a Test suite with two test cases. I created two builds and executed the test cases on both of them. But in the report Test Report in the section Test reports and Metrics the Last Result is always Not Run.
TagsNo tags attached.
Database (MySQL,Postgres,etc)
Browser
PHP Version
TestCaseID
QA Team - Task Workflow Status
Attached Fileszip file icon fix-3440-3451.zip [^] (8,390 bytes) 2010-05-22 07:40
png file icon 3451_exec_01.png [^] (65,174 bytes) 2010-05-24 14:42


png file icon 3451_exec_02.png [^] (43,161 bytes) 2010-05-24 14:43

- Relationships
duplicate of 0003427closedasimon Query metrics displays invalid data 
related to 0003486closedasimon In the execute page, tree doesn't update when you change "Test Plan" in the "Setting" panel 
has duplicate 0003448closedfman Test Report generates all results as "Not Run" 
related to 0003423acknowledgedfman Test Report show all Testcase of the Test Project, not the selected ones in the tree 

-  Notes
(0010095)
fman (administrator)
2010-05-20 21:32

please use attached fix, retest and give feedback
(0010104)
otto_hsu (reporter)
2010-05-21 10:41

fix-3451 work partially. Please see my attachment for details.(testlink_p3451.jpg)
testlink 1.9(beta4)
(0010115)
fman (administrator)
2010-05-21 12:17
edited on: 2010-05-21 12:18

@otto_hsu:
please is not always true that an image is worth 1000 words, then normally na image is additional info, with textual description been necessary.
Please reload image, it was canceled because there was no info about how to use ot.
then:
Please explain and add image

(0010117)
johny2000uwb (reporter)
2010-05-21 13:10

I'm going to try explain this problem. I have the same problem too. I prepared testcase, add this testcase to testplan.
I run this testcase a set faild.

This test is showed as "Not Run" at this report.
Last Result Not Run

I tried to use your attached fix, but this problem is still exists.
(0010118)
rsteele (reporter)
2010-05-21 13:12

I concur with the previous user, I tried the new patch and all my passed and failed test cases still report as Not Run.
(0010121)
schasez (reporter)
2010-05-21 14:59

Same as 3427 I also opened
(0010129)
fman (administrator)
2010-05-22 07:42

Please use NEW attached fix, retest and let us know.
(0010134)
johny2000uwb (reporter)
2010-05-24 07:56

I tried it, but it not works again.

At the "Test REport" evert TC has Last result as "Not run".
(0010136)
fman (administrator)
2010-05-24 09:21

Reminder sent to: Julian

may I ask you to use fix on 1.9 beta 4 and give me feedback ?
(0010137)
Julian (reporter)
2010-05-24 10:41

does not work for me either.
(0010138)
johny2000uwb (reporter)
2010-05-24 11:46

I tried this fix on 1.9 beta 4 version. It's not work.
(0010139)
Falcon1 (reporter)
2010-05-24 14:42

I have the 1.9 beta 4, and I applied the attached patch. Now the result seems correct, but the information about build/release is totally wrong. I have many projects, test plan and build/release. Sometimes it indicates a build that doesn't exist on that test plan, but on a test plan of another project. So I made many tests and it seems that the first execution has problem.
Here the detail of a user case carried out.
1. Create a new test plan copying from an existing one with one test suite and two test cases.
2. Create two build 2.0.0 and 2.1.0.
3. Execute the test cases: first test case passed and second test cases failed.
4. The execution on the Execute section seems ok.
5. Logout and login.
6. If you go on the Execute section, the result is shown in the attached image 3451_exec_01.png. In the tree the result is correct, but in the main section there is no result.
7. In the report the Last Result is correct but the Build is wrong, as you can see in the attached image 3451_exec_02.png. The build 1.1.0 doesn't exist on that Test Plan.
(0010140)
fman (administrator)
2010-05-24 20:50

@Falcon1:
I apologize for all these problems, and thanks you a lot for your testing.
I will work on this issue with maximun priority, and will ask you for more patience and more tests when next fix will be ready.

Thanks a lot
(0010142)
Falcon1 (reporter)
2010-05-25 09:20
edited on: 2010-05-25 11:07

I did further investigation about my use case, checking the data saved in the database. I found that although in the Setting section it was selected Test Plan 03 and 2.0.0 in the database the data were saved with another build_id. So the problem was in saving data. I created a new project with new test suite and new test cases and now it seems correct. The results displayed with attached patch will be correct if you choose the entire Test Plan, but if you choose a Test Suite in the tree it will show Not run.
I think the problem is in the left section in the execute page. Although the check box Update tree after every operation is selected, sometimes the tree on the lower left part doesn't update correctly, and it can happen that the information stored in the database is wrong (created 0003486). I did these new tests clicking on the button Apply on section filters every time, with 5 Test plan, executing tests on one or two build, and more that one execution on some build.
ALL the results are correct now, but only if you choose the entire Test Plan. If you choose a Test Suite in the tree, it will show Not run. Moreover, it include all test cases of that Test Suite, including those not associated to that Test Plan.

A note about Test report
I found out that I can generate reports only about the last run for each Test Plan. So if I have two builds, for example 1.0.0 and 1.1.0 and the last execution was done on the 1.1.0, I cannot generate reports about execution on the 1.0.0 build. Moreover, if I execute the test cases again on the build 1.0.0, I will be able to generate reports only on this execution, and won't be able to generate reports about build 1.1.0 or the previous execution on the build 1.0.0.

I hope it is clear.

(0010149)
Belatu-Cadros (reporter)
2010-05-25 20:58

Hello,

I've applied the fix, here is what I have observed so far...

When the root of the tree is chosen for the test report, all results are reported fine.

If a test suite is chosen from the tree for the test report, all results are reported as "Not Run"
(0010200)
johny2000uwb (reporter)
2010-05-31 08:12

I agree with Belatu-Cadros. I have the same behavior.
(0010205)
skilletaudio (reporter)
2010-06-02 00:32

Greetings,

Additional feedback on the recent fix:

I have applied fix-3440-3451.zip, created a new test plan, assigned platforms A,B,C to the plan. Assigned test case X to the test plan for platforms A,B,C.

I executed test case X, specifying platform A. I generated the test report by clicking on the root of the tree (As indicated by other reporters).

Result:
The generated report shows the result of the test, but the result for test case X on platform A is also reported as the test result for test case X on platforms B and C, even though I haven't executed the test case for those platforms yet.
(0010360)
penkosel (reporter)
2010-06-18 16:11

Is there an update on this issue? I can't continue to use a tool that reports results incorrectly. We're nearing the end of a pilot phase and I'm having to tell our evaluators that the tool is great - except we can't get correct metrics out of it.

This goes over like a lead balloon in test circles.
(0010361)
fman (administrator)
2010-06-18 16:53

Dev team is working on this, and expect to have a fix in one or two weeks, because a big refactoring on filtering is near the end phase, and have impacts on this.
(0010668)
ktharp (reporter)
2010-07-15 23:36

I have installed the 1.9 Beta 5 and am still having the issues mentioned here. The duplicate 3448 shows as being repaired in beta 5, but I am still showing all test cases as not run when attempting to get any kind (HTML, word...) of Test Report.
(0010741)
asimon (developer)
2010-07-23 10:10

I finally found the error causing the "not run" results on every test report.
It is fixed in CVS now, please retest with next public beta and feel free to reopen issue if the bug should still persist.
(0011129)
fman (administrator)
2010-08-31 20:28

Release BETA 6 - 20100831

- Issue History
Date Modified Username Field Change
2010-05-11 20:49 Falcon1 New Issue
2010-05-18 21:03 fman Relationship added has duplicate 0003448
2010-05-20 21:02 fman Assigned To => fman
2010-05-20 21:02 fman Status new => acknowledged
2010-05-20 21:31 fman File Added: fix-3451.zip
2010-05-20 21:32 fman Note Added: 0010095
2010-05-20 21:32 fman Status acknowledged => feedback
2010-05-21 10:40 otto_hsu File Added: testlink_p3451.jpg
2010-05-21 10:41 fman File Deleted: testlink_p3451.jpg
2010-05-21 10:41 otto_hsu Note Added: 0010104
2010-05-21 12:17 fman Note Added: 0010115
2010-05-21 12:18 fman Note Edited: 0010115 View Revisions
2010-05-21 13:10 johny2000uwb Note Added: 0010117
2010-05-21 13:12 rsteele Note Added: 0010118
2010-05-21 14:59 schasez Note Added: 0010121
2010-05-21 15:53 fman Relationship added duplicate of 0003427
2010-05-22 07:40 fman File Deleted: fix-3451.zip
2010-05-22 07:40 fman File Added: fix-3440-3451.zip
2010-05-22 07:41 fman Status feedback => assigned
2010-05-22 07:42 fman Note Added: 0010129
2010-05-22 07:42 fman Status assigned => feedback
2010-05-24 07:56 johny2000uwb Note Added: 0010134
2010-05-24 09:21 fman Note Added: 0010136
2010-05-24 10:41 Julian Note Added: 0010137
2010-05-24 11:46 johny2000uwb Note Added: 0010138
2010-05-24 14:42 Falcon1 Note Added: 0010139
2010-05-24 14:42 Falcon1 Status feedback => assigned
2010-05-24 14:42 Falcon1 File Added: 3451_exec_01.png
2010-05-24 14:43 Falcon1 File Added: 3451_exec_02.png
2010-05-24 20:50 fman Note Added: 0010140
2010-05-25 09:20 Falcon1 Note Added: 0010142
2010-05-25 11:07 Falcon1 Note Edited: 0010142 View Revisions
2010-05-25 18:08 fman Relationship added related to 0003486
2010-05-25 20:58 Belatu-Cadros Note Added: 0010149
2010-05-26 16:09 asimon Assigned To fman => asimon
2010-05-26 16:09 asimon Category Reports => asimon_thesis
2010-05-27 20:52 fman Relationship added related to 0003423
2010-05-31 08:12 johny2000uwb Note Added: 0010200
2010-06-02 00:32 skilletaudio Note Added: 0010205
2010-06-18 16:11 penkosel Note Added: 0010360
2010-06-18 16:53 fman Note Added: 0010361
2010-07-15 23:36 ktharp Note Added: 0010668
2010-07-23 10:10 asimon Note Added: 0010741
2010-07-23 10:10 asimon Status assigned => resolved
2010-07-23 10:10 asimon Fixed in Version => 1.9 Beta 6
2010-08-31 20:28 fman Note Added: 0011129
2010-08-31 20:28 fman Status resolved => closed



Copyright © 2000 - 2020 MantisBT Team
Powered by Mantis Bugtracker