Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007288TestLinkTest Executepublic2015-10-08 13:082015-10-12 21:21
ReporterArteliar 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityalways
StatusassignedResolutionopen 
PlatformOSOS Version
Product Version1.9.14 (2015 Q3) 
Fixed in Version 
Summary0007288: Problem with Execution order and mixed TC IDs between Test Sets
DescriptionI have a problem in my project with Execution order. On the picture [b]Add Test Cases.jpg[/b] u can noticed, that I set 0 to Execution order to all tests from Test Set [b]Ustawienia[/b] and 99999 Execution order to all tests from [b]Symulacja[/b] set. So i expect that all tests from [b]Ustawienia[/b] set will be before this from [b]Symulacja[/b].
And the problem is presented on picture [b]Execute Tests.jpg[/b]. As you can see, 2 tests from [b]Ustawienia[/b] are placed after whole [b]Symulacja[/b] set.
I think that it's caused by TC's ID and that [b]Symulacja[/b] set is belong to [b]Ustawienia[/b] set. As you can notice two TC which are the last have ID [b]3555[/b] and [b]3556[/b]. And the first TC in [b]Symulacja[/b] set have ID = [b]3554[/b]. Coz of that im not able to set order that i want.

I hope I explained it clear enough and you can understand my problem.
TagsNo tags attached.
Database (MySQL,Postgres,etc)ALL
Browser
PHP Version
TestCaseID
QA Team - Task Workflow StatusTBD
Attached Filesjpg file icon Add Test Cases.jpg [^] (275,797 bytes) 2015-10-08 13:08
jpg file icon Execute Tests.jpg [^] (53,116 bytes) 2015-10-08 13:10


jpg file icon case 1.jpg [^] (123,784 bytes) 2015-10-09 07:30


jpg file icon case 2.jpg [^] (126,144 bytes) 2015-10-09 07:31


jpg file icon case 3.jpg [^] (122,842 bytes) 2015-10-09 07:31


jpg file icon case 4.jpg [^] (131,122 bytes) 2015-10-09 07:31


jpg file icon case 5.jpg [^] (131,117 bytes) 2015-10-09 07:31


jpg file icon case 6.jpg [^] (131,223 bytes) 2015-10-09 07:32


jpg file icon cases.jpg [^] (152,440 bytes) 2015-10-09 07:32

- Relationships

-  Notes
(0024104)
fman (administrator)
2015-10-08 16:58
edited on: 2015-10-08 17:01

Test case order is intended INSIDE it's test suite, and has no impact on test suite order on tree

explanation is simpler: test suite order by default is set to ZERO.
TRy to set for test cases exec order to 1, and recheck

please do not use markup that is NOT HMTL because makes text impossible to read

(0024105)
Arteliar (reporter)
2015-10-09 07:41

I made a new project with just few TC to make situation clear and easy to reproduce.
First I added 7 TC to this project:
 - TC-1A
 - TC-1B
 - TC-1C
 - TC-1D
 - TC-1E
 - TC-1F
 - TC-1G
Then I made 2 TS in my main TS called TS-1:
 - TS-1-1
 - TS-1-2

After that I prepered structure by moving TC to TS and my structure looks like that:
 + TS-1
 +-+ TC-1A
   + TC-1B
   + TC-1E
   + TC-1G
   + TS-1-1
   +-+ TC-1C
   | + TC-1F
   + TS-1-2
   +-+ TC-1D

Then I start to test execute order. On the pictures called "case X.jpg" you have orders that i tested and on picture "cases.jpg" you have results of this order configurations. As you can see CASE 1 and CASE 2 were wrong, but CASE 3 was correct for my structure. So the next step that I did was adding new TC called TC-1H and i put it into main TS, so my new structure looks like that:

 + TS-1
 +-+ TC-1A
   + TC-1B
   + TC-1E
   + TC-1G
   + TC-1H (added TC)
   + TS-1-1
   +-+ TC-1C
   | + TC-1F
   + TS-1-2
   +-+ TC-1D

After that I wasnt able to make correct order like I have in above structure. Cases CASE 4, CASE 5 and CASE 6 show different configurations of order, but none of them is correct. I think problem is that I made TC-1H after this two TS (TS-1-1 and TS-1-2).

P.S.
Dunno why I used this tags in problem description and sorry for that. I saw that it is hard to read after I posted topic, but I couldnt find possibility to edit it. Or at least preview before post would help alot, but couldnt find it aswell.
(0024110)
fman (administrator)
2015-10-10 08:37

In addition to add more info (only way to have a chance to solve an issue) have you done what I've requested?
Ignoring requests for checkings for dev team is not a goog thing
(0024111)
fman (administrator)
2015-10-10 09:28

a test to be useful for the dev team (not for the reporter) needs

1) detailed steps to reproduce
2) expected results
3) actual results

is clear from dev team point of view that written explanation (when things go wrong) are needed in addition to images.
You can not expect I go back & forth from one image to another to understand what i wrong.
(0024112)
fman (administrator)
2015-10-10 09:29

more detailed information and explanations are needed to do any actions.
(0024113)
Arteliar (reporter)
2015-10-12 08:54

Seems like I did not understand your request at first time and checked it in wrong way. Now I set 1 for every TC that I have and it's looks fine.

But still I don't get this mechanism. I'll try to explain what I mean on example.

Steps to reproduce:
1. First make a simple structure as I did above in comment 0024105 (hope there is explained clear enough to reproduce structure)
2. Create Test Plan (TP-01) and Test Release (TR-01)
3. Go to Add/Remove Test Cases
4. Add whole structure to TR-01 (order that is set is presented on picture case 1.jpg)
5. Go to Execute tests
Expected result:
I expect, that i will see same tree on left side as I have in Test Specification(same as I have above in comment 0024105)
Actual result:
Actual result is showed on picture cases.jpg as img CASE 1

The question is why default order is so wired? I mean why in TS-1-1 first test have setted order value on 20 and second one have setted on 50?

And second example
Steps to reproduce:
1. Same steps as in first example without 5th step (do steps 1. to 4.)
2. In Add/Remove Test Cases set and save execute order to same as on picture case 2.jpg
3. Go to Execute tests
Expected result:
Same structure as showed in comment 0024105
Actual result:
Actual result is showed on picture cases.jpg as img CASE 2 - order is mixed

Hope I gave you enough details to reproduce problem. Or if it is not bug and it have to work like that, then please explain me why this order is mixed and doesnt looks like I expect?

- Issue History
Date Modified Username Field Change
2015-10-08 13:08 Arteliar New Issue
2015-10-08 13:08 Arteliar File Added: Add Test Cases.jpg
2015-10-08 13:10 Arteliar File Added: Execute Tests.jpg
2015-10-08 16:58 fman Note Added: 0024104
2015-10-08 17:01 fman Note Edited: 0024104 View Revisions
2015-10-08 17:01 fman Assigned To => fman
2015-10-08 17:01 fman Status new => feedback
2015-10-09 07:30 Arteliar File Added: case 1.jpg
2015-10-09 07:31 Arteliar File Added: case 2.jpg
2015-10-09 07:31 Arteliar File Added: case 3.jpg
2015-10-09 07:31 Arteliar File Added: case 4.jpg
2015-10-09 07:31 Arteliar File Added: case 5.jpg
2015-10-09 07:32 Arteliar File Added: case 6.jpg
2015-10-09 07:32 Arteliar File Added: cases.jpg
2015-10-09 07:41 Arteliar Note Added: 0024105
2015-10-09 07:41 Arteliar Status feedback => assigned
2015-10-10 08:37 fman Note Added: 0024110
2015-10-10 09:28 fman Note Added: 0024111
2015-10-10 09:29 fman Note Added: 0024112
2015-10-10 09:29 fman Status assigned => feedback
2015-10-12 08:54 Arteliar Note Added: 0024113
2015-10-12 08:54 Arteliar Status feedback => assigned
2015-10-12 21:21 fman QA Team - Task Workflow Status => TBD
2015-10-12 21:21 fman Priority urgent => normal



Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker