Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004064TestLinkNew Featurepublic2010-11-30 12:382014-07-28 20:31
Reportertom1 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityN/A
StatusclosedResolutionfixed 
PlatformLinuxOSUbuntuOS Version10.04
Product Version1.9 (Prague) 
Fixed in Version1.9.11 (2014 Q2 - bug fixing) 
Summary0004064: [TEST STEP RESULT] - Additional test result field for each test step
DescriptionIf you execute a test case manually it would be helpful to add a test note or result for each test step. So you can go through your test procedure step by step and write down the result/note for each step. Nevertheless the overall test note is helpful to note the final state of execution.
I propose to add a field "Test Result" or "Test Note" for each step. This field should be editable on test execution and be part of the table in the test report.
TagsNo tags attached.
Database (MySQL,Postgres,etc)mysql 5.1.41
BrowserFirefox
PHP Version5.3.2
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Files

- Relationships
duplicate of 0004778closedfman [TEST STEP RESULT] - Need Test Step wise Execution and Actual Resut for test case 

-  Notes
(0012773)
fman (administrator)
2010-11-30 21:13
edited on: 2010-11-30 21:16

is not clear if this note is part of test case spec, or part of test case execution.
Adding a note for each test case step on execution, needs lot of refactoring because have inmpacts on reports, and exec history

my vote is do not implement.

(0012776)
Julian (reporter)
2010-11-30 21:57

collect results of each step on notes.
i also do not see a big advantage compared to the effort mentioned by fman.

provide us with some use case that includes:
- set result for each step
- make use of these results

if you think this feature adds value to testlink
(0012777)
tom1 (reporter)
2010-12-01 08:24

During test execution I have to go step by step through the test procedure of a manual test. The test procedure includes execution steps and the expected results. During the execution I would like to note for each step the execution result.

Example:
Factory acceptance test with our customer. Demonstration of a manual test.
I'm sitting on one of our target machines and go through the test steps.

Step 1: Press button x
Expected result: Icon color changes to green
Test Result: Ok

Step 2: Insert value x in field y and press enter
Expected result: Display of calculated value z
Test result: Displayed value differs from z

etc.

Summary test result could be passed with conditions.

During the presentation I could show the steps and insert the results online.
Such a feature is provided by the HP test director. A popup window shows you the test procedure and you can insert the results per step.

The generated test report should than also include an additional column with the results in the table with the steps.

This feature would be very helpful for my work. Unlikely I have a lot of manual tests and the main task is acceptance testing in front of the customers.
(0012778)
Julian (reporter)
2010-12-01 08:32

this case is very special in my eyes and will make testing for most of the people less comfortable if every step requires user input.

maybe you find a workaround like misusing test suite as test case -> every test case is a test step. when executing you can use "save and move to next" button.

i do not know how many tests you are showing. but for demo purposes this could maybe help.
(0012779)
tom1 (reporter)
2010-12-01 09:07

A normal test session takes 1 or 2 weeks and my test team presents about 100 test cases to our customer. Ok, this is maybe not a common use case and I try also to establish QF-Test for automatic testing. But our customers requires a online presentation of a least some test cases.
Nevertheless thanks for your attention. I worked a lot with testlink the last weeks and tried to fit it in our development process. I like it a lot and I was successful in most of the cases. I wrote some DXL script for data exchange between Doors and testlink and tried to use the XMLRPC API to connect some tools for test automation. Let me point here your attention to another issue ;-)
http://mantis.testlink.org/view.php?id=4021 [^]
(0012794)
fman (administrator)
2010-12-01 21:36

On past we have had some discussion on saving exec result AT STEP LEVEL, but amount of effort to do this and impacts on lot of areas (probably also on performance), IMHO makes this kind of issue something to do not implement.
(0013840)
alonzo (reporter)
2011-03-10 12:06

Hi,
We are trying to use this tool at our company named Sapiens.
Currently the main issue is with step status and its past.

At our daily tests, we encounter a lot the situation when we can complete only parts of the test case and not necessarily in normal order.

It is actually got to the point where the manual testers says that this is mandatory for normal day to day use.

Unless having this feature, most testers do not want to use this tool.

Therefore, I support greatly having this feature developed.

Many thanks,
Alonzo.
(0013849)
fman (administrator)
2011-03-10 18:11

Your support is great only if you can contribute with development.
(0013878)
kai (reporter)
2011-03-16 11:01

it's rather a question of granularity of testcases
as said: results per steps will make testing for most of the people less comfortable if every step requires user input.
(0014300)
ns20 (reporter)
2011-03-26 10:05

I completely support tom1 and alonzo position. I consider that the bug is when the expected result doesn't correspond to the received. Therefore there should be a separate mark for each step. As there should be a column with the description of a bug for each step. alonzo,tom, you began to use other product?
(0014302)
fman (administrator)
2011-03-26 13:03

>> alonzo,tom, you began to use other product?
how we do have to interpreted this ?
please explain
(0014303)
difool (reporter)
2011-03-26 13:16

This question also arose in my QA team.
It's really a matter of granularity as already said.

The solution is to limit steps count for each test case to less than 10, mainly between 3 to 6. If you need more then divide your test case into for exeample: Test case 1 - Init, Test case 2 - Execution, Test case 3 - Finalization; or Test case - phase 1, Test case - phase 2, Test case - phase 3.

Notes associated for execution of tests cases is enough because they are principally used whenever the test fail and then writing down the step which fail.
(0014306)
alonzo (reporter)
2011-03-27 09:05

Hi, we are still using the tool and haven't change it.

We are currently doing a workaround.
We have created a template for each step. At "test Action" we have created a table with 2 lines.Line 1 is for the actions to perform and Line 2 is for the expected results.
At the "Expected Results" we have inserted a table of 5 columns that will hold information about the step (Pass/Fail) and the history of that. 5 columns are for #, date of run, test data, status, bug number.

This is how we have managed to work with the tool.

We give a status for the whole test case when it is fully done but we lose the option to get statistics on that.

Alonzo.
(0015871)
sqaopen (reporter)
2011-10-03 13:14

Hello !

+1 for this RFC (Request For Change).

We need (the SQA people) to have the ability to define a result at the Test Step level when running our Test Cases. It is not, in my opinion, a problem of granularity since, for each Steps, you have an attendee. Then, for each step, you could have a result "Fail" or "Pass". A step could be "Fail" but not critical so you can run the all Test Case, and not stop at the failed step. Then, it is (in my opinion), mandatory to specify which step is failed.

It could have another positive impact : if you import TAP files with the results of automated tests, you have the ability to specify steps result automaticaly.

I've seen some comparisons between TestLink and others products where we can read that it miss the possibility to specify the result at a step level.

Juste an exemple, about GUI testing.
If you have to describe the way to test the GUI of a web application, you can create a test case about the launching, the connection...
But to describe the testing of the main screen, you will have only 1 test case with some steps, each step describing the control of a component (button, label, etc). You will not have one test case for each button, label, of the main screen. Then, if a label is not correct, the step will fail but the other steps could be passed. You will have a result "Fail" for the entire Test Case and could be not able to specify which step fails. But it is really important to be able to do this.
(0015918)
GABQA (reporter)
2011-10-25 15:01

My vote for this request, too!!
I propose naming it "FeedBack" and letting it update with Pass/ Fail/ Remarks
And letting insert some comment & images.
(0016417)
mbeccaria (reporter)
2012-03-05 15:13
edited on: 2012-03-05 15:16

Hello
my experience was to work with test procedures implemented as tables, one step for each row. The columns in the table were typically:
- step #
- step description
- expected results
- actual result

The column actual result was very often resulting as a "tick mark" to say that the step was passed.

I think this should be implemented with testLink, if we had the possibility to link custom fields to test steps.

(0016800)
chiefwiggum (reporter)
2012-05-30 10:33

Hello :)

I'd also like to add my +1 to this, from a test execution perspective, it makes no difference to me if tests pass or fail at test case or test step. But from a reporting perspective, i'm contantly being asked for failed test step reports, i've tried using custom fields to capture this, but its just not giving the detail needed. rather than recoding loads of TL couldn't custom fields be amended to appear at step level? giving users the flexibility to add what they want.

The DB code is already there for custom fields, so surely its a small code change to show at step level?
(0016801)
fman (administrator)
2012-05-30 12:10

>> ...so surely its a small code change to show at step level?
ok if you have checked that change is so small, we can accept your contributions.
Do not forget to add also changes to all available reports

thanks
(0016802)
chiefwiggum (reporter)
2012-05-30 13:10

TBH i've not checked that it is that easy - i'm just assmuing it is! ;-) Please put me straight if i'm wrong.

It just seems to me, that this suggestion is an easier solution to this scenario, rather than ripping the code appart and making huge changes which will only satisfy a handful of people. :)
(0017719)
nepomuk3000 (reporter)
2012-10-23 13:39

Hello,

The lack of this fuctionnality is the only think why I wont use TestLink and ask for my compagny to buy Quality Center. It's a pity.
(0018024)
Frank (reporter)
2012-12-11 20:35

Is there any update as if this issue has been or will be added to the product?
(0018248)
JulienBreton (reporter)
2013-01-30 14:05

My vote for this request, too!!
(0018262)
chemchoum (reporter)
2013-02-03 02:34

Hi,
I’m working on a very big project and I’m trying to convict all groups to use TestLink as a test tool management but some of them are reticent because results by steps are not supported.
We used to work with steps as follow:
- Steps by default are NT= Note Tested
- The result of a step can be pass, fail or not tested (for example by using multiselection list next to the expected result of each step)
- we can save a partial execution of a test case if only some steps has been executed and finish the execution of the other steps after on.

So I vote for this request too.
(0018263)
fman (administrator)
2013-02-03 09:13

see note (0012794)
(0018991)
Frank (reporter)
2013-05-21 14:52

Based on note (0012794) - Will this feature be added or not. Should we look at other alternatives? As this product is configured today, it is unuseable without the features requested on this issue (0004064) and with the duplicate issue (0004778).
(0018994)
fman (administrator)
2013-05-21 15:34

we are not going to implement this.

- Issue History
Date Modified Username Field Change
2010-11-30 12:38 tom1 New Issue
2010-11-30 21:13 fman Note Added: 0012773
2010-11-30 21:16 fman Note Edited: 0012773 View Revisions
2010-11-30 21:57 Julian Note Added: 0012776
2010-12-01 08:24 tom1 Note Added: 0012777
2010-12-01 08:32 Julian Note Added: 0012778
2010-12-01 09:07 tom1 Note Added: 0012779
2010-12-01 21:36 fman Note Added: 0012794
2011-03-10 12:06 alonzo Note Added: 0013840
2011-03-10 18:11 fman Note Added: 0013849
2011-03-16 11:01 kai Note Added: 0013878
2011-03-26 10:05 ns20 Note Added: 0014300
2011-03-26 13:03 fman Note Added: 0014302
2011-03-26 13:16 difool Note Added: 0014303
2011-03-27 09:05 alonzo Note Added: 0014306
2011-10-03 13:14 sqaopen Note Added: 0015871
2011-10-20 16:33 fman Relationship added duplicate of 0004778
2011-10-25 15:01 GABQA Note Added: 0015918
2012-03-05 15:13 mbeccaria Note Added: 0016417
2012-03-05 15:16 mbeccaria Note Edited: 0016417 View Revisions
2012-05-30 10:33 chiefwiggum Note Added: 0016800
2012-05-30 12:10 fman Note Added: 0016801
2012-05-30 13:10 chiefwiggum Note Added: 0016802
2012-09-30 19:00 Frank Note Added: 0017595
2012-10-01 21:04 fman Note Added: 0017601
2012-10-01 21:05 fman Note Deleted: 0017595
2012-10-01 21:05 fman Note Deleted: 0017601
2012-10-23 13:39 nepomuk3000 Note Added: 0017719
2012-11-02 08:42 fman Task Workflow Status => TBD
2012-11-02 08:42 fman Summary Additional test result filed for each test step => [TEST STEP RESULT] - Additional test result filed for each test step
2012-12-11 20:35 Frank Note Added: 0018024
2013-01-30 14:05 JulienBreton Note Added: 0018248
2013-01-30 14:22 fman Summary [TEST STEP RESULT] - Additional test result filed for each test step => [TEST STEP RESULT] - Additional test result field for each test step
2013-02-03 02:34 chemchoum Note Added: 0018262
2013-02-03 09:13 fman Note Added: 0018263
2013-05-21 14:52 Frank Note Added: 0018991
2013-05-21 15:34 fman Note Added: 0018994
2014-07-27 10:00 fman QA Team - Task Workflow Status TBD => READY FOR TESTING
2014-07-27 10:00 fman Status new => resolved
2014-07-27 10:00 fman Fixed in Version => 1.9.11 (2014 Q2 - bug fixing)
2014-07-27 10:00 fman Resolution open => fixed
2014-07-27 10:00 fman Assigned To => fman
2014-07-28 20:31 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker