Anonymous | Login | Signup for a new account | 2019-12-10 21:39 UTC | ![]() |
Main | My View | View Issues | Change Log | My Account |
View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
0001310 | TestLink | Test Specification | public | 2008-01-22 15:28 | 2014-07-28 20:33 | ||||
Reporter | francisl | ||||||||
Assigned To | fman | ||||||||
Priority | high | Severity | feature request | Reproducibility | N/A | ||||
Status | closed | Resolution | fixed | ||||||
Platform | OS | OS Version | |||||||
Product Version | |||||||||
Fixed in Version | 1.9.11 (2014 Q2 - bug fixing) | ||||||||
Summary | 0001310: [multiple steps] - Multiple test steps and expected results | ||||||||
Description | 1. Usually, test case consists of multiple test steps. Each test step associated with an expected results. It would be useful if TL could support this characteristics of test case. 2. If point 1 above is implemented, it would be useful if a test case can refer to another test case which acts as a container of some common test steps. This is similar to the concept of php include files or subprograms. | ||||||||
Tags | Attachment | ||||||||
Database (MySQL,Postgres,etc) | |||||||||
Browser | |||||||||
PHP Version | |||||||||
TestCaseID | |||||||||
QA Team - Task Workflow Status | READY FOR TESTING | ||||||||
Attached Files | ![]() | ||||||||
![]() |
||||||||||||||||||||||||||||||||||||
|
![]() |
|
(0002996) fman (administrator) 2008-01-22 15:58 |
1. TL architecture will not allow TestCase to be a TestCase container 2. Please next time look for similar issues before reporting. 3. Today you have a workaroung -> create tables inside steps and expected results. 4. ISSUE is: Any step will have it's result during execution ? if anwser is yes, Which will be the algorithm to compute TestCase exec status using steps status ? Need deep discussion |
(0003021) francisl (reporter) 2008-01-23 14:22 edited on: 2008-01-23 14:22 |
1. Sorry, I don't understand this point. Need more clarification. Please mail me. 2. I have searched Mantis but found not similar issue. Need to improve my searching skill. 3. Interesting workaround. However, it is not different to use number list to associated steps and expected results. 4. Yes, each step could have its execution result. The rules I used to determine final results are: a) test failed if any of the steps failed; b) test result overridden if justifications supplied by tester. I am happy to further discuss this request as I would like to see TL works better for testing. |
(0003024) fman (administrator) 2008-01-23 15:26 |
1. TL architecture will not allow TestCase to be a TestCase container: means if you analise Database schema (or read old doc in document repository on CVS) you will find that TC can contain just TCversions. We will not change this. |
(0003027) francisl (reporter) 2008-01-23 15:54 edited on: 2008-01-23 15:59 |
Thank you for the explanation. So, I assume it refers to item 2 in my suggested features. 1. The request is based on good faith and my experience. 2. Version of TC is certainly important for configuration control. 3. No matter we like it or not, the fact is that it is very common to have a set of steps repeating in many test cases. 4. Tools should facilitate works instead of bending work to fit a tool. 5. Reading database schema is rare while evaluating a test tool. By the way, it is also a good idea to able to group steps as suggested in the other request. |
(0003028) fman (administrator) 2008-01-23 16:29 |
>> So, I assume it refers to item 2 in my suggested features. yes >> 1. The request is based on good faith and my experience. no doubts. >> 2. Version of TC is certainly important for configuration control. >> 3. No matter we like it or not, the fact is that it is very common to have a set of steps repeating in many test cases. OK, then you can COPY common steps on new testcases and add new one. >> 4. Tools should facilitate works instead of bending work to fit a tool. I do not fully agree, because sometime is not possible. who can THE VISION when trying to develop a product/solution ? >> 5. Reading database schema is rare while evaluating a test tool. I'm just explaining why can not be done. Need more detailed info (examples, ecc) in order to try to evaluate a solution. (not in the short time) regards |
(0003029) fman (administrator) 2008-01-23 16:38 |
TARGET RELEASE: 1.9 |
(0003120) olivier_houdas (reporter) 2008-01-29 18:20 |
If the ability to sort the TC execution order inside a Testplan is given, wouldn't this be a good way to address the issue ? |
(0003121) fman (administrator) 2008-01-29 18:36 |
>> If the ability to sort the TC execution order inside a Testplan is given, >> wouldn't this be a good way to address the issue ? 1. can not be done (in short/mid time) due to: a. lot of changes at DB level b. not as simple as seems due to testcase link to testcase implementation. 2. will be just a workaround, (used once for a customer) where you consider Test Suite ad object that will have status. 3. if we want steps inside testcase, we need to implement it as clean as possible |
(0003147) francisl (reporter) 2008-01-30 07:40 |
> who can THE VISION when trying to develop a product/solution ? I have a vision but I don't want to force it on other people's project. :-) > I'm just explaining why can not be done. Sorry, I misunderstood your meaning. :-( > If the ability to sort the TC execution order inside a Testplan is given, wouldn't this be a good way to address the issue ? Don't understand "to sort the TC execution order". Need clarification. |
(0003148) fman (administrator) 2008-01-30 15:45 |
1. I'm wrong: test cases execution order = test case order inside test suite. 2. >>I do not fully agree, because sometime is not possible. >>who can THE VISION when trying to develop a product/solution ? wrong english: who can have THE right VISION when trying to develop a product/solution ? |
(0003149) fman (administrator) 2008-01-30 16:04 |
@ olivier_houdas : I've think over your idea. I will have some documentation during next week |
(0003203) olivier_houdas (reporter) 2008-02-07 17:43 |
Well, I am new to Testlink, and I just discovered that you can order TCs and Testsuites. So in fact, I found the solution to what I needed. Regarding the present MANTIS item, isn't it the solution : - A test case with multiple steps should in fact just be a sub-test suite; - Inside the sub-test suite, you can create as many TCs as there are steps; - If you forgot one step, you just need to add it at the end, then click on the Reorder children button, which is available when you click on the sub test suite item in the tree. francisl, isn't it what you were actually looking for ? |
(0003296) francisl (reporter) 2008-02-15 21:42 |
Hello, olivier_houdas, Before I answer your question, I would like to clarify the relationships of these terms. 1. A test suite (test set) consist of multiple tests and/or another test suite; 2. A test consists of a test case and test data. 3. A test case consists of multiple test steps and expected results. Based on TestLink's sample project, it satisfies point 1 above. Certainly, you can use test suite to document a test and use test case to document a test step. But, this does not improve the usability of the tool. |
(0003711) jorgec (reporter) 2008-06-19 03:50 |
joc - 06/18/2008: Definitely we need multi-steps within same test case as other TMF are supporting now i.e.~SalomeTMF, aqManager, etc. |
(0003727) talame (reporter) 2008-06-26 01:59 |
There are two ways Test Case forms, the top-down approach and the side-by-side approach. Top-dowjn Approach: Steps: Step 1. Step 2. Step n. Expected Result: (expected result) Side-by-Side approach: Steps: Expected Result: ----- ---------------- 1.Enter Username, type: XYZ 1. Username displayed 2.Enter Password, type: abc 2. Password masked. 3.Click on the Submit button 3. The <TATA> screen is displayed. Which ever approach is dependented on your needs. I switch between both. |
(0003729) mhavlat (reporter) 2008-06-26 15:39 |
I can confirm that it is in scope of TestLink 1.9. My view: REQ 1 User can choose TC type: htmltext steps/expctedResults, simpletext steps/expctedResults and step by step. REQ 2 Step by step design User can add/modify/delete rows that contain two simpletext areas for steps and expected results data. User can changes order of steps. We can implement basic html tags for the data according resources. FCKEditor will have special menu definition in this case. REQ 3 Step by step results Test case execution will define overall TC result and results for particular steps. The next four statuses are applicable for steps only: not run (default), pass, fail and blocked. Timestamp and a user_id is recorded for each step. User can fill test results and test note for particular steps. User can fill test results for all steps by one click. User can modify overall test results independently on steps. TestLink automatically modify overall test result if - all step PASSED - result FAILED is added to any step - result BLOCKED is added to any step and overall test result is Not run. REQ 4 Performance We should consider to show just one Test case in execution page (instead of list) to compass performance problems. All other pages should show just overall status. REQ 5 Rendering Test case content rendering should be extracted to extra templates per TC type. Database changes consideration There is required a new TC attribute tc_type. The simplest solution to store steps are using field steps to store all structure. The similar way is applicable to steps test results with the filed 'note'. Robust solution means to introduce two new tables tc_steps and tc_steps_result. Is somebody able to compare this draft with other tools? Do I miss something? |
(0003741) francisl (reporter) 2008-06-27 21:41 |
Below are my suggestions and comments: 1. REQ 3: Step by step results 1.1 Since it is rare to have steps executed by separate testers, there is no need to record timestamp and user Id for each step. Just record timestamp and user Id for each TC execution should be adequate. 1.2 I think the following rules should be used to determine the overall TC execution result: a) If all steps passed, TC execution result should be "Passed". b) If any step failed, TC execution should be terminated and results as "Failed". c) If any step blocked, TC execution should be terminated and results as "Blocked". d) If execution result is failed and justification is provided, results should be overwritten as "Passed". 1.3 It would be helpful if creator of a TC execution record is allowed to update actual results and notes. Tester may find typos in their record and need to correct them after the record is saved. 2. REQ 4: Performance 2.1 TC execution report should present results of last execution as default. However, user should be allowed to present all execution results of a TC. 3. Database changes 3.1 There may be situation that one step associated with multiple expected results. However, by allowing to use numbered list in the expected results should be able to resolve this. 3.2 It would be great if common/shared steps can be defined like a subroutine and called by TC's. For example, steps to login a system may be defined as common/shared steps. It can be called by various TC's to form a complete TC. This feature is available from TestDirector. |
(0006554) ssingh (reporter) 2009-04-23 22:46 |
See the step and results screen shot given in attached file Steps_Results - which will be perfect scenario to have |
(0006561) francisl (reporter) 2009-04-24 14:45 |
Regarding REQ 3, I would like to repeat my suggestion: The overall TC execution result should be: a) If all steps passed, TC execution result should be "Passed". b) If any step failed, TC execution should be terminated and results as "Failed". c) If any step blocked, TC execution should be terminated and results as "Blocked". d) If execution result is failed and justification is provided, results should be overwritten as "Passed". Occasionally, there is a need for scenario (d). |
(0009664) tarun3kumar (reporter) 2010-04-07 11:53 |
Having ability to capture results for different steps of test case is must. I am yet to dig in to Reporting capabilities of TestLink but I believe that it should be reflected in Test Reports also. I completely agree with francisl last comment on this issue. This issue was fist reported on -> 2008-01-22 07:28 and is still in Open Status. Is there any deadline towards fixing this? ~ T |
(0009665) fman (administrator) 2010-04-07 20:35 |
>> This issue was fist reported on -> 2008-01-22 07:28 >> and is still in Open Status. And will not completely fixed on 1.9 => result at step level will not be available due to amount of rework to be done. |
(0021004) fman (administrator) 2014-05-24 08:33 |
https://gitorious.org/testlink-ga/testlink-code/commit/480f91743d81e7d6efdef0dfa182eaaa13ecc979 [^] |
![]() |
|||
Date Modified | Username | Field | Change |
2008-01-22 15:28 | francisl | New Issue | |
2008-01-22 15:58 | fman | Note Added: 0002996 | |
2008-01-23 14:22 | francisl | Note Added: 0003021 | |
2008-01-23 14:22 | francisl | Note Edited: 0003021 | |
2008-01-23 15:26 | fman | Note Added: 0003024 | |
2008-01-23 15:29 | fman | Relationship added | related to 0000507 |
2008-01-23 15:29 | fman | Relationship added | related to 0000422 |
2008-01-23 15:54 | francisl | Note Added: 0003027 | |
2008-01-23 15:59 | francisl | Note Edited: 0003027 | |
2008-01-23 15:59 | francisl | Note Edited: 0003027 | |
2008-01-23 16:29 | fman | Note Added: 0003028 | |
2008-01-23 16:38 | fman | Status | new => assigned |
2008-01-23 16:38 | fman | Assigned To | => fman |
2008-01-23 16:38 | fman | Note Added: 0003029 | |
2008-01-29 18:20 | olivier_houdas | Note Added: 0003120 | |
2008-01-29 18:36 | fman | Note Added: 0003121 | |
2008-01-30 07:40 | francisl | Note Added: 0003147 | |
2008-01-30 15:45 | fman | Note Added: 0003148 | |
2008-01-30 16:04 | fman | Note Added: 0003149 | |
2008-01-30 21:02 | fman | Summary | Multiple test steps and expected results => [multiple steps] - Multiple test steps and expected results |
2008-02-07 17:43 | olivier_houdas | Note Added: 0003203 | |
2008-02-15 21:42 | francisl | Note Added: 0003296 | |
2008-06-19 03:50 | jorgec | Note Added: 0003711 | |
2008-06-26 01:59 | talame | Note Added: 0003727 | |
2008-06-26 15:39 | mhavlat | Note Added: 0003729 | |
2008-06-26 15:39 | mhavlat | Priority | normal => high |
2008-06-26 15:39 | mhavlat | Projection | none => redesign |
2008-06-26 15:39 | mhavlat | ETA | none => < 1 month |
2008-06-27 21:41 | francisl | Note Added: 0003741 | |
2009-04-02 21:46 | ssingh | Note Added: 0006178 | |
2009-04-08 16:57 | fman | Note Deleted: 0006178 | |
2009-04-23 22:43 | ssingh | Tag Attached: Attachment | |
2009-04-23 22:44 | ssingh | File Added: Steps_Results | |
2009-04-23 22:46 | ssingh | Note Added: 0006554 | |
2009-04-24 14:45 | francisl | Note Added: 0006561 | |
2009-05-14 14:50 | mhavlat | Relationship added | parent of 0000718 |
2009-05-31 01:36 | mhavlat | Relationship added | has duplicate 0002555 |
2009-09-06 01:06 | bighux | Tag Attached: test | |
2009-09-06 01:07 | bighux | Tag Detached: test | |
2010-04-07 11:53 | tarun3kumar | Note Added: 0009664 | |
2010-04-07 20:35 | fman | Note Added: 0009665 | |
2011-03-12 10:45 | Julian | Relationship added | parent of 0004318 |
2012-04-28 17:08 | fman | Relationship added | has duplicate 0004986 |
2014-05-24 08:33 | fman | QA Team - Task Workflow Status | => READY FOR TESTING |
2014-05-24 08:33 | fman | Note Added: 0021004 | |
2014-05-24 08:33 | fman | Status | assigned => resolved |
2014-05-24 08:33 | fman | Fixed in Version | => 1.9.11 (2014 Q2 - bug fixing) |
2014-05-24 08:33 | fman | Resolution | open => fixed |
2014-05-24 10:35 | fman | Relationship added | child of 0006313 |
2014-07-28 20:33 | fman | Status | resolved => closed |
Copyright © 2000 - 2019 MantisBT Team |