Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008100TestLinkInventory / Platformspublic2017-10-09 19:182018-06-22 17:53
Reporterrusty 
Assigned Tofman 
PrioritynormalSeverityfeature requestReproducibilityalways
StatusfeedbackResolutionopen 
PlatformALLOSALLOS VersionALL
Product Version1.9.16 (2016 Q4) 
Fixed in Version 
Summary0008100: Need ability to control testing under multiple different platforms
DescriptionI run the testing for a complex product which is a host - client scenario. I have severe need of support a test tracking system which tracks multiple different platforms. Previous person in my position was using testlink, it seems like a good idea to stick with his previous choice but it has proven problematic.
Example:
The server is supported on multiple Operating systems (Let's say 6 different types for now), the client is supported on 6 different operating systems. This specialized server has special hardware which completely changes the possible outcome of the test(Let's say 3 different types of hardware).

This means, of the 80 actual test cases I have. I either have to make 108 different platforms to describe the exact nature of what is being executed, or I need to make test cases in duplicate. Making the test cases in duplicate is what the last employee did which resulted in 8640 test cases with only about 80 unique.

This is so infuriating as a tester because it is soooooo disorganized.. It also makes soo much sense as to why the last guy was let go.

What needs to happen is that each test plan should have a list of added platform types associated to it. Each platform type should have a list of plaforms that the test plan will associate with. I realize that many people don't have such complex products that might only require a single platform, but for those of us who do have such complicated projects.. this is a god send.

PS. We are currently adding a GUI to our product which will mean that each of those 80 test cases would have been copied again making a total of 34,560 test cases based on 432 different permutations of different platforms or combonations of plaforms.
Steps To ReproduceFeature request.
Just make a project that needs more than one type of platform.
TagsNo tags attached.
Database (MySQL,Postgres,etc)mysql
BrowserALL
PHP Version
TestCaseID
QA Team - Task Workflow StatusTBD
Attached Files

- Relationships

-  Notes
(0027583)
rusty (reporter)
2018-06-15 09:06
edited on: 2018-06-15 09:10

Can I be granted permissions to update this ticket to read "platform types"?

(0027584)
fman (administrator)
2018-06-16 16:41

If you provide me with the full summary you want I will change it, but can not give you more permissions

in order to understand your request very detailed examples are needed because your generic description is not clear for me.

In any case, this does not mean that your requirement is going to be solved immediately.

You can explore (if not already done) all workarounds that can be constructed using custom fields, in order to create a limited set of test cases but a lot of executions.

You can use as approach duplicate test cases but instead of copy any time the same content with the maintenance nightmare, you can use the ghost feature or use the link to the master test case.

I'm not proficient on other test case management systems, but I do not think they provide the solution you need out of the box.

again I can try to help you to do some kind of brainstorming, but I need much more detailed data & examples.
(0027629)
rusty (reporter)
2018-06-22 06:52
edited on: 2018-06-22 06:58

Many of them do, but they cost $7,000 US for 4 users. Q-test, and test-Rail both have the functionality, even if it takes some time and effort to set up.

From your response, it sounds like I need to expound upon the problem.

Here is the dilemma and crux of the problem: A single list of platforms to permute against in test plans is not sufficient as it does not allow many test plan designers the flexibility needed to require the proper kinds permutations that are needed for each test case in their test plans.

Take for instance Testing a SAN (Storage Attached Network). SAN's are VERY complex products and have a lot of moving parts.

We build SAN's.

For every test case in testing a SAN:
We have the OS that the SAN is running on.
We have the OS of the machine that is connecting to the SAN.
We have the fabric that is bridging the connection (ISCSI, Fibre Channel, Infiniband, NVMEoE, etc..)
We have the storage configuration of the SAN (Raid-10, Raid-5, JBOD, ETC..).
We have the clustering configuration that the SAN is in (Write through, Write back, Shared Journal, ETC...).
We have the internet browser which is making changes to the SAN (Firefox, Internet Explorer, Chrome).
We have the traffic profile that we are using to validate data on the SAN (Read, write, readwrite, read and 4k, etc..).

All of these lists could potentially fit the description of a "Platform". The issue is that, in TestLink, we only have ONE list of platforms to work with. This is deficient for testing something as complex as a SAN or even anything that needs more than one list of platforms because when making test plans no suitable permutation list can be made to understand total coverage.

For simple products that require one list of platforms the number of needed test runs to satisfy each permutation in the single list of platforms is relatively small; one per element in the list of platforms. Usually 1-12 needed test runs. However, for some of the more complex projects, the number of test runs required to declare success of that test case is multiplicative product for each permutation of every platform in every list of needed platforms.

Example:
              
Test Case 1
            Platform List1 |Platform List2|Platform List3|
            |Platform 1 |Platform A X |Platform Z |
            |Platform 2 X |Platform B X |Platform X X |
            |Platform 3 |Platform C X |Platform Y |
            |Platform 4 |Platform D X |Platform W X |
            |Platform 5 |Platform E X |Platform V |
Number of needed test runs = 1 * 5 * 2 = 10
Test Case 2
            Platform List1 |Platform List2|Platform List3|
            |Platform 1 |Platform A X |Platform Z |
            |Platform 2 X |Platform B X |Platform X X |
            |Platform 3 |Platform C X |Platform Y X |
            |Platform 4 X |Platform D X |Platform W X |
            |Platform 5 |Platform E X |Platform V |
Number of needed test runs = 2 * 5 * 3 = 30
Test Case 3
            Platform List1 |Platform List2|Platform List3|
            |Platform 1 X |Platform A X |Platform Z X |
            |Platform 2 X |Platform B X |Platform X X |
            |Platform 3 X |Platform C X |Platform Y X |
            |Platform 4 X |Platform D X |Platform W X |
            |Platform 5 X |Platform E X |Platform V X |
Number of needed test runs = 5 * 5 * 5 = 125
Test Case 4
            Platform List1 |Platform List2|Platform List3|
            |Platform 1 |Platform A |Platform Z |
            |Platform 2 |Platform B |Platform X |
            |Platform 3 |Platform C |Platform Y |
            |Platform 4 |Platform D |Platform W |
            |Platform 5 |Platform E |Platform V |
Number of needed test runs = 0 * 0 * 0 = 1 */Behaves just as if no platforms were selected normally and a minimum of 1 test run is needed/*
Any combination of any platform will satisfy the testing the test case requires.

My goal is to make this happen to increase the functionality that test link has to offer to better service more complex testing scenarios.

Does this make sense?


- Issue History
Date Modified Username Field Change
2017-10-09 19:18 rusty New Issue
2017-10-09 19:19 rusty Note Added: 0026912
2017-10-10 12:24 fman PHP Version Doesn't matter =>
2017-10-10 12:24 fman QA Team - Task Workflow Status => TBD
2017-10-10 12:24 fman Steps to Reproduce Updated View Revisions
2017-10-10 12:24 fman Note Deleted: 0026912
2018-06-15 09:06 rusty Note Added: 0027583
2018-06-15 09:10 rusty Note Edited: 0027583 View Revisions
2018-06-16 16:41 fman Note Added: 0027584
2018-06-22 06:52 rusty Note Added: 0027629
2018-06-22 06:58 rusty Note Edited: 0027629 View Revisions
2018-06-22 17:53 fman Assigned To => fman
2018-06-22 17:53 fman Status new => feedback



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker