Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005970TestLinkTest Execution - Bulk Modepublic2013-10-10 19:572013-11-17 09:53
Reporterdevendratapdia 
Assigned Tofman 
PriorityhighSeverityfeature requestReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.9.8 (2013 Q3 - bug fixing) 
Fixed in Version1.9.9 (2013 Q4 - bug fixing) 
Summary0005970: Bulk execution for automated group of Test Case IN a TEST SUITE
DescriptionThere is requirement for single click multiple test case execution at "Test Suite" Level.

For example, there should be marking option at test suite level, to select multiple test cases to be run at once simultaneously. Once we select the tc's we should be able to click "Execute and Save results" here at test suite level (Similar to test case level). And this way run multiple test cases.

If simultaneous automated tc's execution runs in performance issues, we can atleast opt for QUEUED method of execution for this, where test cases will run one after another.

I have already referred this bug 0002056 http://mantis.testlink.org/view.php?id=2056 [^]
Steps To ReproduceClick on "Test Execution" > Select any "Test suite" with multiple tc's

At test suite level there is no option for bulk automated test execution.
Additional InformationQUEUE option looks good for this
TagsNo tags attached.
Database (MySQL,Postgres,etc)MySQL
BrowserFirefox
PHP Version5.4.19
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Files

- Relationships

-  Notes
(0019817)
fman (administrator)
2013-10-11 05:32

Bulk execution already exists and is enabled using

// This can help to avoid performance problems.
// Controls what happens on right frame when user clicks on a testsuite on tree.
// ENABLED -> show all test cases presents on test suite and children test suite.
// DISABLED -> nothing happens, to execute a test case you need to click on test case
$tlCfg->exec_cfg->show_testsuite_contents = DISABLED;

I suggest you read carefully config.inc.php (and signal clearly that you have done it on your report), because there is lot of info that
we have no time to put on a user manual.

Regarding Autmated Test Cases: test link is not responsible of any automation
if you want to manage a QUEU the right side to do it (IMHO) is the server that will run the tests, and not create a complex feature on TestLink like a queu manager.
(0019818)
devendratapdia (reporter)
2013-10-11 07:38
edited on: 2013-10-11 07:43

I have this option Already ENABLED in my env. It's for bulk manual test case update and not for bulk automated test cases execution. Queue was just a suggestion.

Also with everyone moving from manual to automation, doesn't this become necessity now to support bulk automated test execution. Automated test frameworks are generally built from bulk execution point of view. Shouldn't testlink support the same feature to integrate with these framework well.

(0019822)
fman (administrator)
2013-10-11 18:25
edited on: 2013-10-11 18:27

Now things are a little bit clear.
On [TL_INSTALL_DIR]\third_party\user_contribution\fakeRemoteExecServer
we have provided details for a possible way to implement delayed EXECUTION that has to be controlled by execution server.

This is content of README.TXT
<TL INSTALL DIR>third_party/user_contribution/fakeRemoteExecServer/README.TXT

Here you will find simple PHP examples that can help you to play with the feature
present on execution screen, that allow you to call an external server to execute test case
and return results.

- Directory contents -
fakeXMLRPCTestRunner.php - the simple server
client4fakeXMLRPCTestRunner.php - a client to test simple server OUTSIDE execution feature
all_testsuites_remote_fake_exec.xml - a test suite that has been configured to be used to test this feature

Surely we need to improve documentation, I've to review the code but I think that a test suite can be enabled to allow execution of ALL TEST CASES present on it, with some custom field.


You said that QUEUE is just a suggestion, but is part of issue subject then
this means is part of requirement.

(0019823)
devendratapdia (reporter)
2013-10-11 20:04
edited on: 2013-10-11 20:05

>> I've to review the code but I think that a test suite can be enabled to allow execution of ALL TEST CASES present on it, with some custom field.

Currently there isn't a way for bulk automated test execution at test suite level. Custom fields are just for email, checkbox, list, string, server url, script url,etc. Even in case of server url, it doesn't appear during test case execution looks like it's a bug, I just found.

My requirement is to have same "Execute and save results" (similar to test case level) button/option at test suite level, AND it should also provide option to select automated test cases to run in bulk.

>> You said that QUEUE is just a suggestion, but is part of issue subject then
this means is part of requirement.

I think now it's no more editable. Sorry for that. Shouldn't have put in title. It's not a requirement from me to have a queue. I just want bulk automated execution option.

>> On [TL_INSTALL_DIR]\third_party\user_contribution\fakeRemoteExecServer
we have provided details for a possible way to implement delayed EXECUTION that has to be controlled by execution server.

I am already executing automated test cases using this server but they are one at a time. I have to go to particular test case and execute. I have provided that fakeserver address for now.

So I guess you understood now what my actual requirement is? Please let me know if it's still not clear. Thanks.

(0019824)
fman (administrator)
2013-10-11 20:09
edited on: 2013-10-11 20:10

From cfield_mgr.class.php
/**
 * Retrieves the XML-RPC Server Parameters specified through custom fields.
 * 
 * Done searching CARVED in the stone Custom Field Names on different
 * (AGAIN CARVED in the stone) CF value tables in this way:
 *
 * CF name will have 3 pieces separated by _ (underscore)
 *
 * RE-XMLRPC_url_tsuite
 * RE-XMLRPC_url_tcase
 * RE-XMLRPC_url_link
 *
 * Part 1: 	RE-XMLRPC_ FIXED value, used as search key to get automatically 
 *			CF to be analised.
 *
 * Part 2: 	url will be key on returned hash, and is part of 'contract' with caller,
 *			i.e. caller will use this key.
 *			This key is a FREE choice of developer of Remote Execute modules to use
 *			with TL.
 *
 * Part 3:	this part is domain (link,tcase,tsuite)
 *			work this way:
 *			To specify Remote Execution server parameters we have provided 3 choices
 *			a. on test case version LINKED to Test Plan + Platform (Test Plan Design time)
 *			b. on test case version BUT at Test Spec Design time.
 *			   In this way if is OK to have always same parameters no matter 
 *			   test plan + platform where test case version has been linked, we configure
 *			   this just ONCE.
 *			c. on test suite (can be done ONLY at Test Spec Design time), all test case versions
 *			   contained on this test suite branch (and children Test suites) will share this
 *			   configuration.
 *	
 *
 *
 * @param integer $node_id Accepts current node id from nodes hierarchy level
 * @return mixed An array of config params if found, else returns null
 *
 * @internal rev:
 * 
 * 20110123 - franciscom -	need refactoring after we have choose to link custom field
 *							values to test case version not to test case
 *
 **/
function getXMLRPCServerParams($nodeID,$tplanLinkID=null)
{



Need to right good doc to allow people to use it.
Meanwhile you can dig the code to try to understand how you can use it (
it will be great if you can create doc for other users)
TestLink rocks!!!

(0019825)
devendratapdia (reporter)
2013-10-11 20:21
edited on: 2013-10-11 20:21

I have all these custom fields. I don't know what you are talking about. Why are you pointing to documentation? Please be clear.

(0019826)
fman (administrator)
2013-10-11 20:34

ON Template execSetResults.tpl

Change from
  {if $cfg->exec_cfg->show_testsuite_contents && $gui->can_use_bulk_op}
      <div>
      

      <input type="button" id="do_export_testcases" name="do_export_testcases" value="{$labels.btn_export_testcases}"
                 onclick="javascript: openExportTestCases('export_testcases',{$gui->node_id},{$gui->tproject_id},
                                                          {$gui->tplan_id},{$gui->build_id},{$gui->platform_id},
                                                          '{$gui->tcversionSet}');" />


to
  {if $cfg->exec_cfg->show_testsuite_contents && $gui->can_use_bulk_op}
      <div>
      

      <input type="button" id="do_export_testcases" name="do_export_testcases" value="{$labels.btn_export_testcases}"
                 onclick="javascript: openExportTestCases('export_testcases',{$gui->node_id},{$gui->tproject_id},
                                                          {$gui->tplan_id},{$gui->build_id},{$gui->platform_id},


      {if $tlCfg->exec_cfg->enable_test_automation}
        <input type="submit" id="execute_cases" name="execute_cases"
                     value="{$labels.execute_and_save_results}"/>
      {/if}


Then give a try.
I can not provide quicker assistance than this.
(donation will be welcomed)
(0019827)
devendratapdia (reporter)
2013-10-11 20:42

Ok. You mean to change and customize it according to my requirement. Can't we make it general, let's say if it works for me i.e. commit to testlink codebase.
(0019828)
fman (administrator)
2013-10-12 06:35

I never do things that can be useful to the comunity, privated and just for one people.
I've just tried to provide you a quick solution to solve your issue, request you that the fix work, request you to write a document about this in order to make things also simple for others.
Is pretty obvious that this will be part of standard code.

I will wait for your feedback and the document
(0019829)
devendratapdia (reporter)
2013-10-12 08:08

ok. I will try and let you know.
(0019837)
fman (administrator)
2013-10-16 09:52

any news ?
(0019838)
devendratapdia (reporter)
2013-10-16 10:43

You can close this for now. I will let you know or create one more bug referencing this one.
(0019842)
fman (administrator)
2013-10-16 20:06

ok, but what about your collaboration with the document, that I've requested on note (0019828)
(0019844)
devendratapdia (reporter)
2013-10-17 09:49

I still didn't understand about document. What kind of document are you asking for?
(0019845)
fman (administrator)
2013-10-17 09:52

>> ... request you to write a document about this in order to make things also
>> simple for others.
(0019846)
devendratapdia (reporter)
2013-10-17 10:03

ok. Sure. If this one works for me. I will create a doc too.
(0020055)
fman (administrator)
2013-11-17 09:53

1.9.9 Released

- Issue History
Date Modified Username Field Change
2013-10-10 19:57 devendratapdia New Issue
2013-10-11 05:32 fman Note Added: 0019817
2013-10-11 05:32 fman QA Team - Task Workflow Status TBD => READY FOR TESTING
2013-10-11 05:32 fman Status new => resolved
2013-10-11 05:32 fman Resolution open => no change required
2013-10-11 05:32 fman Assigned To => fman
2013-10-11 07:38 devendratapdia Note Added: 0019818
2013-10-11 07:38 devendratapdia Status resolved => assigned
2013-10-11 07:38 devendratapdia Resolution no change required => reopened
2013-10-11 07:43 devendratapdia Note Edited: 0019818 View Revisions
2013-10-11 18:25 fman Note Added: 0019822
2013-10-11 18:26 fman Note Edited: 0019822 View Revisions
2013-10-11 18:27 fman Note Edited: 0019822 View Revisions
2013-10-11 20:04 devendratapdia Note Added: 0019823
2013-10-11 20:05 devendratapdia Note Edited: 0019823 View Revisions
2013-10-11 20:09 fman Note Added: 0019824
2013-10-11 20:09 fman Note Edited: 0019824 View Revisions
2013-10-11 20:10 fman Note Edited: 0019824 View Revisions
2013-10-11 20:11 fman Summary Bulk execution for automated tc's using QUEUE => Bulk execution for automated group of Test Case IN a TEST SUITE
2013-10-11 20:21 devendratapdia Note Added: 0019825
2013-10-11 20:21 devendratapdia Note Edited: 0019825 View Revisions
2013-10-11 20:34 fman Note Added: 0019826
2013-10-11 20:42 devendratapdia Note Added: 0019827
2013-10-12 06:35 fman Note Added: 0019828
2013-10-12 07:20 fman Status assigned => feedback
2013-10-12 08:08 devendratapdia Note Added: 0019829
2013-10-12 08:08 devendratapdia Status feedback => assigned
2013-10-13 17:32 fman Status assigned => feedback
2013-10-16 09:52 fman Note Added: 0019837
2013-10-16 10:43 devendratapdia Note Added: 0019838
2013-10-16 10:43 devendratapdia Status feedback => assigned
2013-10-16 20:06 fman Note Added: 0019842
2013-10-16 20:07 fman Status assigned => resolved
2013-10-16 20:07 fman Fixed in Version => 1.9.9 (2013 Q4 - bug fixing)
2013-10-16 20:07 fman Resolution reopened => fixed
2013-10-17 09:49 devendratapdia Note Added: 0019844
2013-10-17 09:52 fman Note Added: 0019845
2013-10-17 10:03 devendratapdia Note Added: 0019846
2013-11-17 09:53 fman Note Added: 0020055
2013-11-17 09:53 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker