Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000056TestLinkNew Featurepublic2005-08-18 18:332015-04-26 16:32
Reporterschlundus 
Assigned To 
PrioritynormalSeverityfeature requestReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformOSOS Version
Product Version 
Fixed in Version 
Summary0000056: [SF 825060 ] Automated Test Integration
DescriptionTestLink looks like a really great tool for tracking
manual tests.

I'd like to see a mechanism for defining automated
tests within the TestlInk framework.

To be useful, of course, some mechanism would be needed
to cause the tests to be executed and the results
posted back to TestLink.

One implementation would be to export information about
testcases flagged as 'automated'. Another process,
independant of TestLink, could extract the test
execution information from the exported data, execute
the tests. The same process could gather the test
results and update the TestLink database for the
specified tcid and build.

Another implementation could be totally external to
TestLink, except for creating the automated testcase
definition. A query of the TestLink database could be
made to select all testcases for a particular testplan
identified as 'automated'' where results have not been
posted. Then for each testcase, the test could be
executed and the results posted back to the TestLink
datbase.
TagsRemote Execution
Database (MySQL,Postgres,etc)
Browser
PHP Version
TestCaseID
QA Team - Task Workflow Status
Attached Fileszip file icon Testlink-Support-for-automated-execution.zip [^] (974,631 bytes) 2007-10-26 20:30
doc file icon Testlink-automationIntegration.doc [^] (48,640 bytes) 2007-11-09 14:51
zip file icon automationintegrationfiles.zip [^] (26,968 bytes) 2007-11-16 16:15

- Relationships
parent of 0002953acknowledged Contribution: exporting automated testcases in test plan to XML and executing when click Execute and Save Result button 
related to 0000296closedfman Support for adding Watir scripts in TL 
related to 0001650closedfman Using Custom fields for TestPlan assignment for manage test automation data 
child of 0007083new Mantis Issues proposed to be closed 
Not all the children of this issue are yet resolved or closed.

-  Notes
(0000048)
carlwaite (reporter)
2005-08-30 21:38

Just for information, we are using testlink to manage both manual and automated tests.
We have no link between testlink and the execution of the tests, but the plan is prepared in testlink and a results file is generated from the automation tool/scripts. The results file is then parsed and put into the relevant sql commands to update the testplan through mysql interface, it obviously needs to grab the right references for tc id's, build's etc.
What would be useful, or perhaps more generic, would be the possibility to "mass update" tests from a certain file format ... csv?
(0000936)
lesterdsz (reporter)
2006-08-10 16:44

We are also using testlink to manage both automated and manual tests - currently the automation tool used is silktest. The automation suite creates a text dump with the testcase ID and pass fail status, we then use a perl script to update these test results.
We are looking at integrating testlink with our autoamtion suite (silktest as well as other automation test - API test suite), where we can create a test plan with the automated testcases and trigger the automation run from testlink.
(0002323)
mhavlat (reporter)
2007-10-10 02:17

Manish would like to contribute the implementation:

Correspondence:
Now the good news is that I have a been able to get the working code
for adding a xmlrpc client in the TL, which is able to invoke the XML-RPC
server api, which essentially are the tests to be executed and get the
result back.

To summarize, We have added a "Execute Test" button to every
test case and pressing it will invoke the test and the result will be
displayed on the same page.

We are using the custom field feature of TL to create inputs for
specifying the remote server host and port parameter. By the
custom field feature is very neat and we found it very useful in
this particular case.

We are working towards submitting the patches and related
documentation (readme, reference xml server, and the patches
with the current cvs code).

Martin, as you had earlier suggested that I should write the
web services specification, but don;t think I really know what
I need to produce in this case, as we nave not really implemented
a web service. What will be important is the interface the clinet
expects the server to be implemented. If you think having this
interface definition in the form of wsdl will help, I can produce
that.

Thanks a lot for your time, and looking forward to submit the
patches as soon as possible.

Thanks and Regards,
Manish
(0002461)
manish (reporter)
2007-10-26 20:31

Hi,
I have uploaded the patch for automated test execution support in testlink.

To summarize, We have added a "Execute Test" button to every
test case and pressing it will invoke the test and the result will be
displayed on the same page.

We are using the custom field feature of TL to create inputs for
specifying the remote server host and port parameter. By the
custom field feature is very neat and we found it very useful in
this particular case.
(0002473)
fman (administrator)
2007-10-30 19:58

Reminder sent to: manish

Would you mind to contact me to:
francisco.mancardi@gmail.com
(0002489)
mhavlat (reporter)
2007-11-02 21:26

Status:
- target release: 1.8
- phpxmlrpc third party libraries will be introduced into project
- Francisco review the code developed by Swanand Pagnis

Additional requirements:
- Feature is optional (default is disabled)
- Executing Automated test must primary running from Test Plan (~ Test Execution page)
(0002490)
fman (administrator)
2007-11-02 21:38

1. I will add a config parameter for enable/disabling this feature
2. Refactored code (work not finished yet), has been commited to CVS HEAD
(0002625)
halperinko (reporter)
2007-11-09 03:38

I am wondering regarding ability to investigate into failed automatic test cases:
If written in good practice, Automation framework will be written in a modular fashion, with several layers of abstraction, so we will have the lowest level functions doing real basic things, then above them functions which perform more complicated tasks based on the lower ones etc.
To the Test Link only final result will be sent, thus we need to figure out how to interconnect to the execution engine, in order to be able to drill down the logs while investigating a test execution.
Remember, that eventually, Automation testers MAINLY work on investigating the cause of automatic execution failures - in order to declare a Bug or fix the script.
Any chance to integrate with such an execution control tool ?
(0002628)
lesterdsz (reporter)
2007-11-09 14:34

WE have alsao implemented automation integration with testlink. To achieve this we are using the STAF framework. we have added an automation configurationpage where the user can enter the IP of the machine where the automation suite exists, command to be executed and a dtat folder. In the test execution page we have proved a "execute automated tests" link which when clicked will create an XML file with test case ID's, description of all test cases in the suite that have keyword 'automated'. This XML file is then copied to the remote machine using the STAF file system seervices (remote machine should also have STAF installed and running). Then the specified command is trigerred in the remote machine again using STAF. Once the automation is completed the results are posted back via http.
(0002629)
lesterdsz (reporter)
2007-11-09 14:50

Doc describing the integration has been attached
(0002645)
fman (administrator)
2007-11-10 18:31

lesterdsz :
is possible to have your code ???
(0002675)
lesterdsz (reporter)
2007-11-16 16:14

Attached the files.

thanks,
lester
(0002676)
fman (administrator)
2007-11-16 16:58

Great!, I will try to give a look asap

regards
(0004052)
mhavlat (reporter)
2008-09-08 20:26

Guys,
could anybody told me. What functionality is already implemented in TL 1.8? I would like to add info into User guide. Is the attached document "Testlink-automationIntegration.doc" valid?
(0004057)
swanand (reporter)
2008-09-09 15:07

Hi,

There have been some changes in the code we (manish and swanand) submitted. We have added an installer and a few more custom-field parameters (e.g. Method name, to which XMl-RPC call will be made).
But I am unable to look into it owing to my current responsibilities on a different project. I will update the latest code within a week or two (hope this isn't too late).
Would also provide an ETA soon.

Thanks,
Swanand
(0004998)
dina (reporter)
2009-01-14 05:08

I have only been using Testlink for a day, but have found its ease of use and its ability to output reports in addition to test case management, make it an extremely valuable tool. My request would be to add the ability to automate scripts using a recording mechanism that records key strokes and mouse clicks. Of course, the ability to modify the test scripts to make them more robust would be helpful. another great bonus would be for the application to automatically capture all objects on a give page and assign them a name (during recording). As we all know, sometimes there is several available inputs for a single field, with that in mind, an ability to use excel to create data that the script can loop through would be a plus. I have tons of ideas, just limited a program. Outputting the results in a quick overview and graph similar to your output for manual tests would be helpful as well.
(0005006)
azl (administrator)
2009-01-15 01:45

@dina most of this functionality is unrelated to a test case tracking tool like Testlink. It sounds like you are confusing Testlink with tools that are used for record playback test automation.

If you do have other ideas specific to Testlink though it would be best to file a new issue here in mantis.
(0005386)
amitkhullar (reporter)
2009-02-11 11:46

Reminder sent to: swanand

Hi Swanand any updates on this?

Amit
(0007064)
yaliu_2000 (reporter)
2009-05-27 21:56

Hello,

I am new to TL. We are very interested in using TL as part of our automation framework. We want TL to be able to do two things:
1) export TestPlan with build and user info to XML file.
2) add a button in Execute tab. When the user click this button, TL will export TestPlan and pass the exported XML to STAF service.

Any advices on how to do those? Your input will be greatly appreciated!

Another question: In current 1.8.2 version, I can only add individual test case into TestPlan. Is any way I can add the whole Test Suite to test plan?

Thanks,
Ann
(0007070)
fman (administrator)
2009-05-28 16:24

@yaliu_2000:
please consider following thing:
1. if issue is not completely similar to you desired one, add a new one.
   this allow better traceability

2. do not add a completely different issue:
>> Another question: In current 1.8.2 version, I can only add individual test
>> case into TestPlan. Is any way I can add the whole Test Suite to test plan?
this has nothing to do with automation
second: Mantis is NOT for questions about how to use TL, just only to report (in a detailed way) new features, or bugs

regards
(0007085)
yaliu_2000 (reporter)
2009-05-29 02:52

Oops, I posted my question in the wrong place. I will post my question in forum then.

For the feature I desired, can I add a new feature request and I will work on it?

Thanks,
Ann
(0007313)
mhavlat (reporter)
2009-06-20 02:19

I take ownership as there is no progress for long time.
(0023195)
Mr.Bricodage (updater)
2015-04-26 16:32

Test Execution can be automated (Jenkins, API KEY, ...)

- Issue History
Date Modified Username Field Change
2005-08-18 18:33 schlundus New Issue
2005-08-30 21:38 carlwaite Note Added: 0000048
2006-08-10 16:44 lesterdsz Note Added: 0000936
2006-12-11 07:38 mhavlat Relationship added related to 0000002
2006-12-11 07:39 mhavlat Note Added: 0000979
2006-12-11 07:39 mhavlat Status new => confirmed
2007-01-20 22:46 schlundus Project Testlink 1.6.x => TestLink
2007-10-10 02:14 mhavlat Relationship deleted related to 0000002
2007-10-10 02:15 mhavlat Note Deleted: 0000979
2007-10-10 02:17 mhavlat Note Added: 0002323
2007-10-10 02:26 mhavlat Status confirmed => assigned
2007-10-10 02:26 mhavlat Assigned To => manish
2007-10-26 20:30 manish File Added: Testlink-Support-for-automated-execution.zip
2007-10-26 20:31 manish Note Added: 0002461
2007-10-30 19:58 fman Note Added: 0002473
2007-11-02 21:26 mhavlat Note Added: 0002489
2007-11-02 21:38 fman Note Added: 0002490
2007-11-09 03:38 halperinko Note Added: 0002625
2007-11-09 14:34 lesterdsz Note Added: 0002628
2007-11-09 14:50 lesterdsz Note Added: 0002629
2007-11-09 14:51 lesterdsz File Added: Testlink-automationIntegration.doc
2007-11-10 18:31 fman Note Added: 0002645
2007-11-16 16:14 lesterdsz Note Added: 0002675
2007-11-16 16:15 lesterdsz File Added: automationintegrationfiles.zip
2007-11-16 16:58 fman Note Added: 0002676
2008-09-08 20:26 mhavlat Note Added: 0004052
2008-09-09 15:07 swanand Note Added: 0004057
2009-01-14 05:08 dina Note Added: 0004998
2009-01-15 01:45 azl Note Added: 0005006
2009-02-11 11:46 amitkhullar Note Added: 0005386
2009-05-27 21:56 yaliu_2000 Note Added: 0007064
2009-05-28 16:24 fman Note Added: 0007070
2009-05-29 02:52 yaliu_2000 Note Added: 0007085
2009-06-20 02:15 mhavlat Relationship added related to 0000296
2009-06-20 02:18 mhavlat Relationship added child of 0001650
2009-06-20 02:18 mhavlat Relationship replaced related to 0001650
2009-06-20 02:19 mhavlat Note Added: 0007313
2009-06-20 02:19 mhavlat Assigned To manish => mhavlat
2009-06-20 02:19 mhavlat Status assigned => acknowledged
2010-02-08 05:41 mhavlat Relationship added parent of 0002953
2010-02-28 22:23 mhavlat Note Added: 0009238
2010-02-28 22:26 mhavlat Assigned To mhavlat =>
2011-01-23 17:26 fman Tag Attached: Remote Execution
2015-04-26 15:58 Mr.Bricodage Relationship added child of 0007083
2015-04-26 16:32 Mr.Bricodage Note Added: 0023195
2016-03-26 19:13 fman Note Deleted: 0009238



Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker