Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008331TestLinkAPI - XMLRPCpublic2018-07-16 06:452018-07-23 07:07
Reporterloicmbakop 
Assigned Tofman 
PriorityhighSeverityfeature requestReproducibilityN/A
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version1.9.17 (2018 Q1) 
Fixed in Version1.9.18 (2018 Q3) 
Summary0008331: add a new method to allow user to set the step result of a step in a test case using the xmlrpc api
DescriptionSet step result for a test case
Additional InformationHello, I am working on a test automation project, and I am new to testlink. The tools i use for my project are the following

Testlink 1.9.17.
Testlink Python API version 1.1
Jenkins version 2.107.3
I am using the python api to read steps of each tests cases of a testpan and execute some actions. What i need to do is to be able to set the status of each step in a test case, and in the python api I have not seen any method that allow me to do that.
In other words The method reportTCResult only allow me to set the status of a test case and to manually set the status each step like this
newResult = myTestLink.reportTCResult(None, newTestPlanID_A, None, 'f', '', guess=True,
testcaseexternalid=tc_aa_full_ext_id,
platformname=NEWPLATFORM_A,
execduration=3.9, timestamp='2015-09-18 14:33'
steps=[ {'step_number' : 6, 'results' : 'p', 'notes" : 'result note for passed step 6'},
{'step_number' : 7, 'results' : 'f', 'notes" : 'result note for failed step 7'}] )
What i would like to do is to set the status of each of my steps first during the execution, and depending on that, set the global status of the test case. I have not seen any method to do that in the python api and i am currently stucked.

Thank you for your help
TagsNo tags attached.
Database (MySQL,Postgres,etc)xamp: apche, mysql
BrowserFirefox
PHP Version
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Files

- Relationships
child of 0008241new Availables hot-fixes for 1.9.17 & How To get full fixed package from GitHub 

-  Notes
(0027695)
fman (administrator)
2018-07-16 18:37
edited on: 2018-07-16 18:40

reportTCResult allows to work as you can work on GUI

you need to set WHOLE TC exec status and single step exec status, but TestLink will not set whole Test Case exec status analizing test case steps exec status.

What you can do is call twice (or more times) reportTCresults

a) set whole status to a new one PENDING (you need to configure everything to do this), and set step exec
b) call reportTCResults with overwrite option

(0027696)
loicmbakop (reporter)
2018-07-16 19:57

Thank you very much for your answer. I have tried to call the reportTCResult each time I loop on a step to set the status of the step with that method in my python script, but on my final report I alway obtain the result for the last step. My guess is that each time i call the reportTCResult on a step I erase the old one created, so i only get the report for the last step of my looped on. Regarding the first point you mentionned, I suppose there is maybe something to configure to set the status to a new one pending. Can you please tell me how i can do that.
In fact i have tried to call the reportTCResult with the no option as a 4th parameter, instead of 'p' for passed, 'f' for failed, and 'b' for blocked , i tried to put empty brackets like this '', but i had an error on the call of the method
(0027697)
fman (administrator)
2018-07-16 21:39

In each iteration you need to provide ALL steps results.

>> Can you please tell me how i can do that.
give a look to forum and to *custom* files

>> n fact i have tried to call the reportTCResult with the no option as a 4th parameter,
>> instead of 'p' for passed, 'f' for failed, and 'b' for blocked ,
>> i tried to put empty brackets like this '', but i had an error on the call of the method
guessing instead of trying to access documentation or reading actual code is always a very,very bad practice
(0027700)
loicmbakop (reporter)
2018-07-17 09:12

It does not make any sense to me to give all steps results in one iteration. For me an iteration is the action associated to a step, so at one iteration i only provide one step result. furthermore since i want to automate the report generation, i dont see how using the reportTCResult with all my steps given in an iteration would help for automation, considering that i'm not always going to have the same amount of steps in my test cases.
I have used the documentation of the testlink Python API to work and I have tried to add the overwrite =True option in the reportTCResult, but it did not work and still gave only the last result.
(0027701)
loicmbakop (reporter)
2018-07-17 09:12

It does not make any sense to me to give all steps results in one iteration. For me an iteration is the action associated to a step, so at one iteration i only provide one step result. furthermore since i want to automate the report generation, i dont see how using the reportTCResult with all my steps given in an iteration would help for automation, considering that i'm not always going to have the same amount of steps in my test cases.
I have used the documentation of the testlink Python API to work and I have tried to add the overwrite =True option in the reportTCResult, but it did not work and still gave only the last result.
(0027702)
loicmbakop (reporter)
2018-07-17 09:55

The overwrite option only allow testlink to overwrite all the reports generated. That is not what my requirement is. I want to have only one report with all my steps status set inside that only report for one or many test cases. Each time i call the reportTCResult method in a loop to set the step status, a new report is generated. So for a test case containing 3 steps, I generate 3 reports, one for each steps i set using the reportTCResult method. The overwrite option only write the generated report over the old ones and erase them.
(0027703)
fman (administrator)
2018-07-17 15:00

As stated right now API works in the same way that GUI does:
You need to provide a WHOLE TEST CASE Exec Result in order to save also TEST CASE STEP exec result.

On change that can be done is add a parameter to do not delete result for test step without an exec result.

>> The overwrite option only allow testlink to overwrite all the reports generated
this sentence is not clear for me.
overwrite do just overwrite the execution ( I do not remember now if you can specifiy exec id or always latest exec is overwritten)
(0027704)
loicmbakop (reporter)
2018-07-18 09:59

Yes i know that the API work the same as the GUI. THE GUI allow users to manually set the result of a test by setting the all steps result and the global result of the test case. Now me i want to do the exact same thing but with automation. With the python API that i am using, there is only one method to set the result of a test case. You told me earlier the following sentence

>>you need to set WHOLE TC exec status and single step exec status, but TestLink will not set whole Test Case exec status analizing test case steps exec status.

What you can do is call twice (or more times) reportTCresults

when i call the reportTCResult more than once, it generate a new report each time. Let me explain my self. For exemple in my test case i have 3 steps. after my first step execution i call the reportTCResult method to set the result of my first step. That call generate a report. Now i move on to my second step, and after the execution of the second step i call the reportTCResult method again to set the result of my second step, that second call generate a second report with the step result associated to my second test case. And for a test case with 3 steps i end up with 3 report generated containing my 3 steps but only one step result and note set in each report. I thought that the overwrite option would add the additional information on the last execution report as it is said in the API documentation i quote " overwrite : if present and true then the last execution for (testcase, testplan, build, platform) will be overwritten", but it completely erase the old execution result and replace it by the new one
(0027705)
fman (administrator)
2018-07-18 10:25

I only provide support and fix isses on PHP Side, no other API Libraries like JAVA, PERL or Python.

I'm going to do some tests but iMHO all code need is there.
(0027706)
loicmbakop (reporter)
2018-07-18 10:36

Ok pls let me know if you find a way to fix that
(0027707)
loicmbakop (reporter)
2018-07-18 10:46

PLs do you know where should i post a topic to get support on testlink Python api. I already posted a topic on the git page of the testlink pythin API and i have been redirected on this page
(0027714)
fman (administrator)
2018-07-21 15:56

just checked, overwrite option does not manage steps.
I'm working on fixing this issue
(0027715)
fman (administrator)
2018-07-21 19:36

https://github.com/TestLinkOpenSourceTRMS/testlink-code/commit/d05548c9d7818f6b5bff7c64fb9d849f4193c934 [^]
(0027716)
loicmbakop (reporter)
2018-07-21 19:57

Thank you very much for your assistance. I assume that you have made some changes on the php code to take in account my requirement, but my question now is how can i set my steps results using the python api. Does the overwrite option now add the steps results in only one report or do i need to add a special argument
(0027717)
fman (administrator)
2018-07-22 08:51

Give a look to the sample client written in PHP that is part of the commit.
You need to use overwrite option.

I can not provide any help on the python side
(0027718)
loicmbakop (reporter)
2018-07-23 07:07

Alright thanks. But since i am using the python api and i have already developped all my functions using that api, at this point i cannot go back and rewrite all my function in php. Can you pls tell me where i should post or who i should contact to have that functionnality using the python API ?

- Issue History
Date Modified Username Field Change
2018-07-16 06:45 loicmbakop New Issue
2018-07-16 18:37 fman Note Added: 0027695
2018-07-16 18:40 fman Note Edited: 0027695 View Revisions
2018-07-16 18:41 fman Assigned To => fman
2018-07-16 18:41 fman Status new => feedback
2018-07-16 19:57 loicmbakop Note Added: 0027696
2018-07-16 19:57 loicmbakop Status feedback => assigned
2018-07-16 21:39 fman Note Added: 0027697
2018-07-17 05:30 fman QA Team - Task Workflow Status => TBD
2018-07-17 05:30 fman Status assigned => resolved
2018-07-17 05:30 fman Resolution open => no change required
2018-07-17 09:12 loicmbakop Note Added: 0027700
2018-07-17 09:12 loicmbakop Note Added: 0027701
2018-07-17 09:12 loicmbakop Status resolved => assigned
2018-07-17 09:12 loicmbakop Resolution no change required => reopened
2018-07-17 09:55 loicmbakop Note Added: 0027702
2018-07-17 15:00 fman Note Added: 0027703
2018-07-18 09:59 loicmbakop Note Added: 0027704
2018-07-18 10:25 fman Note Added: 0027705
2018-07-18 10:36 loicmbakop Note Added: 0027706
2018-07-18 10:46 loicmbakop Note Added: 0027707
2018-07-21 15:56 fman Note Added: 0027714
2018-07-21 15:56 fman Status assigned => feedback
2018-07-21 19:36 fman QA Team - Task Workflow Status TBD => READY FOR TESTING
2018-07-21 19:36 fman Note Added: 0027715
2018-07-21 19:36 fman Status feedback => resolved
2018-07-21 19:36 fman Fixed in Version => 1.9.18 (2018 Q3)
2018-07-21 19:36 fman Resolution reopened => fixed
2018-07-21 19:37 fman Relationship added child of 0008241
2018-07-21 19:57 loicmbakop Note Added: 0027716
2018-07-22 08:51 fman Note Added: 0027717
2018-07-23 07:07 loicmbakop Note Added: 0027718



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker