MantisBT - TestLink
View Issue Details
0006159TestLinkAPI - XMLRPCpublic2014-01-15 21:022014-04-25 17:38
1.9.9 (2013 Q4 - bug fixing) 
1.9.10 (2014 Q1 - bug fixing) 
postgres 9.2
0006159: getTestCasesForTestPlan returns confusing exec_status [POSTGRESQL]
users of the xml-rpc api reports, that getTestCasesForTestPlan returns 'exec_status' values, which does not match the last execution state of a test case.

Following the discussion in 0005996, the expectation is, that getTestCasesForTestPlan will return no execution states anymore, cause getLastExecutionResult should be used.
But in TL 199, getTestCasesForTestPlan still returns the confusing 'exec_status'.
manual steps in TL:
1. create project 'P-A' with test suite 'TS-A', test case 'TC-A',
2. create test plan 'TP-A' with build 'B-A' and add test case 'TC-A'
3. save FIRST execution of test case 'TC-A' = passed

API call from client
4. call xml-rpc api method getTestCasesForTestPlan with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'exec_status': 'p' - ok
   {'6047': [{'tcase_id': '6047', ... , 'exec_status': 'p'}]}
manual steps in TL:
5. save SECOND execution of test case 'TC-A' = failed

API call from client
6. call xml-rpc api method getTestCasesForTestPlan with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'exec_status': 'p' ->> CONFUSION - 'f' expected
   {'6047': [{'tcase_id': '6047', ... , 'exec_status': 'p'}]}
7. call xml-rpc api method getLastExecutionResult with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'status': 'f' - ok
   [{'build_id': '281', 'status': 'f', ... }]
No tags attached.
child of 0006048closed fman Availables hot-fixes for 1.9.9 & How To get full fixed package from gitorious 
xml ISSUE-0006159.testproject-deep.xml (928) 2014-01-18 10:23
xml tree_TPLAN-A.xml (797) 2014-01-18 10:25
png test01-issue6159.png (24,357) 2014-01-18 10:43

png test02-issue6159-exec-to-passed.png (24,456) 2014-01-18 10:43

png test03-issue-6159-second-run-to-failed.png (24,316) 2014-01-18 10:43

? client_issue6159_t1.php (757) 2014-01-18 10:47
? client_issue6159_t2.php (781) 2014-01-18 10:54
png test02-using-testcase-internal-id.png (25,226) 2014-01-18 10:54
Issue History
2014-01-15 21:02lczubNew Issue
2014-01-16 06:38fmanNote Added: 0020332
2014-01-16 06:38fmanNote View State: 0020332: public
2014-01-16 06:42fmanNote Edited: 0020332bug_revision_view_page.php?bugnote_id=20332#r3302
2014-01-17 18:56lczubNote Added: 0020335
2014-01-17 20:46fmanNote Added: 0020337
2014-01-18 10:23fmanFile Added: ISSUE-0006159.testproject-deep.xml
2014-01-18 10:25fmanFile Added: tree_TPLAN-A.xml
2014-01-18 10:27fmanNote Added: 0020339
2014-01-18 10:28fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3306
2014-01-18 10:43fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3307
2014-01-18 10:43fmanFile Added: test01-issue6159.png
2014-01-18 10:43fmanFile Added: test02-issue6159-exec-to-passed.png
2014-01-18 10:43fmanFile Added: test03-issue-6159-second-run-to-failed.png
2014-01-18 10:44fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3308
2014-01-18 10:46fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3309
2014-01-18 10:46fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3310
2014-01-18 10:47fmanFile Added: client_issue6159_t1.php
2014-01-18 10:47fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3311
2014-01-18 10:48fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3312
2014-01-18 10:48fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3313
2014-01-18 10:49fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3314
2014-01-18 10:49fmanNote View State: 0020339: public
2014-01-18 10:50fmanAssigned To => fman
2014-01-18 10:50fmanStatusnew => feedback
2014-01-18 10:54fmanFile Added: client_issue6159_t2.php
2014-01-18 10:54fmanFile Added: test02-using-testcase-internal-id.png
2014-01-18 10:55fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3315
2014-01-18 10:55fmanNote Edited: 0020339bug_revision_view_page.php?bugnote_id=20339#r3316
2014-01-18 11:01lczubNote Added: 0020340
2014-01-18 11:01lczubStatusfeedback => assigned
2014-01-18 22:04lczubNote Added: 0020341
2014-01-18 22:12fmanNote Added: 0020342
2014-01-18 22:13fmanQA Team - Task Workflow Status => TBD
2014-01-18 22:13fmanSummarygetTestCasesForTestPlan returns confusing exec_status => getTestCasesForTestPlan returns confusing exec_status [POSTGRESQL]
2014-01-18 22:40fmanNote Edited: 0020341bug_revision_view_page.php?bugnote_id=20341#r3318
2014-01-18 22:49fmanNote Added: 0020343
2014-01-19 10:22fmanNote Added: 0020344
2014-01-19 10:22fmanQA Team - Task Workflow StatusTBD => READY FOR TESTING
2014-01-19 10:22fmanStatusassigned => resolved
2014-01-19 10:22fmanFixed in Version => 1.9.10 (2014 Q1 - bug fixing)
2014-01-19 10:22fmanResolutionopen => fixed
2014-01-19 10:22fmanRelationship addedchild of 0006048
2014-04-25 17:38fmanNote Added: 0020848
2014-04-25 17:38fmanStatusresolved => closed

2014-01-16 06:38   
(edited on: 2014-01-16 06:42)
1. confusing means nothing => subject is not ok and same applis to description.
if confusing means: ' .. is returning and exec status, and there is no documentation that explains how the result is computed ..' then is a very, very different thing.

2. I can not understand why from 0005996 where request is:
'Need XMLRPC api for getExecutionResult that takes a build paramter'
you expect '.... getTestCasesForTestPlan will return no execution states anymore, ...'
If getTestCasesForTestPlan() returns the exec status and you do not need them , then just ignore it.

2014-01-17 18:56   
Hello fman,

thanks for your answer.
I will forward it to your customers of your api, who ask me, why you return in getTestCasesForTestPlan an exec_state, what is not up to date.

Thanks, Luiko
2014-01-17 20:46   
may be it would be better that your customer create an user here and ask directly instead of asking you and then you asking here, because in all this process things can be distorted.

In addition instead of using 'confusing status', why not explain in simple words that what seems to happens is that ALLWAYS first execution is returned.
Is this what is happening ?
Instead of copying the debug output is absolutely much better explain clearly and highlighting the important pieces.
Providing facts instead of having expectations is the approach I preferr in order to provide effective support.

In addition comparing results of getLastExecutionResult() with getgetTestCasesForTestPlan() may be has no sense.

If you have paying customer fot your services regarding TestLink, may be you can start thinking about supporting TestLink work in some way.
2014-01-18 10:27   
(edited on: 2014-01-18 10:55)

1. create test project ISSUE-0006159, with prefix IU6159
2. import test spec using file ISSUE-0006159.testproject-deep.xml
3. create test plan TPLAN-A
4. create build 1.0
5. populate test plan using file tree_TPLAN-A.xml
6. DO NOT EXECUTE any test case
7. call API using file (client_issue6159_t1.php)(copy it on directory where all other php samples exist and adjust devkey and testplan id)

8. see attached image test01-issue6159.png, where you will see details of parameter used on call and return value.
execution value is N => not run => OK

9. Using TestLink GUI, set exec result to Passed
10. call API AGAIN using file client_issue6159_t1.php.

11 see attached image(test02-issue6159-exec-to-passed.png)
Expected result is exec set to PASSED, got PASSED => OK

12. Using TestLink GUI, set exec result to FAILED
13. call API AGAIN using file client_issue6159_t1.php.

14 see attached image (test03-issue-6159-second-run-to-failed.png),
Expected result is exec set to FAILED, got FAILED => OK

then with latest code from gitoriuos (this means 1.9.9 with Latest fixes => what will become 1.9.10) I can not reproduce issue.

If time available I will try to do more tests.

I suppose you can also said to your customer the test I've executed in order to
check if issue will be present on next version.

15. using client_issue6159_t2.php (call to API is done providing testplan and testcase internal id, still no issue (see attached file test02*.png)

2014-01-18 11:01   
thanks for your retests.

Some remarks to your feedback 20337:
1. I have no paying customers. I just try to help in my fry time.
2. I thought, best way to support your project would be, to send your detail data your system creates. I understand now, that this does not help you. Sorry, my failure.
3. I will start a retest with your latest code in the next days
2014-01-18 22:04   
(edited on: 2014-01-18 22:40)
retest with latest code from GITORIOUS and postgresql 9.2 TL installation fails again, getTestCasesForTestPlan still returns NOT up-to-date exec_states.

BUT running equal test code against the TL 199 demo application returns correct up-to-date exec_states.

Suggestion - the used database TYPE has an influence.

Maybe, the sql statement , build in
  lib\functions\testplan.class.php -> getLinkedTCVersionsSQL
is not correct or behaves different between psql / mysql?

In psql, the union['exec'] statement returns for each execution of a test case a separate row. But we need only the row "MAX(" .

With following change although the postgresql 9.2 TL installtions returns up-to-date exec_states.


  function getLinkedTCVersionsSQL($id,$filters=null,$options=null)
    $union['exec'] = "/* {$debugMsg} sqlUnion - executions */" . $commonFields .
         " AND E.platform_id = TPTCV.platform_id " .
     // start add
         " AND = " .
     // end add
        $buildClause['exec_join'] .

2014-01-18 22:12   
My fault: I've not give enough attention to DBMS type.
I've tested on MySQL.

Can be some MySQL thing, because MySQL tends to accept SQL that are not completely right.
Or may be my SQL was wrong but tested on MySQL, due to some MySQL implementation details, returns what I expect.

I'm going to try to install postgres on my fedora 20 box, to try your fix
Need to test on other TestLink feature also.

thanks for your help
2014-01-18 22:49   
JUST commited (not tested yet), added ondition on both pieces of UNION
2014-01-19 10:22   
Both MySQL and Postgres return (without the fix) same recordset BUT IN DIFFERENT ORDER.
In my tests (passed, failed): MySQL return record 1 for passed, 2 for failed,
while Postgres returns 1 for failed and 2 for passed.
The implementation on TestLink database class, probably (I've not checked it)
uses the LATEST record on record set, that on MySQL provides right results but on Postgres WRONG

Then issue reason is multiple:
1. bad SQL Sentence (missing piece) (the MAIN REASON)
2. order that different DBMS use to return records
3. Testlink database class method implementations.
2014-04-25 17:38   
1.9.10 released