Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006159TestLinkAPI - XMLRPCpublic2014-01-15 21:022014-04-25 17:38
Reporterlczub 
Assigned Tofman 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.9.9 (2013 Q4 - bug fixing) 
Fixed in Version1.9.10 (2014 Q1 - bug fixing) 
Summary0006159: getTestCasesForTestPlan returns confusing exec_status [POSTGRESQL]
Descriptionusers of the xml-rpc api reports, that getTestCasesForTestPlan returns 'exec_status' values, which does not match the last execution state of a test case.

Following the discussion in 0005996, the expectation is, that getTestCasesForTestPlan will return no execution states anymore, cause getLastExecutionResult should be used.
But in TL 199, getTestCasesForTestPlan still returns the confusing 'exec_status'.
Steps To Reproducemanual steps in TL:
1. create project 'P-A' with test suite 'TS-A', test case 'TC-A',
2. create test plan 'TP-A' with build 'B-A' and add test case 'TC-A'
3. save FIRST execution of test case 'TC-A' = passed

API call from client
4. call xml-rpc api method getTestCasesForTestPlan with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'exec_status': 'p' - ok
   {'6047': [{'tcase_id': '6047', ... , 'exec_status': 'p'}]}
   
manual steps in TL:
5. save SECOND execution of test case 'TC-A' = failed

API call from client
6. call xml-rpc api method getTestCasesForTestPlan with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'exec_status': 'p' ->> CONFUSION - 'f' expected
   {'6047': [{'tcase_id': '6047', ... , 'exec_status': 'p'}]}
7. call xml-rpc api method getLastExecutionResult with
   - testplanid = id of test plan 'TP-A'
   - testcaseid = id of test case 'TC-A'
-> return values includes 'status': 'f' - ok
   [{'build_id': '281', 'status': 'f', ... }]
TagsNo tags attached.
Database (MySQL,Postgres,etc)postgres 9.2
Browser
PHP Version
TestCaseID
QA Team - Task Workflow StatusREADY FOR TESTING
Attached Filesxml file icon ISSUE-0006159.testproject-deep.xml [^] (928 bytes) 2014-01-18 10:23
xml file icon tree_TPLAN-A.xml [^] (797 bytes) 2014-01-18 10:25
png file icon test01-issue6159.png [^] (24,357 bytes) 2014-01-18 10:43


png file icon test02-issue6159-exec-to-passed.png [^] (24,456 bytes) 2014-01-18 10:43


png file icon test03-issue-6159-second-run-to-failed.png [^] (24,316 bytes) 2014-01-18 10:43


? file icon client_issue6159_t1.php [^] (757 bytes) 2014-01-18 10:47
? file icon client_issue6159_t2.php [^] (781 bytes) 2014-01-18 10:54
png file icon test02-using-testcase-internal-id.png [^] (25,226 bytes) 2014-01-18 10:54

- Relationships
child of 0006048closedfman Availables hot-fixes for 1.9.9 & How To get full fixed package from gitorious 

-  Notes
(0020332)
fman (administrator)
2014-01-16 06:38
edited on: 2014-01-16 06:42

1. confusing means nothing => subject is not ok and same applis to description.
if confusing means: ' .. is returning and exec status, and there is no documentation that explains how the result is computed ..' then is a very, very different thing.

2. I can not understand why from 0005996 where request is:
'Need XMLRPC api for getExecutionResult that takes a build paramter'
you expect '.... getTestCasesForTestPlan will return no execution states anymore, ...'
If getTestCasesForTestPlan() returns the exec status and you do not need them , then just ignore it.

(0020335)
lczub (reporter)
2014-01-17 18:56

Hello fman,

thanks for your answer.
I will forward it to your customers of your api, who ask me, why you return in getTestCasesForTestPlan an exec_state, what is not up to date.

Thanks, Luiko
(0020337)
fman (administrator)
2014-01-17 20:46

may be it would be better that your customer create an user here and ask directly instead of asking you and then you asking here, because in all this process things can be distorted.

In addition instead of using 'confusing status', why not explain in simple words that what seems to happens is that ALLWAYS first execution is returned.
Is this what is happening ?
Instead of copying the debug output is absolutely much better explain clearly and highlighting the important pieces.
Providing facts instead of having expectations is the approach I preferr in order to provide effective support.

In addition comparing results of getLastExecutionResult() with getgetTestCasesForTestPlan() may be has no sense.

If you have paying customer fot your services regarding TestLink, may be you can start thinking about supporting TestLink work in some way.
(0020339)
fman (administrator)
2014-01-18 10:27
edited on: 2014-01-18 10:55

ALL TESTS DONE USING Latest code from GITORIOUS

1. create test project ISSUE-0006159, with prefix IU6159
2. import test spec using file ISSUE-0006159.testproject-deep.xml
3. create test plan TPLAN-A
4. create build 1.0
5. populate test plan using file tree_TPLAN-A.xml
6. DO NOT EXECUTE any test case
7. call API using file (client_issue6159_t1.php)(copy it on directory where all other php samples exist and adjust devkey and testplan id)

8. see attached image test01-issue6159.png, where you will see details of parameter used on call and return value.
execution value is N => not run => OK

9. Using TestLink GUI, set exec result to Passed
10. call API AGAIN using file client_issue6159_t1.php.

11 see attached image(test02-issue6159-exec-to-passed.png)
Expected result is exec set to PASSED, got PASSED => OK

12. Using TestLink GUI, set exec result to FAILED
13. call API AGAIN using file client_issue6159_t1.php.

14 see attached image (test03-issue-6159-second-run-to-failed.png),
Expected result is exec set to FAILED, got FAILED => OK

then with latest code from gitoriuos (this means 1.9.9 with Latest fixes => what will become 1.9.10) I can not reproduce issue.

If time available I will try to do more tests.

I suppose you can also said to your customer the test I've executed in order to
check if issue will be present on next version.

15. using client_issue6159_t2.php (call to API is done providing testplan and testcase internal id, still no issue (see attached file test02*.png)

(0020340)
lczub (reporter)
2014-01-18 11:01

thanks for your retests.

Some remarks to your feedback 20337:
1. I have no paying customers. I just try to help in my fry time.
2. I thought, best way to support your project would be, to send your detail data your system creates. I understand now, that this does not help you. Sorry, my failure.
3. I will start a retest with your latest code in the next days
(0020341)
lczub (reporter)
2014-01-18 22:04
edited on: 2014-01-18 22:40

retest with latest code from GITORIOUS and postgresql 9.2 TL installation fails again, getTestCasesForTestPlan still returns NOT up-to-date exec_states.

BUT running equal test code against the TL 199 demo application returns correct up-to-date exec_states.

Suggestion - the used database TYPE has an influence.

Maybe, the sql statement , build in
  lib\functions\testplan.class.php -> getLinkedTCVersionsSQL
is not correct or behaves different between psql / mysql?

In psql, the union['exec'] statement returns for each execution of a test case a separate row. But we need only the row "MAX(execution.id)" .

With following change although the postgresql 9.2 TL installtions returns up-to-date exec_states.

getLinkedTCVersionsSQL

  function getLinkedTCVersionsSQL($id,$filters=null,$options=null)
   ...
    $union['exec'] = "/* {$debugMsg} sqlUnion - executions */" . $commonFields .
    ...
         " AND E.platform_id = TPTCV.platform_id " .
     // start add
         " AND E.id = LEX.id " .
     // end add
        $buildClause['exec_join'] .
    ...

(0020342)
fman (administrator)
2014-01-18 22:12

My fault: I've not give enough attention to DBMS type.
I've tested on MySQL.

Can be some MySQL thing, because MySQL tends to accept SQL that are not completely right.
Or may be my SQL was wrong but tested on MySQL, due to some MySQL implementation details, returns what I expect.

I'm going to try to install postgres on my fedora 20 box, to try your fix
Need to test on other TestLink feature also.

thanks for your help
(0020343)
fman (administrator)
2014-01-18 22:49

JUST commited (not tested yet), added ondition on both pieces of UNION
(0020344)
fman (administrator)
2014-01-19 10:22

Both MySQL and Postgres return (without the fix) same recordset BUT IN DIFFERENT ORDER.
In my tests (passed, failed): MySQL return record 1 for passed, 2 for failed,
while Postgres returns 1 for failed and 2 for passed.
The implementation on TestLink database class, probably (I've not checked it)
uses the LATEST record on record set, that on MySQL provides right results but on Postgres WRONG

Then issue reason is multiple:
1. bad SQL Sentence (missing piece) (the MAIN REASON)
2. order that different DBMS use to return records
3. Testlink database class method implementations.
(0020848)
fman (administrator)
2014-04-25 17:38

1.9.10 released

- Issue History
Date Modified Username Field Change
2014-01-15 21:02 lczub New Issue
2014-01-16 06:38 fman Note Added: 0020332
2014-01-16 06:38 fman Note View State: 0020332: public
2014-01-16 06:42 fman Note Edited: 0020332 View Revisions
2014-01-17 18:56 lczub Note Added: 0020335
2014-01-17 20:46 fman Note Added: 0020337
2014-01-18 10:23 fman File Added: ISSUE-0006159.testproject-deep.xml
2014-01-18 10:25 fman File Added: tree_TPLAN-A.xml
2014-01-18 10:27 fman Note Added: 0020339
2014-01-18 10:28 fman Note Edited: 0020339 View Revisions
2014-01-18 10:43 fman Note Edited: 0020339 View Revisions
2014-01-18 10:43 fman File Added: test01-issue6159.png
2014-01-18 10:43 fman File Added: test02-issue6159-exec-to-passed.png
2014-01-18 10:43 fman File Added: test03-issue-6159-second-run-to-failed.png
2014-01-18 10:44 fman Note Edited: 0020339 View Revisions
2014-01-18 10:46 fman Note Edited: 0020339 View Revisions
2014-01-18 10:46 fman Note Edited: 0020339 View Revisions
2014-01-18 10:47 fman File Added: client_issue6159_t1.php
2014-01-18 10:47 fman Note Edited: 0020339 View Revisions
2014-01-18 10:48 fman Note Edited: 0020339 View Revisions
2014-01-18 10:48 fman Note Edited: 0020339 View Revisions
2014-01-18 10:49 fman Note Edited: 0020339 View Revisions
2014-01-18 10:49 fman Note View State: 0020339: public
2014-01-18 10:50 fman Assigned To => fman
2014-01-18 10:50 fman Status new => feedback
2014-01-18 10:54 fman File Added: client_issue6159_t2.php
2014-01-18 10:54 fman File Added: test02-using-testcase-internal-id.png
2014-01-18 10:55 fman Note Edited: 0020339 View Revisions
2014-01-18 10:55 fman Note Edited: 0020339 View Revisions
2014-01-18 11:01 lczub Note Added: 0020340
2014-01-18 11:01 lczub Status feedback => assigned
2014-01-18 22:04 lczub Note Added: 0020341
2014-01-18 22:12 fman Note Added: 0020342
2014-01-18 22:13 fman QA Team - Task Workflow Status => TBD
2014-01-18 22:13 fman Summary getTestCasesForTestPlan returns confusing exec_status => getTestCasesForTestPlan returns confusing exec_status [POSTGRESQL]
2014-01-18 22:40 fman Note Edited: 0020341 View Revisions
2014-01-18 22:49 fman Note Added: 0020343
2014-01-19 10:22 fman Note Added: 0020344
2014-01-19 10:22 fman QA Team - Task Workflow Status TBD => READY FOR TESTING
2014-01-19 10:22 fman Status assigned => resolved
2014-01-19 10:22 fman Fixed in Version => 1.9.10 (2014 Q1 - bug fixing)
2014-01-19 10:22 fman Resolution open => fixed
2014-01-19 10:22 fman Relationship added child of 0006048
2014-04-25 17:38 fman Note Added: 0020848
2014-04-25 17:38 fman Status resolved => closed



Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker