Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007013TestLinkTest Specificationpublic2015-03-19 21:092015-03-22 19:30
ReporterMilos.Kozina 
Assigned Tofman 
PrioritynormalSeverityfeature requestReproducibilityalways
StatusassignedResolutionopen 
PlatformOSOS Version
Product Version1.9.13 (2015 #1) 
Fixed in Version 
Summary0007013: Filtering according execution type includes all TC version - add filter for last version only
DescriptionFiltering according execution type includes all TC version - add filter for last version only. Such filter should be check box 'Last versions only', which would restrict search for last version of TC. Note TC = Test Case.
Steps To Reproduce1. Create TC-1 in version 1 (v1) with execution type Manual.
2a. Search in Test specification for TC with Execution type Manual => result will
contain TC-1 => OK.
2b. Search in Test specification for TC with Execution type Automated => result will NOT contain TC-1 => OK.
3. Later on (after execution of TC-1) create new version of TC-1 (v2) and switch its execution to Automated.
4a. Search in Test specification for TC with Execution type Manual => result will contain TC-1 (included because of v1) => NOK.
4b. Search in Test specification for TC with Execution type Automated => result will contain TC-1 (included because of v2) => OK.

Result from step 4a. is confusing - you get the same TC in two situations, which should not be the same (either Automated or Manual). Therefore it would be nice, if search can be filtered for last version of TC only by check box.

Proposed behaviour with check box 'Last versions only' checked":
5a. Search in Test specification for TC with Execution type Manual => result will NOT contain TC-1 => OK.
5b. Search in Test specification for TC with Execution type Automated => result will contain TC-1 (included because of v2) => OK.

Proposed behaviour with check box 'Last versions only' unchecked" - steps 4a, 4b.
TagsNo tags attached.
Database (MySQL,Postgres,etc)MySQL
Browser
PHP Version
TestCaseID
QA Team - Task Workflow StatusTBD
Attached Filespng file icon search-testcases.png [^] (41,837 bytes) 2015-03-19 22:46


png file icon TL_search_in_test_specification.png [^] (30,943 bytes) 2015-03-20 07:48


txt file icon tc_versions_query_result_with_timestamps.txt [^] (2,531 bytes) 2015-03-22 11:02 [Show Content]

- Relationships

-  Notes
(0022935)
fman (administrator)
2015-03-19 22:45

Can you add screenshot with search screen?
I'm using 'Search Test Cases', and do not see any posibility to filter by execution type,
(see my attached screen)
(0022936)
Milos.Kozina (reporter)
2015-03-20 07:50

Please see attached screenshot - it is in Test specification module.
(0022937)
fman (administrator)
2015-03-20 09:59

You are filtering on test spec, that unfortunately is different to the feature 'SEARCH TEST CASES', that's why always very detailed step to reproduce (and images only on exceptional situations) are needed
(0022939)
Milos.Kozina (reporter)
2015-03-20 12:06

OK, sorry for putting feature request to wrong category. I meant Test specification module (and I expect, that same filtering is used in Add / Remove Test Cases and in Assign Test Case Execution). Is the requirement valid and clear now?

Thanks, Milos
(0022940)
fman (administrator)
2015-03-20 12:19
edited on: 2015-03-20 12:21

>> (and I expect, that same filtering is used in Add / Remove Test Cases and >>in Assign Test Case Execution).
This is a Thing that IMHO a reported do not have to do, i.e. try to guess.
Things can look the same on GUI but how are applied are different.


THis issue has lot of problems:
is not matter of category, is how info has been described providing misleading context.

You need to open ONE ISSUE for each testlink feature where you want a change no matter how similar they seems.
Then THIS issue is just for filtering tree (that is very different matter than search) ON TEST SPEC.
I've checked (I'm going to double check) but it seems to WORK as expected.

You need to open MORE feature requests if you want to do a request for same field on other TestLink features.


THis is the way I work, following recognized standards on issue reporting.

Here there are more general guide lines
http://www.softwaretestinghelp.com/how-to-write-good-bug-report/ [^]


(0022947)
Milos.Kozina (reporter)
2015-03-22 11:25

During writing of comment (which was lost due to expired session :-/, so I'm writing it again), I've found, that my problem is restricted to some limited TC only. I've added DB queries for them (please ignore file tc_versions_query_result.txt, there's no way, how to delete it) and realized, that affected TC (which are returned in Tree filter for value Automated, although the last version is Manual) have same creation_ts, although it the versions were created by human and therefore it cannot be in same second. This leads me to idea, that either there's bug in creation_ts and it is reproducible. And for such created TC the filtering is not working.

Reproduce steps:
1. Create new TC (TC-1) with Execution type Automated.
2. Create new version of TC-1 in same browser in same session (do not switch to any other TC, etc.) - let Execution type value as Automated.
3. Repeat point 2 more times.
4. Create new version of TC-1 in same browser in same session (do not switch to any other TC, etc.) - and switch Execution type Manual.
5. Check creation timestamp for all created versions.
6. Go to Test specification module, select in filter Execution type Automated and press Apply filter.

Actual result:
5.:There's same creation timestamp for all created versions of TC-1.
6.:TC-1 is included in filtered result in TC tree, although it's last version has Execution type Manual.

Expected result:
5.:Creation timestamp is different for each created version of TC-1.
6.:TC-1 is NOT included in filtered result in TC tree, because it's last version has Execution type Manual.
(0022949)
fman (administrator)
2015-03-22 11:39

Thanks for your help

Ok, I'm going to give a look to this.
From time to time, I've tried different (and seems to be) wrong approachs to get LATEST Version.
Using node id, seems sometimes to generate issues because VERSION that human see as 2 has node id < node id for Version 1.

I think that may be for this I've used time stamp, but as you have found seems that this also have issues.

Anyway, please understand that is much better if you add a issue for each feature where do you think the filter has to work this way.

IMHO when we are accessing Test Specification (and may be is the scenario you have hitted) acting on latest (active only?) version is ok, and simple.
But when you start mixing this with execution results (as happens on execution feature) things become complex and confusing.
(0022950)
Milos.Kozina (reporter)
2015-03-22 12:41

Please see tc_versions_query_result_with_timestamps.txt - there's visible, that newer versions have lower id, then lower versions.

Regarding of creating of other issues - I think, that now I've isolated the origin of the trouble, so I'll wait for fix and then test it in other modules.

Last execution is more complicated, I know, because I've created some direct SQL reports from Testlink DB. Last execution depends on too many inputs - build, TC, TC version, platform, so there it is not easy... Maybe some additional columns could help (but it depends on design of needs).
(0022951)
fman (administrator)
2015-03-22 14:50
edited on: 2015-03-22 15:02

There is something very strange
1) on tc_versions_query_result.txt
you can not have multiple records with same VERSION for SAME external ID
see lines marked with <<<<

mysql> select id, tc_external_id, version, execution_type  from tcversions WHERE tc_external_id IN  
( 1305, 1307 ) order by 2, 3;
+--------+----------------+---------+----------------+
| id     | tc_external_id | version | execution_type |
+--------+----------------+---------+----------------+
|  93560 |           1305 |       1 |              1 |   <<<<   THIS IS NOT OK    
| 274859 |           1305 |       1 |              1 |  <<<<
| 282948 |           1305 |       1 |              1 |  <<<<
| 269592 |           1305 |       1 |              2 |  <<<<
| 269582 |           1305 |       2 |              2 |
| 269572 |           1305 |       3 |              2 |
| 269562 |           1305 |       4 |              2 |
| 269552 |           1305 |       5 |              2 |
| 269542 |           1305 |       6 |              2 |
| 269532 |           1305 |       7 |              2 |
| 269522 |           1305 |       8 |              1 |
| 281977 |           1307 |       1 |              1 | <<<<  THIS IS NOT OK
| 274897 |           1307 |       1 |              1 | <<<< 
|  93566 |           1307 |       1 |              1 |
| 269826 |           1307 |       1 |              2 |
| 269814 |           1307 |       2 |              2 |
| 269802 |           1307 |       3 |              2 |
| 269790 |           1307 |       4 |              2 |
| 269778 |           1307 |       5 |              2 |
| 269766 |           1307 |       6 |              2 |
| 269754 |           1307 |       7 |              2 |
| 269742 |           1307 |       8 |              1 |
+--------+----------------+---------+----------------+


Another BIG issue is that HIGHER VERSIONS has to HAVE a TCV.ID > than lower
because version 5 is created AFTER version 4.

The question is:
how this data has been generated ?
is result of some SQL sentences done OUTSIDE of TestLink ?

Lot of logic has been done, and IMHO is right, knowing that ID you get from nodes_hierarchy table will increase EVER.

(0022952)
fman (administrator)
2015-03-22 15:05

I've created on test case, and then several versions one after another without doing any other operation in between.
All tests done on laptop where testlink is running.
All creation_ts where different, and tcv.id increased in each version
(0022953)
Milos.Kozina (reporter)
2015-03-22 16:05

There can be more TC with same tc_external_id in case, when you copy complete test project or when you have more test projects with more then 1307 test cases. Then it is OK.

That's why I've done second query, where connection by node parent is visible (and view is restricted to this 2 problematic TC 1305 and 1307). Because of tree structure in nodes_hierarchy I don't have easy way to ensure the complete assignment of TC to test project (maybe there's solution via test plan).
(0022954)
fman (administrator)
2015-03-22 16:41

>> There can be more TC with same tc_external_id
OK my fault, anyway on second query still we have issues because node_id has to increase any time a new version is created.

again
what about this:
>> The question is:
>> how this data has been generated ?
>> is result of some SQL sentences done OUTSIDE of TestLink ?
(0022955)
Milos.Kozina (reporter)
2015-03-22 19:23

Data were created by normal usage. I've tried to reproduce it on my local installation and creation timestamp is OK and also id is increasing. So it seems, that it depends on environment... I'll try to find out differences and come back, but I have no idea, what could be wrong.
(0022956)
fman (administrator)
2015-03-22 19:30

OK, thanks.
it's very strange.
Time stamps (TestLink uses ADODB PHP) with certain DBMS (example MySQL) can not have millisec on each DBMS version, then doing this change in stable code will be an issue, and other issue opens: how to migrate old data? => no solution.
If date/time will be stored as mantis does (UNIX timestamp, i.e. an integer) again we have an issue again with migration of old data.

Some fix can be used for TEST CASE VERSIONS (human readable version could be used), but other pieces of TestLink like executions do not have an attribute like this, then only way to have LATEST value is:
a) use MAX(EXECUTION.id)
b) use MAX(creation_ts)

but in anycase what is critic is that on each insert the attribute has a monotonic behaviour, thing that in your case was not verified.
very strange.
Right now IMHO you have not a simple way to fix what is happening

regards

- Issue History
Date Modified Username Field Change
2015-03-19 21:09 Milos.Kozina New Issue
2015-03-19 21:19 fman Note Added: 0022931
2015-03-19 21:33 fman Note Added: 0022933
2015-03-19 21:50 fman QA Team - Task Workflow Status => TBD
2015-03-19 21:50 fman Category Filters => Test Specification
2015-03-19 21:50 fman Note Deleted: 0022933
2015-03-19 21:50 fman Note Deleted: 0022931
2015-03-19 22:45 fman Note Added: 0022935
2015-03-19 22:46 fman File Added: search-testcases.png
2015-03-19 22:46 fman Assigned To => fman
2015-03-19 22:46 fman Status new => feedback
2015-03-19 22:48 fman Category Test Specification => Test Spec. - Search Test Cases
2015-03-20 07:48 Milos.Kozina File Added: TL_search_in_test_specification.png
2015-03-20 07:50 Milos.Kozina Note Added: 0022936
2015-03-20 07:50 Milos.Kozina Status feedback => assigned
2015-03-20 09:59 fman Note Added: 0022937
2015-03-20 12:06 Milos.Kozina Note Added: 0022939
2015-03-20 12:19 fman Note Added: 0022940
2015-03-20 12:19 fman Note Edited: 0022940 View Revisions
2015-03-20 12:21 fman Note Edited: 0022940 View Revisions
2015-03-20 12:21 fman Category Test Spec. - Search Test Cases => Test Specification
2015-03-20 12:22 fman Status assigned => feedback
2015-03-22 09:27 Milos.Kozina File Added: tc_versions_query_result.txt
2015-03-22 11:02 Milos.Kozina File Added: tc_versions_query_result_with_timestamps.txt
2015-03-22 11:25 Milos.Kozina Note Added: 0022947
2015-03-22 11:25 Milos.Kozina Status feedback => assigned
2015-03-22 11:39 fman Note Added: 0022949
2015-03-22 12:41 Milos.Kozina Note Added: 0022950
2015-03-22 14:50 fman Note Added: 0022951
2015-03-22 14:57 fman Note Edited: 0022951 View Revisions
2015-03-22 14:58 fman Note Edited: 0022951 View Revisions
2015-03-22 15:02 fman Note Edited: 0022951 View Revisions
2015-03-22 15:03 fman Note View State: 0022951: public
2015-03-22 15:05 fman Note Added: 0022952
2015-03-22 16:05 Milos.Kozina Note Added: 0022953
2015-03-22 16:40 fman File Deleted: tc_versions_query_result.txt
2015-03-22 16:41 fman Note Added: 0022954
2015-03-22 19:23 Milos.Kozina Note Added: 0022955
2015-03-22 19:30 fman Note Added: 0022956



Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker