|Anonymous | Login | Signup for a new account||2020-02-27 14:48 UTC|
|Main | My View | View Issues | Change Log | My Account|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0004451||TestLink||Reports||public||2011-05-02 17:51||2011-07-02 13:49|
|Platform||Teslink||OS||Windows & Linux-Debian||OS Version||1.9.1|
|Product Version||1.9.2 (2011 Q2 - bug fixing)|
|Fixed in Version||1.9.3 (2011 Q3 - bug fixing)|
|Summary||0004451: Test Result Matrix may indicate invalid test case version when test case is not run for a build|
|Description||Test result matrix report does not take into account the current TC version on builds for which TC is not run, when the same TC has been executed on another build.|
If TC was not executed for any build, the version is correctly displayed. (current version assigned to the test plan)
(see attached hardcopy images)
|Steps To Reproduce||1 - Define a test case (version 1), a test plan with 2 builds (build1 and build 2)|
2 - Assign the version 1 of TC to the test plan
3 - Execute the TC for build1 (result passed for example)
4 - Build report Test Result Matrix (check result => passed[V1] for build1 and notrun [v1] for build2)
5 - Create a new version (2) of the TC and assign this version to the Test Plan
6 - Build report Test Result Matrix (check result => passed[V1] for build1 and notrun [v1] for build2) => now for build2, we should have notrun[v2]
|Tags||No tags attached.|
|Database (MySQL,Postgres,etc)||mySQL 5.x|
|Browser||any (IE,8 - Firefox 3.6.15)|
|QA Team - Task Workflow Status|
|Attached Files|| testresultmatrix.jpg [^] (89,913 bytes) 2011-05-02 17:51
Reminder sent to: Julian
hmm, may be I'm wrong, but I think we have fixed this. Do you remember something ?
I was able to reproduce.
will give a quick look.
$linkedTCVersion = $tcase['version'];
-> this is the version of the test case from a previous build column where a result was set
build1: P [v1]
build2: N -> v1 according to build 1
build3: N -> v1 according to build 1
build4: F [v2]
// If no execution was found => not run
if( $resultsForBuild === null )
$resultsForBuildText .= sprintf($versionTag,$linkedTCVersion);
How about this case:
build1: passed [v1]
build2: not run [v?]
build3: failed [v2]
build4: not run [v?]
AND test case is now linked to test plan in version 3
Let me know which version you expect to be shown for build 2 and build 4.
|@frl: your feedback is required...|
Without user feedback i will not act.
I do not like the behaviour when users only report an issue but do not participate in further investigations.
Sorry for the delay of my response (I just get your request for feedback)
The given example raises several questions (more than I initially thought),
which could lead to full versioning of test plans (like reqs and test cases)
to save history of test case assignment and associated build with test plan
versions. This is only way to give correct value in all cases.
May be a nice feature for future TL versions, but seems out of scope here.
As testplan_tcversions.tcversion_id is overwritten when assigning another TC version to TP, I did not find a workaround to get and use timestamps and retrieve the correct version for each build.
Main expectation here is to get a "confidence level" on result given by TC execution on a build, in the case of TC version update (new TC version assigned to the TP)
If you execute a TC on a build on which the TC was never executed, TL will set result for the version now linked to TP => This should be the version indicated
=> Therefore NotRun[Vx] means that Vx will be the TC version
which will determine the result for this build (according to current TP definition)
In your last example if I execute now the TC, without any change on TC version assignment on TP :
I would get F,P or B [v3] once execution is saved (even for build2 !!!).
This is quite strange, I agree, but without an history of TC version assignment for each build, I do not find more accurate result.
Moreover if TC has been executed for all builds => we cannot see in the report that a new TC version is now linked to the TP
Another (simple) possibility could be to add a separate column with the current TC version linked to TP, and remove the displayed version for build on which test was not run, giving a resyult like this one (with your example)
TestSuite TestCase Version build1 build2 build3 build4
my_test_suite my_test_case [v3] P[v1] N F[v2] N
=> Seems to be the best solution (avoiding all confusions)
As you already mentioned test plan versioning is out of our scope.
I personally prefer the simple solution of just removing version from not run test cases as i think this information is not very valuable.
Removed version tag for not run test cases.
|2011-05-02 17:51||frl||New Issue|
|2011-05-02 17:51||frl||File Added: testresultmatrix.jpg|
|2011-05-04 21:36||fman||Note Added: 0014791|
|2011-05-05 06:54||Julian||Note Added: 0014793|
|2011-05-05 07:18||Julian||Note Added: 0014794|
|2011-05-05 07:18||Julian||Assigned To||=> Julian|
|2011-05-05 07:18||Julian||Status||new => feedback|
|2011-05-06 14:03||Julian||Note Added: 0014807|
|2011-05-10 13:59||Julian||Note Added: 0014822|
|2011-05-12 00:07||frl||Note Added: 0014830|
|2011-05-12 00:07||frl||Status||feedback => assigned|
|2011-05-12 06:30||Julian||Note Added: 0014833|
|2011-05-12 06:54||Julian||Note Added: 0014835|
|2011-05-12 06:54||Julian||Status||assigned => resolved|
|2011-05-12 06:54||Julian||Fixed in Version||=> 1.9.3 (2011 Q3 - bug fixing)|
|2011-05-12 06:54||Julian||Resolution||open => fixed|
|2011-05-12 13:06||Julian||Relationship added||child of 0004337|
|2011-07-02 13:49||fman||Note Added: 0015453|
|2011-07-02 13:49||fman||Status||resolved => closed|
|Copyright © 2000 - 2020 MantisBT Team|