Mantis Bugtracker          
testlink.org

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002463TestLinkReportspublic2009-05-01 00:102011-03-18 19:07
Reportermarkhu 
Assigned Toasimon 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version1.7.1 
Fixed in Version1.9 (Prague) 
Summary0002463: General Test Plan Metrics results report: Results by Tester wrong
DescriptionIn the General Test Plan Metrics results report, under the
Results by Tester heading, the Tester column can be wrong because it lists currently-assigned instead of who actually executed.

An example:
On Monday, Mark is assigned 20 test cases, but only executes the first 11.
On Tuesday, Pooja is assigned the same 20 test cases, and executes the remaining 9.
At this point, viewing the report would show that Pooja did all 20, which is clearly wrong.
TagsNo tags attached.
Database (MySQL,Postgres,etc)
Browserall
PHP Version
TestCaseID
QA Team - Task Workflow Status
Attached Filespng file icon testlink-results-report1.png [^] (37,465 bytes) 2009-05-01 06:06


png file icon user1_result.PNG [^] (7,749 bytes) 2010-03-24 16:34


png file icon user2_result.PNG [^] (8,213 bytes) 2010-03-24 16:34


jpg file icon test-1-fman.jpg [^] (25,056 bytes) 2010-04-26 01:23


jpg file icon test-2-fman.jpg [^] (25,242 bytes) 2010-04-26 01:23

- Relationships

-  Notes
(0006704)
fman (administrator)
2009-05-01 00:20

may be just label is wrong.
(0006708)
markhu (reporter)
2009-05-01 06:07

attached a screenshot to illustrate. It seems to be doing the query based on current assignments rather than execution history.
(0006722)
fman (administrator)
2009-05-03 22:29

that is waht I meant for wrong label, we have used tester instead of assigned tester, then if with assigned tester figures are ok, problem is not feature but label
(0009503)
slither (reporter)
2010-03-24 16:04

If the fix for this is to change the label, then may I request that a real "Results by Tester" section be added to the "General Test Plan Metrics" report?

Metrics by User is pretty useful to gauge the work done by a specific user.

Thanks.
(0009504)
Julian (reporter)
2010-03-24 16:12
edited on: 2010-03-24 17:17

this should be improved in my eyes.

how about showing "Results by Assigned Tester for Latest Build" ?

I would like to again give steps to reproduce "wrong" behaviour:

Create 5 test cases
Create 2 users

Create Build 1
Assign 5 test cases to user1
Execute the 5 assigned test cases with user1
-> result by tester: user1 = 100% (Attachment: user1_result.png)

Create Build 2
Assign the same 5 test cases to user2
Do NOT excute test cases
-> actual result by tester: user2 = 100% (Attachment: user1_result.png)
-> expected result by tester: user2 = 0%

(0009800)
Julian (reporter)
2010-04-21 12:51

Reminder sent to: fman

Hi fman,

please check this. do not think this is a huge change.
i would like to show Results by Assigned Tester for Latest Build on General Test Plan Metrics.

This would at least give reasonable values for the latest build. right now those values are not very helpful in my eyes. (They only make sense if only 1 build is defined)
(0009809)
fman (administrator)
2010-04-25 23:51

1. we need to accept that Latest Build will be always last created.
2. on your example we will display figures regarding, how many of the assigned tst cases has been executed by tester, then will be 0 on each column, because he/she has executed NO ONE. is ok ?

3. one additional issue that we have is we do not have history of test case execution - user assignments, then we are not able to evaluate how effective is a user (evaluate how many of assigned test cases they really execute). And I do not know if this kind of metrics can be interesting.
(0009810)
fman (administrator)
2010-04-26 01:23

1. test-1-fman.jpg -> test cases already executed by admin has been assigned to TESTERONE

2. test-2-fman.jpg -> TESTERONE logon and execute a test case that previouly has been executed by admin
(0009811)
Julian (reporter)
2010-04-26 02:25

i do not like this behaviour - is even more confusing than before.

we should discuss.

do you think it would be a good idea to specify the build result by tester is wanted for ?

this would make reports little more dynamic.
(0009814)
fman (administrator)
2010-04-26 16:47

1. disagree IMHO is not confusing, is clear WHO has executed test cases no matter to who execution was assigned.

2. is not possible relate BUILD to tester because ASSIGNMENT is done on TEST LAN not BUILd.
Please explain what do you mean
(0009816)
Julian (reporter)
2010-04-26 16:54
edited on: 2010-04-26 17:01

we could use a combobox offering all builds.

if user chooses a build of this list only results of this build will be used to create results by user.

algorithm could looke something like that

1. get all assignments for test plan
2. get all results for chosen build
3. "compare" results of chosen builds with assignments

EDIT: to keep current behaviour we could add another item on the combobox.

(0009817)
fman (administrator)
2010-04-26 17:40

>> we could use a combobox offering all builds.
then launch of this report need to be changed.

Why do not create a SEPARATED NEW report that regards ONLY RESULTS by USERS ?

>> EDIT: to keep current behaviour we could add another item on the combobox.
not clear explain better
(0009818)
Julian (reporter)
2010-04-26 17:45

a new page would be fine too.

lets dicuss this via skype when you find the time to avoid chat here ;)
(0013791)
Julian (reporter)
2011-03-06 17:30

should be resolved with testlink 1.9
(0013929)
fman (administrator)
2011-03-18 19:07

release - 1.9.2 - 20110320

- Issue History
Date Modified Username Field Change
2009-05-01 00:10 markhu New Issue
2009-05-01 00:10 markhu Browser => all
2009-05-01 00:20 fman Note Added: 0006704
2009-05-01 06:06 markhu File Added: testlink-results-report1.png
2009-05-01 06:07 markhu Note Added: 0006708
2009-05-03 22:29 fman Note Added: 0006722
2009-06-02 05:22 mhavlat Status new => assigned
2009-06-02 05:22 mhavlat Assigned To => mhavlat
2010-02-25 00:44 mhavlat Status assigned => acknowledged
2010-02-28 22:29 mhavlat Assigned To mhavlat =>
2010-03-24 16:04 slither Note Added: 0009503
2010-03-24 16:12 Julian Note Added: 0009504
2010-03-24 16:28 Julian Note Edited: 0009504
2010-03-24 16:34 Julian File Added: user1_result.PNG
2010-03-24 16:34 Julian File Added: user2_result.PNG
2010-03-24 16:35 Julian Note Edited: 0009504
2010-03-24 16:35 Julian Note Edited: 0009504
2010-03-24 17:17 Julian Note Edited: 0009504
2010-04-21 12:51 Julian Note Added: 0009800
2010-04-25 23:51 fman Note Added: 0009809
2010-04-26 01:23 fman Note Added: 0009810
2010-04-26 01:23 fman File Added: test-1-fman.jpg
2010-04-26 01:23 fman File Added: test-2-fman.jpg
2010-04-26 02:25 Julian Note Added: 0009811
2010-04-26 16:47 fman Note Added: 0009814
2010-04-26 16:54 Julian Note Added: 0009816
2010-04-26 16:55 Julian Note Edited: 0009816
2010-04-26 17:01 Julian Note Edited: 0009816
2010-04-26 17:40 fman Note Added: 0009817
2010-04-26 17:45 Julian Note Added: 0009818
2011-03-06 17:30 Julian Note Added: 0013791
2011-03-06 17:30 Julian Status acknowledged => resolved
2011-03-06 17:30 Julian Fixed in Version => 1.9 (Prague)
2011-03-06 17:30 Julian Resolution open => fixed
2011-03-06 17:30 Julian Assigned To => asimon
2011-03-18 19:07 fman Note Added: 0013929
2011-03-18 19:07 fman Status resolved => closed



Copyright © 2000 - 2020 MantisBT Team
Powered by Mantis Bugtracker