MantisBT - TestLink
View Issue Details
0004958TestLinkChartspublic2012-04-03 10:042012-09-01 19:58
awiechert 
fman 
urgentblockalways
closedfixed 
Red Hat Enterprise Linux Server 6.1
1.9.3 (2011 Q3 - bug fixing) 
1.9.4 (2012 Q3 - bug fixing) 
MYSQL
Firefox, IE, Safarie
5.3.3
READY FOR TESTING
0004958: General Test Plan Metrics shows white page - Bug tracking integration enabled
The test report: General Test Plan Metrics always shows a white page. In the error log of http I got the following:

[Tue Apr 03 05:06:18 2012] [error] [client 87.193.254.34] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 88 bytes) in /var/www/domains/managed.contegix.com/talend02/htdocs/testlink/lib/functions/object.class.php on line 107, referer: http://testlink.talendforge.org/lib/results/resultsNavigator.php [^]

We have inceased the memory_limit in php.ini from 128M to 256M and finaly to 512M.
Also we have set the following options in config.inc.php

ini_set("memory_limit","512M");
ini_set ("max_execution_time", "300");
No tags attached.
tar issue-4958.tar (112,640) 2012-04-07 10:22
http://mantis.testlink.org/file_download.php?file_id=2653&type=bug
Issue History
2012-04-03 10:04awiechertNew Issue
2012-04-03 18:25fmanNote Added: 0016491
2012-04-04 08:41awiechertNote Added: 0016494
2012-04-04 08:41awiechertStatusnew => feedback
2012-04-04 10:26fmanNote Added: 0016497
2012-04-04 10:32awiechertNote Added: 0016499
2012-04-04 10:32awiechertStatusfeedback => new
2012-04-04 10:36techieguyNote Added: 0016500
2012-04-04 11:42awiechertNote Added: 0016501
2012-04-04 11:59techieguyNote Added: 0016502
2012-04-04 12:36awiechertNote Added: 0016503
2012-04-04 14:20techieguyNote Added: 0016504
2012-04-04 14:23techieguyNote Edited: 0016504bug_revision_view_page.php?bugnote_id=16504#r1736
2012-04-04 14:38awiechertNote Added: 0016505
2012-04-04 17:11fmanNote Added: 0016510
2012-04-04 17:12fmanSummaryGeneral Test Plan Metrics shows white page => General Test Plan Metrics shows white page - Bug tracking integration enabled
2012-04-04 17:47awiechertNote Added: 0016514
2012-04-04 17:56fmanNote Added: 0016516
2012-04-04 18:06techieguyNote Added: 0016518
2012-04-04 18:22fmanNote Added: 0016519
2012-04-04 18:26fmanNote Edited: 0016519bug_revision_view_page.php?bugnote_id=16519#r1738
2012-04-05 11:39awiechertNote Added: 0016526
2012-04-05 14:12fmanNote Added: 0016529
2012-04-07 10:21fmanNote Added: 0016535
2012-04-07 10:22fmanFile Added: issue-4958.tar
2012-04-07 16:24fmanAssigned To => fman
2012-04-07 16:24fmanStatusnew => feedback
2012-04-10 08:28awiechertNote Added: 0016554
2012-04-10 08:29awiechertStatusfeedback => assigned
2012-04-10 10:16fmanNote Added: 0016555
2012-04-12 21:36fmanNote Added: 0016577
2012-04-12 21:39fmanNote Edited: 0016577bug_revision_view_page.php?bugnote_id=16577#r1763
2012-04-13 07:22awiechertNote Added: 0016578
2012-04-13 08:06fmanNote View State: 0016577: public
2012-04-13 08:06fmanNote Added: 0016580
2012-04-16 14:17awiechertNote Added: 0016591
2012-04-16 14:34fmanNote Added: 0016592
2012-04-16 14:36awiechertNote Added: 0016593
2012-04-16 15:45fmanNote Added: 0016594
2012-04-17 05:48fmanNote Added: 0016596
2012-04-20 22:23fmanNote Added: 0016602
2012-04-20 22:28fmanNote Edited: 0016602bug_revision_view_page.php?bugnote_id=16602#r1770
2012-07-04 09:39techieguyNote Added: 0016951
2012-07-04 12:12fmanNote Added: 0016952
2012-07-04 12:12fmanNote Edited: 0016952bug_revision_view_page.php?bugnote_id=16952#r1889
2012-08-17 18:18fmanTask Workflow Status => READY FOR TESTING
2012-08-17 18:18fmanStatusassigned => resolved
2012-08-17 18:18fmanFixed in Version => 1.9.4 (2012 Q3 - bug fixing)
2012-08-17 18:18fmanResolutionopen => fixed
2012-09-01 19:58fmanNote Added: 0017277
2012-09-01 19:58fmanStatusresolved => closed

Notes
(0016491)
fman   
2012-04-03 18:25   
unfortunatelly details provided do not allow any kind of action.
you need to provide:

1. sample database
2. detailed steps to reproduce
3. how many test project do you have ?
4. how many test suites and test cases ?
5. how many test plans ?
6. how many builds ?
7. how many executions ?
(0016494)
awiechert   
2012-04-04 08:41   
1. What kind of sample database. I cannot give it to you cause there are company related parts in there. I'll need to ask.
2. Choose a large project like TDIQA. Goto: Test Report: General Test Plan Metrics. Here you got the whit page with the failure described below.

testprojects = 8
testsuites = 718
testplans = 60
builds = 591
executions = 89463
testcase = There is no table in database
(0016497)
fman   
2012-04-04 10:26   
>> Choose a large project like TDIQA
without the db this is meaninless for us.
In addition large means nothing.

>> testcase = There is no table in database
then this means you do not have testcases defined ?
how do you have executions then ?

please remember that we are willing to help, when you are also willing to do this.
I do not want select count(0) for each table.
We need numbers you get from user interface

unfortunatelly if you are not able to provide this info we will be forced to close issue.
(0016499)
awiechert   
2012-04-04 10:32   
Guys what kind of information do you need and where do I find them?


I did not use that tool. I'm one of the admins here.

So please explain to me what you want/need. By the way if the database contains company secrets I can't give it out. I hope you understand.
(0016500)
techieguy   
2012-04-04 10:36   
i faced a similar issue which was related to the api i used for the bug tracking client ( jira ) which was taking too long and the page expires. the best is you can go and search the apache.log or the php.log which can tell u the exact error. i refined the code to make less calls to the jira and it solved my problem also i set a parameter to increase the wait time for a web request for a response in apache. dont remember the exact param.
(0016501)
awiechert   
2012-04-04 11:42   
@techieguy.

Yes I gues this is a related issue. We use jira as bugtracker. When I set $g_interface_bugs='JIRA'; to $g_interface_bugs='NO'; everthing is fine. So it seams to be a timeout while connecting the jira database?
(0016502)
techieguy   
2012-04-04 11:59   
2 options:
1: use the db interface for Jira, if you have it hosted locally or on same server to decrease the response time instead of soap interface (jirasoap) soap is slower as compared to direct db access.
2: modify the code to reduce the calls to Jira. The current code makes repetitive call for bug info to jira so I took help from one of the developers - Amit Khullar who wrote the jirasoap interface to sort this out. Check with him once.
(0016503)
awiechert   
2012-04-04 12:36   
Hi,

we do not use the soap inteface. We use the db interface. What do you mean with step 2. ?
(0016504)
techieguy   
2012-04-04 14:20   
(edited on: 2012-04-04 14:23)
You can ignore the 2nd option if u don't use the soap Api , also the db access should be faster as compared to soap and u shouldn't be having delays check the logs for more clarity

(0016505)
awiechert   
2012-04-04 14:38   
Hm, not clear to my what to do know.
(0016510)
fman   
2012-04-04 17:11   
>> 2: modify the code to reduce the calls to Jira. The current code makes >>repetitive call for bug info to jira so I took help from one of the developers >>- Amit Khullar who wrote the jirasoap interface to sort this out. Check with >>him once.

is suggesting a change on TestLink code.

On 1.9.4 we will try to deprecate all integration not done via API.
(0016514)
awiechert   
2012-04-04 17:47   
Ok I understand now. But how can we solve this issue? It's a real blocker for us.
(0016516)
fman   
2012-04-04 17:56   
I will try to give a look in order to provide a fix.
If this will solve your issue, we will apreciate if possible some sort of contribution to the project.
is fine for you ?
(0016518)
techieguy   
2012-04-04 18:06   
>> On 1.9.4 we will try to deprecate all integration not done via API.

What do you mean by the above statement ?

Will you remove support for direct db access of the JIRA Client / Bug trackers and let all be handled using API's ?
(0016519)
fman   
2012-04-04 18:22   
(edited on: 2012-04-04 18:26)
>> On 1.9.4 we will try to deprecate all integration not done via API.
This means clearly that we plan to remove Database access as method to integrate with bts.

@techguy
please do not start a discussion about this design choice in this issue, is not the right place

(0016526)
awiechert   
2012-04-05 11:39   
Yes its fine for us to do a contribution.
(0016529)
fman   
2012-04-05 14:12   
@awierchert
I've started work yesterday, I think I will be able to send you a patch next week
(0016535)
fman   
2012-04-07 10:21   
@awierchert
attached fix.
Changes to results class constructor interface, in order to indicate if access to BUG TRACKING SYSTEM has to be used or not on report requested.

In 1.9.4 we will need to create new methods on interface with BTS, to decrease number of call needed when BTS info is needed on reports.

Today implementation is poor because do lot of calls in loop.

Please let me know if this solves your problems

regards

Francisco
(0016554)
awiechert   
2012-04-10 08:28   
Hi Francisco,

sorry to tell you it hasn't changed the result. It took now some time more then before but the log looks the same:

[Tue Apr 10 03:33:20 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:21 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:23 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:27 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:28 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:29 2012] [error] Exception KeyError: KeyError(140699642537952,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Tue Apr 10 03:33:43 2012] [error] [client 87.193.254.34] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 77 bytes) in /var/www/domains/managed.contegix.com/talend02/htdocs/testlink/lib/functions/object.class.php on line 263, referer: http://testlink.talendforge.org/lib/results/resultsNavigator.php [^]
(0016555)
fman   
2012-04-10 10:16   
@awiechert
you provide error log on /usr/lib64/python2.6/threading.pyc, do not understand why, because we do not use python.

What I've done theorically is the same that you did (but in a controlled way) to DO NOT USE BTS integration while creating 'General Test Plan Metrics'

All I can do is add a log to understand if change is really working.
As stated if you can provide your PRODUCTION DB or you can provide me some access to your site (via VPN) I will be able to help more if you install a sandbox.

regards
(0016577)
fman   
2012-04-12 21:36   
(edited on: 2012-04-12 21:39)
@awiechert:
I've edited the SQL dump, to removed all records regarding events and transaction tables, then imported.

I need more help from you.
my HW is a Z60M laptop, fedora 14, running lammp 1.7.7, 2 GB RAM, 60 GB disk
1. configured my TestLink with BTS enabled (not jira but Mantis, just to see performance effects due to BTS integration).
 
2. login as infra
3. Test Project TDIQA (1632 test cases on test spec tree)
4. Click on Test Reports and Metrics
5. Test Plan Patch 4.2.3
6. Click on General Test Plan Metrics
7. got results after: Processing time (seconds) 2.98
  79 calls to buildBugString()

8. images sent by email

could you said to me is this test is good enough to reflect your situation ?

Please provide test project and test plans to use.

(0016578)
awiechert   
2012-04-13 07:22   
Hi fman,

any news?
(0016580)
fman   
2012-04-13 08:06   
give a look to note 16577
(0016591)
awiechert   
2012-04-16 14:17   
Hi fman,
I've tried your hint with the database tables events and transactions. Doesn't help to solve this issue.
(0016592)
fman   
2012-04-16 14:34   
ok,anyway this clean up is not a bad thing
 but do you have some feedback from your QA team ?

if you want wa can try to chat via skype later (my time zone is ROME)
(0016593)
awiechert   
2012-04-16 14:36   
ok good idea via skype. I'll try to get to QA team on baord as well.
(0016594)
fman   
2012-04-16 15:45   
ok textual chat => my id is fmancardi
(0016596)
fman   
2012-04-17 05:48   
issue found on results.class.php, when calling get_linked_tcversions().
Work in progress
(0016602)
fman   
2012-04-20 22:23   
(edited on: 2012-04-20 22:28)
results class - method createMapOfLastResult() needs refactoring => uses too much memory

memory usage traced to wrong implementation too many queries due to
getPriority()

(0016951)
techieguy   
2012-07-04 09:39   
Hey fman,

Was this issue fixed? I am more concerned with the performance / memory leakage fixed code. In case it has been done please can u mention the checkin here.

Thanks
TG
(0016952)
fman   
2012-07-04 12:12   
Need to do some checks, not sure will be able to provide all commits.
We are working to provide fix on 1.9.4 (we except release it at end of this month on beta).

what you can do is get tarball with 1.9.4 from gitorious, setup a test box and do some tests, this info will be very useful for us.
Unfortunatelly we have had no more feeback from reporter

(0017277)
fman   
2012-09-01 19:58   
1.9.4 released