Cognos server freeze while bursting 800 reports

[abhishek dharga]

I was doing some analysis on cognos server machine and it seems that Garbage collector for "BIBusTKServerMain.exe" in not working properly. I submitted one batch Job yesterday , which was failed after 13 hours. Since then we didn't run any report and batch service is idle for last 12 hours. However its still showing high memory utilization. Please find below details about various steps I carried out and my observations,

Steps :
Cognos server was started on "04/01/2009 6:35:13 AM".
Report batch Job was submitted on "04/01/2009 6:49:30 AM".
Report Job failed after 12 hours i.e. "04/01/2009 6:52:22 PM"
No new job was submitted after that.

Observation :
One "BIBusTKServerMain.exe" service was created when Cognos server was started i.e. at "04/01/2009 6:35:13 AM".
Second "BIBusTKServerMain.exe" service was created when Report Job was started i.e. at "04/01/2009 6:49:30 AM".
Initially both the report services were showing minimum CPU and memory utilization.
Memory utilization for second process started growing as Cognos started report output generation for batch Job.
Memory utilization for second service reached to a peak value and report started failing.
We tried to run a single report but even that stopped working.
Second "BIBusTKServerMain.exe" is still showing heavy memory utilization while no one is using Dev-05 server.

Screen shot of process tree of cognos server machine which I captured at ''04/02/2009 5:05:01 AM'' is attached.
(The'BIBusTKServerMain.exe' process highlighted in blue, which is consuming heavy memory utilization )

Per our understanding second "BIBusTKServerMain.exe" should release VM once report execution is completed but it seems that its not able to release this extra memory. As a result we are getting
"RQP-DEF-0177 An error occurred while performing operation 'sqlOpenResult' status='-28'. UDA-SQL-0114 The cursor supplied to the operation "sqlOpenResult" is inactive. UDA-SQL-0107 A general exception has occurred during the operation "AOpen"
error when we are trying to invoke new process (report).

[tel03]

Hello Abhishek,

Please let me the solution for the below issuse ....right now we are facing the same iisssuse in PRD which is very urgent.

Please let me know what are changes that i need to do.

I was doing some analysis on cognos server machine and it seems that Garbage collector for "BIBusTKServerMain.exe" in not working properly. I submitted one batch Job yesterday , which was failed after 13 hours. Since then we didn't run any report and batch service is idle for last 12 hours. However its still showing high memory utilization. Please find below details about various steps I carried out and my observations,

Steps :
Cognos server was started on "04/01/2009 6:35:13 AM".
Report batch Job was submitted on "04/01/2009 6:49:30 AM".
Report Job failed after 12 hours i.e. "04/01/2009 6:52:22 PM"
No new job was submitted after that.

Observation :
One "BIBusTKServerMain.exe" service was created when Cognos server was started i.e. at "04/01/2009 6:35:13 AM".
Second "BIBusTKServerMain.exe" service was created when Report Job was started i.e. at "04/01/2009 6:49:30 AM".
Initially both the report services were showing minimum CPU and memory utilization.
Memory utilization for second process started growing as Cognos started report output generation for batch Job.
Memory utilization for second service reached to a peak value and report started failing.
We tried to run a single report but even that stopped working.
Second "BIBusTKServerMain.exe" is still showing heavy memory utilization while no one is using Dev-05 server.

Screen shot of process tree of cognos server machine which I captured at ''04/02/2009 5:05:01 AM'' is attached.
(The'BIBusTKServerMain.exe' process highlighted in blue, which is consuming heavy memory utilization )

Per our understanding second "BIBusTKServerMain.exe" should release VM once report execution is completed but it seems that its not able to release this extra memory. As a result we are getting
"RQP-DEF-0177 An error occurred while performing operation 'sqlOpenResult' status='-28'. UDA-SQL-0114 The cursor supplied to the operation "sqlOpenResult" is inactive. UDA-SQL-0107 A general exception has occurred during the operation "AOpen"
error when we are trying to invoke new process (report).

[abhishek dharga]

What is the Cognos suite version you guys are using.
One of the solution is increase the memory RAM on servers.
secondly is upgrade to Cognos 8.4.(upgrade is the permanent fix).

Work around:
Split the bursting of report in Jobs.
I mean suppose u wanna burst the report for 800 inputs or burst key.
Build the 2 report views each one has the 400 prompt values.
Then make a JOB and put those report view in that and set the run option as sequential basis.
Then run the JOB so once your first view complete the execution of 400 reports then second view will start running.

Let me know if u need more clarification

[tel03]

Thanks for replay

we use cognos 8.4.1

its not like excatly 800 reports we have count  it can be 1500 0r 1000 any number so i am not sure how this thing works.

We have a event for this so initially the event check the condition once its met then it runs the store procedure and update the status i.e TASK1

Task2: run the report which is both XML and PDF version  saving to shared network location with brusting check mark.

This event is scheduled every night.

The other problem is once this event fails all the other jobs are also failing throwing the same.

Please suggest me more on this

What is the Cognos suite version you guys are using.
One of the solution is increase the memory RAM on servers.
secondly is upgrade to Cognos 8.4.(upgrade is the permanent fix).

Work around:
Split the bursting of report in Jobs.
I mean suppose u wanna burst the report for 800 inputs or burst key.
Build the 2 report views each one has the 400 prompt values.
Then make a JOB and put those report view in that and set the run option as sequential basis.
Then run the JOB so once your first view complete the execution of 400 reports then second view will start running.

Let me know if u need more clarification

[abhishek dharga]

Some quick questions for you...

1> How you are saying it could be 15000 or 1000?
I mean how you are supplying the prompt value to decide burst 1000 or 15000 reports.

2>Your Task2 is trigger by the Task1 ,what is the triggering mechanism you are using to start bursting of report.

3>Did u try to burst the report less than 1000 report or some more lesser?Just check for how many reports cognos is successfully bursting on server.If you have another server too just check the numbers there too.