Confused about Cognos on IBM Websphere Application Server

[pargol4]

We have decided to use IBM Websphere Application Server for Cognos 10.2.2 environments. I am not the admin, but I have questions about exactly what is needed for the best experience.

Do we still need to have a webserver installed and configured, or does 'build the application files' in Cognos Configuration do all of it for you? I have tried to read about te Cognos Servlet Gateway, in which the documentation just tells you if you use Cognos Servlet Gateway instead of a webserver, you just type in a different URL (hostname:80/ServletGateway.

What should we have in the Gateway URI? I've only ever seen it as cognos.cgi, cognosisapi.dll, and apache_mods.

[MFGF]

We have decided to use IBM Websphere Application Server for Cognos 10.2.2 environments. I am not the admin, but I have questions about exactly what is needed for the best experience.

Do we still need to have a webserver installed and configured, or does 'build the application files' in Cognos Configuration do all of it for you? I have tried to read about te Cognos Servlet Gateway, in which the documentation just tells you if you use Cognos Servlet Gateway instead of a webserver, you just type in a different URL (hostname:80/ServletGateway.

What should we have in the Gateway URI? I've only ever seen it as cognos.cgi, cognosisapi.dll, and apache_mods.

Hi,

Cognos 10.2.1 and all prior versions came with a bundled application server called Tomcat. You had the choice of either sticking with this, or using a different application server such as WAS - but it was the full-blown WAS so was quite demanding in terms of system resources required.
Cognos 10.2.2 onwards comes with a bundled application server called Websphere Liberty. It's a "lite" version of WAS - cut down to the bare minimum to support Cognos processing. You have the choice of either sticking with this or using a different application server - such as the full-functionality WAS.

Using either Tomcat, Websphere Liberty or a third-party application server such as WAS, you then have the choice of either using a web server (IIS/Apache etc) or using the servlet gateway. When using the servlet gateway you connect directly to the dispatcher URI (eg http://hostname:9300/p2pd/servlet/dispatch) whereas when using a web server you connect to the gateway component (eg http://webserverhostname:80/ibmcognos/cgi-bin/cognos.cgi or http://webserverhostname:80/ibmcognos/cgi-bin/cognosisapi.dll etc)

There are advantages and disadvantages to each. When using a web server, you have a more challenging configuration to handle, since the web server needs to be configured for use with Cognos BI (which is a disadvantage) but you gain the ability to browse images using WEBDAV, you can use Theme Designer and DQA easily etc. When using the servlet gateway there is less to configure, but you then have more limited functionality eg no WEBDAV/Theme Designer etc.

Cheers!

MF.

[pargol4]

Thanks for that information. Time to revisit though.

We are looking at modifying our multi-server environment. I would like to understand what pieces play what parts in all of this.

Currently (in Prod) we have:
Gateway components installed on Server 1.
App Tier components installed on Server 2.
IBM Websphere 8.5 installed on Server 2.
Content Manager components installed on Server 3.
Content Manager database on Server 3 (SQL Server).

This is deployed to Websphere. I do not know anything about Websphere or deploying, or what is involved. I do know the basics of multiserver environment, including Gateway, App Tier and CM Tier should all be on separate physical servers, as should the CM Database and Reporting Database(s).

I'm trying to figure out a few things here.
1. Do any of the [Cognos] Gateway, App, and/or CM servers still do anything, once Cognos has been deployed to WAS? Can they be shut down?
2. I know we want to separate CM server from the CM database server, does CM get deployed to WAS as well, through Cognos Configuration? Or does deploying the 'App Tier' take care of everything?
3. If the Application server is only used for configuration, is there any need to separate out Gateway, App Tier, and CM components?
4. Should all three components be installed on one server, then deployed to WAS?
5. Should we still separate Gatweay components from the rest, but have App Tier and CM on the same server?

I hope I've provided enough information. And I appreciate any help with this matter.

[smiley]

You are really opening up Pandora's box here, by moving the java components into a system you are not comfortable with.
The BI engine are java servlets that run on ANY supported java server. You get one that works perfectly fine, free or charge with the product.

The java servlets communicate with additional parts of the installed BI software.

So on server 2, you install WAS, install BI app tier, configure it, and export the deployment from cognos configuration into WAS.
The BI service itself never get´s started.

On server 3, you install WAS, install BI CM tier, configure it, and export the deployment from cognos configuration into WAS.
The BI service itself never get´s started.

Look at the WAS licensing you are consuming here, just to replace the free tomcat or websphere liberty.

[Marva55]

How can I detect if our cognos 10.2.2 servers are using Websphere Liberty or the full blown Websphere Application Server (WAS)?

And how to monitor and configure these products for getting performance in Cognos?

[smiley]

If you have a service (assuming windows here) running called "ibm Cognos", then you are using the tomcat or websphere libery.
If you have WAS installed on the local server, you should be able to recognize their services/processes.

Cognos BI capacity settings are done in the webbased administrative portal. (concurrent report requests, memory management etc)
Cognos BI relies on the underlying OS to provide cpu cycles and memory blocks at request, so a correct server sizing helps. 

[Marva55]

OK then we are using WebSphere Liberty.

By "webbased administrative portal" you probable mean the "Cognos Administration" tool right?

I would like to change memory settings which are described here:
https://www.ibm.com/support/knowledgecenter/SSEP7J_10.2.2/com.ibm.swg.ba.cognos.crn_arch.10.2.2.doc/c_adgmemorystngs.html

We are having performance issues in a SDK application as described in this topic:
http://www.cognoise.com/index.php/topic,31412.0.html

[smiley]

The memory setting you are referring to is in Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos.
The default value is 768.
The advised value for production servers as from 10.2 is a minimum of 4096 (4 gig)

The memory has to be physical present though.
Rule of thumb is:

Total OS memory minus:
- 2 gig for kernel mode memory space (ie used by the OS itself)
- 4 gig for the internal Cognos logic (the value you want to set)
- x gig for the DQM java stack, if you use 64 bits reporting.
- x gig for the bibusses, if you also use 32 bits reporting.

So in theory, if you put the internal java to 4 gig, give the DQM stack another 4 gig, and have 2 bibibusses configured, the maximum memory consumption for all processes would be:

2 (OS) + 4 (internal java) + 4 (DQM) + 2 x 2 = 4 (Bibus) = 14 gig in total.

[Marva55]

The memory setting you are referring to is in Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos.
The default value is 768.
The advised value for production servers as from 10.2 is a minimum of 4096 (4 gig)

The memory has to be physical present though.
Rule of thumb is:

Total OS memory minus:
- 2 gig for kernel mode memory space (ie used by the OS itself)
- 4 gig for the internal Cognos logic (the value you want to set)
- x gig for the DQM java stack, if you use 64 bits reporting.
- x gig for the bibusses, if you also use 32 bits reporting.

So in theory, if you put the internal java to 4 gig, give the DQM stack another 4 gig, and have 2 bibibusses configured, the maximum memory consumption for all processes would be:

2 (OS) + 4 (internal java) + 4 (DQM) + 2 x 2 = 4 (Bibus) = 14 gig in total.

Kan you find me a source for your statement that the advised value for production servers as from 10.2 is a minimum of 4096 (4 gig)?

I can only find this, which says 1536MB i.e not 4096MB...

Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty has a default setting of 768MB. If memory is available on the server, this allotment should be increased to 1152 or 1536MB.

The 1536MB recommendation is for the maximum memory for Websphere Liberty Profile in Cognos Configuration. The IBM recommendation of 4GB is for the JVM heap size for the DQM service.

Source: http://www.senturus.com/blog/senturus-cognos-10-2-2-performance-tuning-webinar-summary/

I'm grateful for you help! [emoji846]

[smiley]

The 4 gig minimum for production servers is a best practice used by IBM Cognos Consulting services.
I can´t give you a source, as it´s not public information.
I don´t know why the Senturus article is hanging onto 1536 max. That is a limitation up to BI 10.1.1, which was 32 bits.

The DQM limitation is not engraved in stone. 4 gig startup, and 8 gig max is a fine starting point.
If you capacity configuration on threading fits in the 8 gig, then life is good.
If you start seeing memory errors in your logging, then either up the max memory, or discuss the problem with the developer, and tell him to stop building intense memory consuming reports.   

[Marva55]

The 4 gig minimum for production servers is a best practice used by IBM Cognos Consulting services.
I can´t give you a source, as it´s not public information.
I don´t know why the Senturus article is hanging onto 1536 max. That is a limitation up to BI 10.1.1, which was 32 bits.

The DQM limitation is not engraved in stone. 4 gig startup, and 8 gig max is a fine starting point.
If you capacity configuration on threading fits in the 8 gig, then life is good.
If you start seeing memory errors in your logging, then either up the max memory, or discuss the problem with the developer, and tell him to stop building intense memory consuming reports.

OK thanks for clarifying it's not public information.

I assume these configuration settings in Cognos Administration is corresponding to the number of bibus processes (which takes 2 GB each when running):
Maximum number of processes for the report service during non-peak period
Maximum number of processes for the report service during peak period

Theses are my setup, the dispatcher servers are running two installations of Cognos simultaneously:
Content Manager (Primary)
32 Core
98 GB Memory
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536

Content Manager (Secondary) (Virtual Machine)
2 Core
8 GB Memory
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536

Dispatcher Server 1 (Dispatcher1,Dispatcher2)
32 Core
98 GB Memory
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536

Dispatcher Server 2 (Dispatcher3,Dispatcher4)
32 Core
98 GB Memory
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536

Dispatcher Server 3 (Dispatcher5,Dispatcher6)
32 Core
98 GB Memory
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536
Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos\Maximum Memory for Websphere Liberty in MB: 1536

General report service configuration settings:
Maximum number of processes for the report service during non-peak period: 6
Maximum number of processes for the report service during peak period: 6
Number of high affinity connections for the report service during non-peak period: 2
Number of low affinity connections for the report service during non-peak period: 8
Number of high affinity connections for the batch report service during peak period: 2
Number of low affinity connections for the batch report service during peak period: 8
Queue time limit of the report service (seconds): 360

[smiley]

The bibusses can actually take 3 gig each, as they are 32 bits processes running on a 64 bits server.

With your current config, each dispatcher instance has for non-peak period:
DQM = 6 bibusses x 8 low affinity = 48 slots.
CQM = 6 bibusses x 8 low affinity + 2 high affinity = 60 slots.

Interesting to read your primary CM has 98 gig memory.
With the Liberty set to 1.5 gig max, that leaves 94.5 gig left to idle on the server.
Or was it a typo, and it´s the same as for the standby CM?

[Marva55]

I wasn't aware that "Maximum Memory for Websphere Liberty in MB" could be set to higher than 1536 MB.

With my Content Manager having 98GB RAM I currently see this in task manager:
Total 98269
Cached 24206
Available 88998
Free 64934

What would you suggest "Maximum Memory for Websphere Liberty in MB" should be set to in my situation?

How does this setting affect the Cognos server environment and more specifically the Content Manager?

[Marva55]

Would 16 384 MB be a good setting to start with? (4 x 4096)

[smiley]

What would you suggest "Maximum Memory for Websphere Liberty in MB" should be set to in my situation?
How does this setting affect the Cognos server environment and more specifically the Content Manager?

I would start with 4096.
You can actually look realtime into the stack memory usage and judge for yourself (CM only).
Open C:\Program Files\IBM\cognos\c10_64\bin64\utilities\cm\CMTools\cmtools.exe
After logon, in the menu click Window, open perspective, cmtools
Then click Window, show view, cmtop.
Select the system tab, then click on the refresh button.
You are now looking directly into the JVM memory usage. (does not refresh automatically. Need to click refresh button again for that)

The set memory in Cognos Configuration only affects the total capacity of the Content Manager (ie concurrent users) 

[smiley]

Would 16 384 MB be a good setting to start with? (4 x 4096)

16 gig for the CM?
In he 32 bits era they did some lab testing, and the 1536 value could store about 1700 concurrent users.
With the logic now being 64 bits, there is roughly a 30% overhead on memory allocation, versus 32 bits.

So very roughly, 16 gig could handle around 13.000 concurrent users.
With the calculation back to named users, as mentioned in the Senturus article (10% connected, 1% active), that would lead to support for a 1.3 million named user system......

As you see, I would suggest to set the CM to 4 gig, and you should be more then covered.  ;D

[Marva55]

16 gig for the CM?
In he 32 bits era they did some lab testing, and the 1536 value could store about 1700 concurrent users.
With the logic now being 64 bits, there is roughly a 30% overhead on memory allocation, versus 32 bits.

So very roughly, 16 gig could handle around 13.000 concurrent users.
With the calculation back to named users, as mentioned in the Senturus article (10% connected, 1% active), that would lead to support for a 1.3 million named user system......

As you see, I would suggest to set the CM to 4 gig, and you should be more then covered.  ;D

Haha [emoji28] ok thank you very much, I will try 4096MB.

Hopefully this will increase the performance in overall. Since we upgrade to 10.2.2 everything got a little slower.

Also I have a SDK application where certain operations like logon, logoff, query for report/report parameters etc. takes up to 2 minutes when running 50 concurrent reports (as interactively user activity).

[Marva55]

The memory setting you are referring to is in Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos.
The default value is 768.
The advised value for production servers as from 10.2 is a minimum of 4096 (4 gig)

The memory has to be physical present though.
Rule of thumb is:

Total OS memory minus:
- 2 gig for kernel mode memory space (ie used by the OS itself)
- 4 gig for the internal Cognos logic (the value you want to set)
- x gig for the DQM java stack, if you use 64 bits reporting.
- x gig for the bibusses, if you also use 32 bits reporting.

So in theory, if you put the internal java to 4 gig, give the DQM stack another 4 gig, and have 2 bibibusses configured, the maximum memory consumption for all processes would be:

2 (OS) + 4 (internal java) + 4 (DQM) + 2 x 2 = 4 (Bibus) = 14 gig in total.

Can you help me with the DQM java stack sizing?

Total OS memory minus:
- 2 GB for kernel mode memory space (ie used by the OS itself) <------ Probably varies a bit depending on OS version
- 4 GB for the internal Cognos logic (the value you want to set) <------ Configured in the Cognos Configuration\Environment\IBM Cognos Services\IBM Cognos section and parameter "Maximum Memory for Websphere Liberty in MB"
- x GB for the DQM java stack, if you use 64 bits reporting. <------ Where is this setting configured and how does the X GB calculation work?
- 2 GB x per process for the bibusses, if you also use 32 bits reporting (actually 3GB if running on 64 bit OS). <------ Configured using "Maximum number of processes for the report service during non-peak period" & "Maximum number of processes for the report service during peak period" parameters in Cognos Administration

[MFGF]

Can you help me with the DQM java stack sizing?
- x GB for the DQM java stack, if you use 64 bits reporting. <------ Where is this setting configured and how does the X GB calculation work?

Hi,

The startup and maximum memory settings for this are in the properties of the Query Service in the Administration console.

Take a look at the below IBM technote:

http://www-01.ibm.com/support/docview.wss?uid=swg21587457

Cheers!

MF.

[smiley]

- x GB for the DQM java stack, if you use 64 bits reporting. <------ and how does the X GB calculation work?

Physical available memory minus the earlier mentioned memory for OS, tomcat/libery stack and CQM allocation = max you can use.
Utilizing that memory optimal, depends on your set capacity. (ie 60. See previous page).

In the end more important with DQM is your report quality testing.
In the old days when a bibus died due to a bad report, it took down a few extra healthy running reports, running in the same proccess.
Now in the DQM time, with EVEYTHING running in that stack will crash with a bad report.

In CQM your report was limited to 3 gig, due to 32 bits limitation. The developer had to work around that by adopting his report.
With DQM, that limitation is gone. That does not mean he can start building "bad" reports, requiring you to keep adding memory to the stack to compensate.

[smiley]

Kan you find me a source for your statement that the advised value for production servers as from 10.2 is a minimum of 4096 (4 gig)?

I can only find this, which says 1536MB i.e not 4096MB...

Found it:
http://www-
01.ibm.com/support/knowledgecenter/SSEP7J_10.2.1/com.ibm.swg.ba.cognos.inst_cr_winux.10.2.1.doc/c_installs_
verify_ram.html?lang=en

[etlingjm]

I would start with 4096.
You can actually look realtime into the stack memory usage and judge for yourself (CM only).
Open C:\Program Files\IBM\cognos\c10_64\bin64\utilities\cm\CMTools\cmtools.exe
After logon, in the menu click Window, open perspective, cmtools
Then click Window, show view, cmtop.
Select the system tab, then click on the refresh button.
You are now looking directly into the JVM memory usage. (does not refresh automatically. Need to click refresh button again for that)

The set memory in Cognos Configuration only affects the total capacity of the Content Manager (ie concurrent users)

I know I am late to the game on this, but there is a tradeoff with startup and shutdown times with higher settings - my experience.

That being said, I have 1124 and have considered increasing by 50-100% for better performance.