script to stop services, reboot servers, and start services in appropriate order

[jeffowentn]

Does anyone have knowledge of how to automate the following processes in a multiple server environment?

1.  Stop all services in the appropriate order
2.  Reboot all servers
3.  Start all services in the appropriate order

I currently have all servers scheduled to reboot at staggered times, but this has the Content Manager server stopping services first and then starting them first before any other services have been stopped, etc.

Windows 2003
SQL Server 2005
Cognos Planning 8.4.1

[ericlfg]

Hey Jeff,

I did a quick google search for this.  Take a look at:
http://stackoverflow.com/questions/1405372/stopping-starting-a-remote-windows-service-and-waiting-for-it-to-open-close

This basically allows you to use the sc.exe (services console) to trigger a restart of a service.  To simplify your process and eliminate one extra step, you could set all cognos services to manual, and script a restart using the shutdown -r -m server01 (where -r is restart, -m is machine, server01 can be replaced by IP or Machine name).  Since the service is set to manual, use a SC.exe command to restart the services in the correct order on each machine.  Example: sc \\server01 start "IBM Cognos 8 Planning 8.4"

You'll probably want to keep this script on a file server that won't be restarted very often.

[jeffowentn]

Thanks, Eric!  Let me do some testing with this in DEV, and I'll get back with you.

[ericlfg]

Sounds good!

I'll be interested to see how you make out.  I like learning about system administration topics in combination with Cognos.  :)

[jive]

Hi Jeff,

In the project I work now we shutdown our windows server rarely. So I have to find a way to reboot the cognos part . The easiest way we find it's script :

C:\WINDOWS\system32\sc.exe Query "IBM Cognos 8" | Find /I "Stopped" > nul
net START "IBM Cognos 8"

With that we closed Cognos ans reopen it each morning. The "IBM Cognos 8" is the service name If it's running we close it and reopen it. That way to do thing have nothing to do with the reboot of the machine. When the windows server is shutdown cognos restart automatically.
I hope this help.

Regards Jacques

[jeffowentn]

Thank you, Jacques.

Is your environment a multiple server environment or does Cognos reside on just the one server?  The main issue I'm running into is the fact that we have multiple servers that need to have the services stopped in one direction (ending with Content Manager server) and started in the opposite direction (starting with Content Manager server).

Are you doing that with the command you just provided?

[corey_hallisey]

I have some VB scripts that talk to the Cognos software to stop/start services as well. 
Our server reboots are separate from the service stop/starts though.

I've attached a ZIP file here with our VB scripts and BAT files that trigger the VB scripts.  These scripts also email us when they're done.

It was recommended to use by Cognos Support to introduce a nightly server reboot between the stop and the start to stabilize our systems.

We have a 5-server farm. 1 master, 4 job serers.  48 CPUs total.

[jeffowentn]

That sounds great, Corey.  We are up to 4 servers, now - 1 app and 3 job, although I will most likely move one of the jobs back to an app server - this excludes our BI/Content Manager server and our Gateway server - when it is all said and done, we should have something like 9 servers in PROD instance.

What version of Cognos are you running?  I only ask, because I've received feedback that suggests our weekly reboot is too frequent and that we should consider a monthly reboot.

I'm also hopeful that going with the scripts will address our issue of the services sometimes not starting properly after a reboot.  ???

[jeffowentn]

Corey - thanks for the all the great files.  I'm working through making the changes for my environment, and I believe this will be my process, for simplicity sake. 

I will change all the services to manual on all of my servers, in general.

1.  Reboot all servers, simultaneously
2.  Stop all services in the correct order
3.  Start all services in the correct order

One thing I'm not clear on, and this is just ignorance on my part, but how do I get the VBS script to run?  When I run a .bat file to call the vbs file, I get the following error message:

Microsoft VBScript runtime error: Path not found

Any suggestions on what I need to do differently?  I'm not sure where I will locate these .bat and .vbs files, yet, so I'm also trying to write everything as dynamically as possible.

[jeffowentn]

Thought I would update this for anyone's benefit - this works without any add'l tools, etc. (still has room for improvement, though):

[had to attach due to character constraint]

[jeffowentn]

Here are some revised versions of the scripts to address issues of a service getting "hung" during the starting or stopping of the service (prevented the script from completing in the earlier version).

I also created several different variations:

RecycleServices_SAMPLE.bat - simply recycles all Cognos services and web services in the environment (12 servers) in the appropriate order

RebootAndRecycleServices_SAMPLE.bat - same as RecycleServices_SAMPLE.bat, but it also reboots the servers in the middle of the process

StopServices_SAMPLE.bat - stops all of the Cognos services and web services in the environment in the appropriate order

StartServices_SAMPLE.bat - starts all of the Cognos services and web services in the environment in the appropriate order

RebootServersOnly_SAMPLE.bat - simply reboots all of the servers in the environment


The first two scripts are scheduled to run weekly (reboot on the weekend and recycle in the middle of the week).  The other three scripts are to simplify/expedite maintenance activities or break/fix activities.

Hope this helps someone...

[jeffowentn]

 attachment related to previous post

[jeffowentn]

 attachment related to previous post

[jeffowentn]

 attachment related to previous post

[jeffowentn]

 attachment related to previous post

[jagadeeshgelivi]

Hi jeffowentn, The links which you have attached are not working now. Can you please provide me the links again.

Thanks in Advance

[dougp]

Is this the same question?
https://www.cognoise.com/index.php/topic,37791.0/topicseen.html

This seems like an odd question.  Is Cognos really that fragile?  I would expect you can start the services in any order and they would wait and check again if dependencies are not yet running.

Telling us what version of Cognos and what operating system you are using would be helpful.

I see batch calls in this thread, so I'll assume you are running Cognos on Windows.  I recommend using PowerShell.

Rebooting the machine will stop the services.  The order in which you stop services should not matter.  What may matter is what order services start (and complete starting).

To start the services in a specific order, you'll want to determine the proper order and build delays and waits into your process.  This script starts the IBM Cognos service, waits for it to start, then returns the new entries in cognosserver.log.

Code Select Expand


    $servername = "MyCognosServer"
    $servicename = "IBM Cognos"
    $coglog = "\\$servername\e`$\Program Files\ibm\cognos\analytics\logs\cognosserver.log"
   
    $linesbefore = (Get-Content $coglog | Measure-Object –Line).Lines
    $cogsvc = Get-Service -ComputerName $servername -Name $servicename
    $cogsvc.Stop()
    "Waiting for the $servicename service on $servername to stop."
    $cogsvc.WaitForStatus("Stopped")
    "The $servicename service on $servername is stopped."
    $linesafter = (Get-Content $coglog | Measure-Object –Line).Lines
    $len = $linesafter = $linesbefore
    Get-Content -Path $coglog -Tail $len

I run this from a different computer as an account that has admin privileges on the server.  You can see my Cognos install is on the E: drive and I'm using the E$ admin share.  I left that in to demonstrate escaping the $.  I could very well have simply shared the logs folder.

And I'm running Cognos Analytics, but I would expect this process for Cognos Planning would be similar.