[Solved] Cognos 8.3 fails on startup

[young198]

My cognos 8.3 will not restart.  when i try to the service using the cognos configuration tool it fails to start successfully and stops the service. no changes where made to the server cognos runs on or the sql server.  I tried to hook it to a blank content store and it rebuilt the tables without a problem but then failed with the same error set i was getting before repointing. the username and password for the db are correct and the configuration tool tests the connection just fine.  it appears that something messing with the connection somewhere but could use help tracking it down


the following is what is in the cogserver.log from a fresh attempt to restart with a fresh content store db

xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:42.894   -5               main   LOGSV   643   1   server.Audit.IPF   StartService   LogService      Success      <parameters><item name="Port"><![CDATA[9362]]></item><item name="Mode"><![CDATA[UDP]]></item><item name="Secure"><![CDATA[FALSE]]></item></parameters>
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:44.816   -5   Initialization_SESS   Initialization_REQ         Thread-16   CM   891   1   Audit.cms.CM   StartService   ContentManagerService      Info      CM-SYS-2057 Creating content store tables (schema version 4.0066).
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:45.284   -5   Initialization_SESS   Initialization_REQ         Thread-16   CM   891   1   Audit.cms.CM   StartService         Warning      CM-CFG-5063 A Content Manager configuration error was detected while connecting to the content store.
CM-CFG-5073 An unexpected event occurred while loading object class definitions into the content store.
CM-CFG-5073 An unexpected event occurred while loading object class definitions into the content store.
DBComms.error reading input. Context:Read packet header, Unexpected end of stream, readBytes:-1, negative read result, PktNumber:0, ReadThisPacket:0, PktDataSize:4096
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:45.284   -5   Initialization_SESS   Initialization_REQ         Thread-16   CM   891   1   Audit.cms.CM   StartService   ContentManagerService      Failure      CM-SYS-5007 Content Manager build 8.3.891.0 failed to start! Review the Content Manager log files and then contact your system administrator or customer support.
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:45.894   -5   pogoStartup   na   na   0   Thread-16   DISP   728   1   Audit.Other.dispatcher.DISP.pogo   pogo   com.cognos.pogo.contentmanager.coordinator.ActiveCMControl      Failure   <messages><message><messageString>DPR-DPR-1035 Dispatcher detected an error.</messageString></message><message><messageString></messageString></message></messages>   DPR-CMI-4007 A request to perform an active content manager election failed.null
com.cognos.pogo.bibus.SoapFaultException
   at com.cognos.pogo.bibus.BIBusCommand.handleResponse(BIBusCommand.java:519)
   at com.cognos.pogo.bibus.BIBusCommand.execute(BIBusCommand.java:280)
   at com.cognos.pogo.contentmanager.StandbyContentManagerElectionCommand.executeIfLocal(StandbyContentManagerElectionCommand.java:58)
   at com.cognos.pogo.contentmanager.coordinator.ActiveCMControl.sendElectCommand(ActiveCMControl.java:227)
   at com.cognos.pogo.contentmanager.coordinator.ActiveCMControl.startTimer(ActiveCMControl.java:174)
   at com.cognos.pogo.contentmanager.coordinator.ActiveCMControl.startupFindCM(ActiveCMControl.java:137)
   at com.cognos.pogo.contentmanager.coordinator.RuntimeInfoPublishHandler.start(RuntimeInfoPublishHandler.java:152)
   at com.cognos.pogo.services.DefaultHandlerService.start(DefaultHandlerService.java:101)
   at com.cognos.pogo.services.DispatcherServices.startInititalServices(DispatcherServices.java:416)
   at com.cognos.pogo.transport.PogoServlet$PogoStartup.run(PogoServlet.java:654)
   at java.lang.Thread.run(Unknown Source)

xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:43:45.878   -5   pogoStartup   na   na   0   Thread-16   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.contentmanager.coordinator.CMBootstrap   getActiveContentManager   ContentManager      Failure   <messages><message><messageString>DPR-CMI-4006 Unable to determine the active Content Manager. Will retry periodically.</messageString></message></messages>   
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:12.972   -5               stopper-monitorService   MS   718   1   Audit.ms.MS   StopService   MonitorService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5               stopper-eventManagementService   EMS   718   1   Audit.ems.EMS   StopService   EventService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:12.941   -5   shutdown   na   na   0   main   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.services.DispatcherServices   StopService   CPS Producer Registration Service      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:12.941   -5   Termination_SESS   Termination_REQ         main   CM   891   1   Audit.cms.CM   StopService   ContentManagerService      Success      CM-SYS-5094 Content Manager stopped.; Build=8.3.891.0
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5               stopper-jobService   JS   652   1   Audit.js.JS   StopService   JobService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5   na   na         stopper-agentService   DISP   728   1   Audit.Other.dispatcher.DISP.pogo   pogo   com.cognos.pogo.services.DefaultHandlerService      Failure   <messages><message><messageString>DPR-DPR-1035 Dispatcher detected an error.</messageString></message><message><messageString></messageString></message></messages>   problem stopping handler agentServiceHandlernull
java.lang.NullPointerException
   at com.cognos.asv.service.ASVHandler.stop(ASVHandler.java:229)
   at com.cognos.pogo.services.DefaultHandlerService.stop(DefaultHandlerService.java:189)
   at com.cognos.pogo.services.DispatcherServices$ServiceStopper.run(DispatcherServices.java:360)

xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5   shutdown   na   na   0   main   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.services.DispatcherServices   StopService   SystemService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5   shutdown   na   na   0   main   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.services.DispatcherServices   StopService   ReportService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5   shutdown   na   na   0   main   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.services.DispatcherServices   StopService   BatchReportService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:12.988   -5               stopper-deliveryService   DLS   694   1   Audit.ds.DLS   StopService   DeliveryService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5   shutdown   na   na   0   main   DISP   728   1   Audit.Other.dispatcher.DISP.com.cognos.pogo.services.DispatcherServices   StopService   ReportDataService      Success      
xxx.xxx.xxx.xx:9300   1196   2015-03-10 18:44:13.066   -5               stopper-logService   LOGSV   643   1   server.Audit.IPF   StopService   LogService      Success      <parameters><item name="Port"><![CDATA[9362]]></item><item name="Mode"><![CDATA[UDP]]></item><item name="Secure"><![CDATA[FALSE]]></item></parameters>


the tomcat log looks like this

INFO  [main] org.apache.coyote.http11.Http11Protocol - Initializing Coyote HTTP/1.1 on port 9300: 10 Mar 2015 18:43:33,831
INFO  [main] org.apache.coyote.http11.Http11Protocol - Starting Coyote HTTP/1.1 on port 9300: 10 Mar 2015 18:43:43,284
ERROR [Thread-23] org.apache.tomcat.util.net.PoolTcpEndpoint - Caught exception trying to unlock accept.: 10 Mar 2015 18:44:13,831
java.net.ConnectException: Connection refused: connect
   at java.net.PlainSocketImpl.socketConnect(Native Method)
   at java.net.PlainSocketImpl.doConnect(Unknown Source)
   at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at java.net.Socket.<init>(Unknown Source)
   at java.net.Socket.<init>(Unknown Source)
   at org.apache.tomcat.util.net.PoolTcpEndpoint.closeServerSocket(PoolTcpEndpoint.java:326)
   at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:397)
   at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:529)
   at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
   at java.lang.Thread.run(Unknown Source)
ERROR [Thread-23] org.apache.tomcat.util.net.PoolTcpEndpoint - Caught exception trying to close socket.: 10 Mar 2015 18:44:13,831
java.lang.NullPointerException
   at org.apache.tomcat.util.net.PoolTcpEndpoint.closeServerSocket(PoolTcpEndpoint.java:338)
   at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:397)
   at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:529)
   at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
   at java.lang.Thread.run(Unknown Source)
WARN  [Thread-23] org.apache.tomcat.util.net.PoolTcpEndpoint - Reinitializing ServerSocket: 10 Mar 2015 18:44:13,831




Cause by a change made to the sql server network packet size. the size was changed from 4096 to 8192 and it was breaking the connection